How it works#
Models remember the start of recent requests. When the next request starts the same way — the same system prompt, the same documents, the same conversation so far — that part is read from cache: faster and cheaper. Cached input tokens cost 25% of the model's normal input price; output is priced as usual.
The cached input price of every model is in the catalog and on its page.
Which models#
- OpenAI, Google, DeepSeek, xAI and most others cache on their own — nothing to do.
- Claude caches only what is marked. For requests without marks flua sets an automatic cache breakpoint itself — in Chat Completions and in Messages. Writing to Claude's cache costs 125% of the input price and reading 25%: it pays off from the second request with the same start.
The cache lives for a few minutes after the last use and needs a minimum prefix length (usually 1–4 thousand tokens). Short requests are not cached.
Hitting the cache more often#
- Put what never changes first: the system prompt, instructions, documents, tool definitions.
- Put what changes last: the current question, the date and time, random ids. One changed character near the start throws away the cache for everything after it.
- Keep the set of tools and their order the same between requests.
How much came from cache#
The usage field of every response shows it, and so does the request log in the dashboard (the number in brackets next to input tokens).
"usage": {
"prompt_tokens": 5120,
"completion_tokens": 310,
"prompt_tokens_details": { "cached_tokens": 4864 }
}"usage": {
"input_tokens": 256,
"cache_read_input_tokens": 4864,
"cache_creation_input_tokens": 0,
"output_tokens": 310
}Your own marks for Claude#
In the Messages API you can place cache breakpoints yourself — up to four blocks with cache_control. flua then adds no automatic one.
client.messages.create(
model="claude-opus-5-5",
max_tokens=4096,
system=[{
"type": "text",
"text": LONG_INSTRUCTIONS,
"cache_control": {"type": "ephemeral"},
}],
messages=[{"role": "user", "content": question}],
)