To use fewer tokens, send less context, reuse what you must resend, and ask for shorter output: trim and summarise conversation history, cache the fixed part of your prompt, route simple tasks to a smaller model, and cap output length. Each lever has a cost in quality or effort, so measure before and after.
Every API response reports what the request consumed in a usage field: input tokens, output tokens, and on caching-capable APIs, cache reads and writes. Log those numbers before you change anything, or you will not know which lever helped. If you are unsure what a token is, how many tokens a word is explains it.
Where the tokens go
In most applications the input grows silently. Anthropic's context-window docs list everything that counts: the system prompt, every message you resend (including tool results, images and documents), tool definitions, and the output, including any thinking tokens. In a chat, the entire history is resent on every turn, so a 40-turn conversation pays for turn 1 forty times.
Output is smaller but usually priced higher. As an example, Anthropic lists Claude Opus 5.5 at $4 per million input tokens and $20 per million output tokens (as of October 2026). OpenAI's reasoning guide notes that reasoning tokens take up context space and are billed as output even though you do not see them.
1. Trim what you send
The cheapest token is the one you never send. Common waste:
- Whole files attached when one section is relevant.
- Verbose JSON with repeated key names on every record. A table in CSV form says the same thing with the keys written once; larely's JSON to CSV converter does this locally.
- HTML pages sent with markup, scripts and navigation instead of extracted text.
- Tool definitions for tools the current step cannot use.
- Old tool results in agent loops (file listings, search results) that are no longer needed. Anthropic offers tool result clearing for this.
Trade-off: remove something the model needed and quality drops. Trim noise and duplication first, then test whether shorter context still gives correct answers.
2. Summarise long chat history
Instead of resending 40 turns, replace the oldest ones with a summary and keep the last few verbatim. Anthropic calls this compaction and offers it server-side for Claude 4.6 and later models; you can also do it yourself with a cheap model call every N turns.
Trade-off: a summary loses detail. A specific number or decision from turn 3 may not survive. Keep facts the user is likely to refer back to (names, figures, decisions) as an explicit list in the summary.
3. Use prompt caching for the fixed part
If many requests start with the same long prefix (a system prompt, tool definitions, a reference document), prompt caching lets the provider reuse its processing of that prefix at a discount. How it works differs by vendor.
| Anthropic (Claude) | OpenAI | |
|---|---|---|
| How to turn it on | Mark a breakpoint with cache_control, or add one top-level cache_control field for automatic placement | On by default for supported models |
| What is cached | The prefix in the order tools, system, messages, up to the breakpoint | The repeated prefix of the prompt |
| Lifetime | 5 minutes by default, refreshed free on each hit; 1 hour optional | Varies by model; earlier models typically 5 to 10 minutes idle, up to 1 hour; GPT-5.6 and later at least 30 minutes by default |
| Price of a cache write | 1.25 × base input (5-minute) or 2 × (1-hour) | Reported separately; see current pricing |
| Price of a cache read | 0.1 × base input on most models, lower on some newer ones | Discounted up to 95%; 0.1 × on GPT-5.6 and later |
| Minimum prefix | 512 to 4,096 tokens depending on the model | 1,024 tokens on GPT-5.6 and later |
Figures are from each vendor's caching docs as of October 2026. Both vendors give the same structural advice: put stable content first and anything that changes per request (timestamps, user names, the question) after it. A single changed character early in the prompt breaks the cache for everything after it. Anthropic's docs add that a change at one level (for example tool definitions) invalidates that level and everything after it, and that prompts shorter than the minimum are processed without caching and without an error.
A worked example with Anthropic's multipliers: a 10,000-token system prompt sent 100 times in 5 minutes costs 1,000,000 input tokens uncached. With caching, it is one write at 1.25 × (12,500 token-equivalents) plus 99 reads at 0.1 × (99,000 token-equivalents), about 111,500 in total, roughly 89% less for that prefix.
Trade-off: caching rewards a stable prompt structure, which can constrain how you build prompts, and with Anthropic a write that is never read costs more than no caching at all.
4. Use a smaller model for simple tasks
Classification, extraction, reformatting and short summaries often work as well on a vendor's smallest model as on its largest, at a fraction of the price per token. A common pattern is routing: a small model handles routine requests, and only hard ones go to a large model.
Trade-off: smaller models make more mistakes on reasoning-heavy or ambiguous tasks. Run your own test set on both before switching; do not assume.
5. Limit output length
Set max_tokens (Anthropic) or max_output_tokens (OpenAI Responses) as a hard ceiling, and ask for brevity in the prompt ("Answer in at most 3 sentences"). The cap alone does not make answers shorter; it cuts them off. OpenAI's reasoning guide notes that max_output_tokens includes reasoning tokens, so a cap that is too tight can leave a reasoning model no room for the visible answer.
On models with adjustable reasoning effort (for example, Anthropic's effort parameter or OpenAI's reasoning.effort), a lower setting uses fewer thinking tokens on easy tasks.
Trade-off: truncated answers are useless. Ask for concise output, and set the cap high enough that a normal answer never hits it.
6. Ask for structured output
Prose answers often include preamble, restated questions and closing remarks. If your code parses the result, ask for JSON matching a schema. Anthropic's structured outputs (output_config.format with a JSON schema) constrain the response to the schema, which also removes retries caused by invalid JSON.
Trade-off: Anthropic's docs note that structured outputs add an injected prompt, so input tokens go up slightly, and that changing the schema invalidates the prompt cache. The saving is on output and retries, not input.
7. Batch work that can wait
If results are not needed immediately, batch APIs process requests asynchronously at a discount. Anthropic's Message Batches API is 50% cheaper, with most batches finishing in under an hour. If you use another vendor, check whether its API has an equivalent batch mode and what it charges.
Trade-off: no real-time responses, and more code to submit, poll and collect results.
8. Do not resend what the model does not need to see again
Large reference material that every request needs can sometimes be replaced by retrieval: store it once, search it per request, and send only the matching passages. This trades a large fixed input for a small, variable one.
Trade-off: retrieval can miss relevant passages, and building it is real work. It pays off when the reference material is much larger than what any single question needs.
Which lever to pull first
| Symptom in your usage data | Try first |
|---|---|
| Input tokens grow with every turn | Summarise history, clear old tool results |
| Same long prefix on every request | Prompt caching |
| Output tokens dominate | Conciseness instructions, lower reasoning effort, structured output |
| Many simple, repetitive requests | Smaller model, batch API |
| Large documents attached to every call | Extract the relevant parts, or retrieval |
Context limits and token costs are two sides of the same budget; what a context window is covers the limit side. To cut model costs to zero for some workloads, you can also run an open model on your own machine; see how to run an LLM locally with Ollama.
Frequently asked questions
- Do cached tokens still count toward the context window?
- Yes. Caching changes what you pay for repeated input, not whether it takes up space. Anthropic's context-window docs say cached prompt prefixes still occupy the window, and cache reads and writes are reported as separate input fields that all count toward it.
- Why did my bill go up after enabling prompt caching?
- On Anthropic's API, writing to the cache costs more than normal input: 1.25 times the base input price for the default 5-minute cache and 2 times for the 1-hour cache. If a cached prefix is written but rarely read again before it expires, you pay the premium without the savings. Check the cache read and cache creation fields in the usage data to see your hit rate.
- Does a shorter prompt always reduce cost?
- It reduces input cost, but if removing context makes the model answer badly, you may spend more on retries or longer follow-ups. Cut repetition, stale history and irrelevant data first, and keep the instructions and examples that make the answer right the first time.
- Are output tokens more expensive than input tokens?
- Usually, yes. As one example, Anthropic lists Claude Opus 5.5 at $4 per million input tokens and $20 per million output tokens (as of October 2026), a 5 to 1 ratio. Reasoning or thinking tokens are billed as output too, so limiting output often saves more per token than trimming input.