Claude Context Window Management in Tool Design

A Claude tool that works in a demo can still struggle on turn 20 of a real session. Tool definitions, tool results and earlier turns pile up in the context window, request after request, and much of that weight is no longer doing anything. Managing the window is a design job, and most of the decisions sit in how you define tools and what those tools send back.

We build custom Claude tools through our AI integration service, so weigh that. Every fact below about Claude’s models, limits, prices and context features was checked against Anthropic’s own documentation in October 2026.

Short answer: Current Claude models (Opus 5.5, Sonnet 5.5 and Haiku 5.5) have a 1M-token context window and can write up to 128k tokens per request. Everything counts toward that window: the system prompt, every message, tool results, your tool definitions and Claude’s output, including its thinking. A bigger window doesn’t fix a bloated one, because Anthropic says accuracy and recall degrade as the token count grows. Keep tool definitions out of context until they’re needed with tool search. Cache the stable part of the prompt with prompt caching, which cuts what you pay but not what fills the window. Clear old tool results with context editing, and summarize long sessions with compaction. Return only the fields Claude needs from each tool, and count tokens before you send.

What Anthropic Documents

As of October 2026, from Anthropic’s documentation:

Sources: Models overview, Context windows, Tool use overview, Pricing, Prompt caching, Token counting, Context editing, Compaction, Tool search tool.

How Claude’s Context Window Actually Works

Tokens in, tokens out, what fills the window

The context window is everything Claude can see when it writes a reply, including the reply itself. In an API request that means the system prompt, every message in the conversation (tool results, images and documents included), your tool definitions and the output Claude generates for that turn (source). Each response reports what the request used in its usage field.

The API doesn’t forget anything for you. Each user message and assistant response stays in the history, so a tool result that returned a large JSON payload on turn 3 is still being sent on turn 20 unless you remove it. That’s how a session that started small grows into a heavy one.

Thinking adds to it. On current models, thinking tokens are part of your max_tokens, billed as output and counted toward the window. On Opus 4.5 and later, Sonnet 4.6 and later and Haiku 5.5, the API keeps previous thinking blocks by default, so they count as input on later turns (source). The manual extended thinking mode with budget_tokens isn’t accepted on the newest models; they use adaptive thinking, where Claude decides how much to think (source).

Size isn’t the only limit. Anthropic says more context isn’t automatically better: as the token count grows, accuracy and recall degrade, which it calls context rot (source). A long session can get worse well before it gets full. And if the input alone is bigger than the window, the API rejects the request with a “prompt is too long” error.

The 1M window on current Claude models

Opus 5.5, Sonnet 5.5 and Haiku 5.5 all have a 1M-token window, and 1M is the default on every model that has it, with no beta header (source). Other models, including Sonnet 4.5, which is deprecated, still have 200k.

Long prompts are billed at standard rates on most current models: a 900k-token request pays the same per-token rate as a 9k-token one. Haiku 5.5 is the exception, with higher prices for prompts over 100,000 tokens (source).

So the question has changed. It used to be “will it fit?” Now it’s “should it be there?” A full million tokens costs money on every request and, by Anthropic’s own account, can lower answer quality. The rest of this post is about keeping the window small on purpose.

Why Tool Definitions Are the Biggest Hidden Context Cost

Tool definitions are easy to forget because nobody types them into the chat. They go out with every request in the tools parameter, and the API adds a tool use system prompt on top: 286 tokens on Opus 5.5, Sonnet 5.5 and Haiku 5.5 when Claude picks the tool itself (source). Anthropic’s own example of five connected services (GitHub, Slack, Sentry, Grafana and Splunk) uses about 55k tokens in definitions before Claude does any work (source).

What a bloated tool schema looks like

Bloat isn’t a long description. Anthropic calls detailed descriptions “by far the most important factor in tool performance” and recommends at least 3–4 sentences per tool: what it does, when to use it and when not to, what each parameter means and what it doesn’t return (source). Cutting those to save tokens tends to cost you more in wrong tool calls.

What bloats the window is weight that isn’t helping Claude choose or call the tool:

• Reference data in the schema. A search_crm_contacts tool that lists every field in your CRM and every filter combination is a database manual, not a tool definition.
• Tools that aren’t needed for this request. An agent with 40 tools sends all 40 every turn, even when the user’s question needs two.
• Near-duplicate tools. Five tools that differ only in one argument make Claude choose between them, and Anthropic notes that selection accuracy degrades past 30–50 tools (source).

Measure before you trim. The token counting endpoint accepts the same tools, system prompt and messages as a real request and returns the input token count, so you can see exactly what your definitions cost (source).

The action parameter pattern: consolidating tools

Anthropic recommends grouping related operations into one tool with an action parameter, rather than creating a separate tool for every action (source). Instead of create_ticket, update_ticket and close_ticket, you define one ticket_manager tool with an action enum of create, update and close. Anthropic says fewer, more capable tools reduce selection ambiguity, and one definition replaces three.

The pattern works when the actions share similar inputs. When the inputs diverge a lot, one tool ends up with a long list of parameters that only apply to some actions, and separate tools are clearer. Anthropic also recommends prefixing tool names with the service, such as github_list_prs or slack_send_message, once your tools span several systems (source).

Four Strategies for Managing Context in Claude Tool Use

Anthropic’s guide to managing tool context lists four approaches, each aimed at a different source of pressure, and says they can be combined (source). Ours differs by one: we cover server-side compaction instead of programmatic tool calling, which lets Claude run a chain of tool calls as one script so the intermediate results never enter the history.

Tool search, definitions out of context until needed

With the tool search tool, you still send every tool definition, but you mark most of them defer_loading: true. Claude sees the search tool, searches your catalog by name and description, and only the tools it picks are loaded. At least one tool, normally the search tool itself, must stay non-deferred (source). There are two variants: one where Claude writes regex patterns, and a BM25 variant that takes natural language queries.

Anthropic says tool search typically cuts definition overhead by over 85 percent, loading only the 3–5 tools a request needs (source). The cost is a little latency, one extra turn to look a tool up, and it suggests tool search for toolsets of roughly 20 tools or more (source). It’s supported on Opus 5.5, Sonnet 5.5 and Haiku 5.5.

Prompt caching, amortizing stable tool definitions

Prompt caching stores a prompt prefix so later requests that start the same way read it from cache. The prefix is built in a fixed order, tools, then system, then messages, so stable tool definitions and a stable system prompt are natural cache targets (source).

The numbers, from Anthropic’s prompt caching page:

• Lifetime: 5 minutes by default, refreshed at no cost each time the cache is used. A 1-hour option costs more to write.
• Write price: 1.25x the base input price for 5 minutes, 2x for 1 hour.
• Read price: 0.05x the base input price on Opus 5.5 and Sonnet 5.5, so $0.20 and $0.10 per million tokens. Most other models, including Haiku 5.5, read at 0.1x.
• Minimum: 512 tokens on the 5.5 models. Shorter prefixes are processed without caching, and no error is returned.

Two rules matter for tool design. Changing a tool definition (name, description or parameters) invalidates the entire cache, so don’t rebuild or reorder your tool list per request (source). And cached tokens still sit in the window: caching changes what you pay for them, not whether they count (source).

Context editing, pruning stale tool results

Context editing clears content from the conversation before the prompt reaches Claude. It’s in beta behind the context-management-2025-06-27 header. The tool result clearing strategy starts once the prompt passes a trigger (100,000 input tokens by default), clears the oldest tool results first, keeps the 3 most recent tool uses by default and replaces each cleared result with a placeholder so Claude knows something was removed (source). Your application keeps the full, unedited history.

You can protect tools whose results must never be cleared with exclude_tools. Clearing breaks the prompt cache at that point, so the docs suggest clear_at_least to make sure each clearing removes enough tokens to be worth the new cache write. A second strategy clears old thinking blocks.

Context editing pairs well with the memory tool. Claude stores notes in a /memories directory that your application runs on its own storage, and when the context approaches the clearing threshold, Claude gets a warning so it can save what matters before old results are cleared (source).

Server-side compaction, when to use it and when not to

Compaction replaces the older turns of a conversation with a summary that Claude writes on the server. Anthropic calls it the primary strategy for long-running conversations and agents (source). It comes in two beta forms. Compaction on demand lets you decide when to compact, with a loop you write. Compaction at a token threshold is one parameter on your normal requests: the API compacts once input reaches your trigger, 150,000 tokens by default and at least 50,000 (source).

A summary is still a summary. Anthropic’s own docs suggest writing your own summarization prompt “when the default summary drops something a later turn needs”, and offer a mode that keeps recent turns word for word (source). Use compaction for long conversational sessions where the gist of earlier turns is enough. Where Claude must refer back to exact tool outputs, such as an order total or an ID, keep those outputs out of the summarized part, or use context editing with exclude_tools instead.

Designing Tool Responses That Don’t Bloat the Window

Every tool result you return stays in the history until something clears it. The cheapest token is the one you never send.

Return only high-signal fields

Anthropic’s guidance is to include only the fields Claude needs to reason about its next step, because bloated responses waste context and make it harder for Claude to find what matters (source). A CRM API might return dozens of fields for one contact: billing address, internal IDs, audit timestamps, fields kept for legacy reasons. If the task needs a name, a company, a deal stage and a last-contact date, return those.

Do the filtering in a thin layer between the external API and the tool result. It’s ordinary code, it’s easy to test, and it means a vendor adding fields to their API doesn’t quietly grow every session.

Semantic identifiers over opaque references

When a tool result refers to something Claude will act on later, such as a ticket, a document or a customer, return a semantic, stable identifier rather than an opaque internal reference. Anthropic gives slugs and UUIDs as examples (source). The point is stability: Claude should be able to pass the identifier back to the next tool call exactly as it received it.

Pair the identifier with a short label where it helps Claude reason, for example the ticket’s subject or the order number a customer would recognize. That’s often more useful to the model than a long raw record.

Real-World Architecture: What This Looks Like in Practice

Here’s how these pieces fit together in a typical internal support agent. It’s an illustration of the pattern, not a client project.

1. Define the tools well, and fewer of them. Group related operations behind an action parameter, write 3–4 sentence descriptions and namespace the names by system.
2. Defer what isn’t always needed. If the toolset passes about 20 tools, keep the core ones loaded and defer the rest behind tool search.
3. Cache the stable prefix. Keep the tool list and system prompt identical across requests and cache them.
4. Filter every tool response. Return the fields the next step needs and a stable identifier, nothing else.
5. Clear and compact as sessions grow. Add tool result clearing once early results stop mattering, protect the tools whose outputs must stay exact, and add compaction for very long sessions.
6. Measure. Count tokens for a typical request before launch, then watch the usage field in production.

Here’s what step 3 does to the bill, as our own worked example from Anthropic’s published Sonnet 5.5 prices (source). Say the tools and system prompt come to 20,000 tokens and the agent handles 1,000 requests, each within 5 minutes of the last. Without caching, that prefix is 20 million input tokens at $2 per million: $40. With caching, the first request writes it for $0.05, and the other 999 read it at $0.10 per million, about $2.00, so about $2.05 in total. The prefix still uses the same 20,000 tokens of window on every request. Only the price changes.

Related Posts

• Claude API rate limiting: token budgets and retry logic before you build
• Custom AI tool economics: cost per use against a monthly subscription
• Connecting Claude to your CRM: sending only the fields a task needs
• AI pipeline orchestration design: structuring multi-step Claude workflows

Frequently Asked Questions

How many tokens do Claude tool definitions typically consume?

It depends on how many tools you define and how much each one says, so count yours with the free token counting endpoint rather than guessing. Two fixed points from Anthropic: the API adds a 286-token tool use system prompt on Opus 5.5, Sonnet 5.5 and Haiku 5.5, and its example of five connected services uses about 55k tokens of definitions before any work starts. Don’t shorten descriptions to save tokens. Anthropic recommends at least 3–4 sentences per tool. Cut unused tools and embedded reference data instead.

What is Claude’s context window limit in 2026?

On Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 it’s 1M tokens, with no beta header, and each request can generate up to 128k output tokens. Some older models, including Sonnet 4.5, have 200k. On most current models a long prompt is billed at the standard rate, but Haiku 5.5 charges more for prompts over 100,000 tokens. A bigger window doesn’t mean you should fill it: Anthropic says accuracy and recall degrade as context grows.

Can I use prompt caching with custom tool definitions?

Yes. Tool definitions come first in the cached prefix, so a stable tool list is a good cache target. On Sonnet 5.5 a cache hit costs $0.10 per million tokens against $2 uncached, and a 5-minute cache write costs 1.25x the base price. The prefix must be at least 512 tokens on the 5.5 models. Any change to a tool’s name, description or parameters invalidates the whole cache, so keep the list identical between requests.

What’s the difference between server-side compaction and context editing?

Compaction summarizes: Claude replaces older turns with a summary written on the server. It’s the broader tool for long sessions, but a summary can drop details. Context editing removes: it clears old tool results or thinking blocks by rule, replaces them with a placeholder and leaves everything else as it was. You can exclude specific tools from clearing. Both are in beta. Use compaction when the gist of earlier turns is enough, and context editing when specific outputs must stay exact.

When should I use tool search instead of sending all tool definitions upfront?

Anthropic suggests adding tool search once a toolset passes roughly 20 tools or the baseline context becomes noticeable, and notes that Claude’s tool selection accuracy degrades past 30–50 tools. Below that, sending every definition and caching them is simpler. Tool search costs one extra turn to look a tool up, so it suits agents where most tools aren’t needed on most requests.

Should tool response filtering happen inside the tool function or in Claude’s prompt?

Inside the tool function, before the result enters context. Telling Claude to ignore fields in the prompt doesn’t save tokens, because those fields are already in the window and stay in the history. Anthropic’s guidance is to return only the fields Claude needs for its next step, with stable identifiers. Keep the filtering layer thin, deterministic and tested.