I spent most of a Tuesday afternoon watching an agent burn through its context window before it had done anything useful. Not during the task — before it. I’d wired six MCP servers into a Claude Code session (Drive, Salesforce, a ticketing system, two internal APIs, a search tool), and by the time the model got around to reading my actual request, something like a third of the window was already gone. Spent on tool definitions it would never touch that session, a textbook case of MCP context bloat.
If you’ve connected more than two or three MCP servers to an agent, you’ve probably felt a version of this. The first response takes longer than it should. The model occasionally grabs the wrong tool out of a dozen similar-sounding ones, the same tool-selection confusion that turns into infinite loops in CrewAI and LangGraph if nothing catches it.
Your token bill for a two-line question looks suspiciously large. The tool list sitting in front of the model is the actual cause, not your prompt.
Here’s what’s actually eating that context, why the “just list every tool” approach falls apart past a certain scale, and the fix, straight from Anthropic’s engineering team and the MCP spec’s own client guidance, that doesn’t require ripping out your server config.
The MCP Context Bloat Problem Nobody Mentions in the Quickstart
Every MCP quickstart looks the same: one server, three tools, a clean demo. Nobody shows you what happens at twelve servers. The Model Context Protocol works as a host-client-server setup: your agent is the host, it spins up one client per connected server, and each server answers a tools/list request with its full catalog of tools, each one carrying a name, a description, and a JSON Schema for its inputs. Naive hosts take that entire catalog, from every connected server, and stuff it into the model’s context before the conversation even starts (the MCP architecture docs walk through the full tools/list exchange if you want the JSON-RPC details).
That’s fine for three tools. It is not fine for three hundred. The MCP docs put it plainly: loading every tool definition upfront “wastes tokens, increases latency, and degrades model performance.” Anthropic’s own engineering write-up on the problem describes agents connected to thousands of tools burning hundreds of thousands of tokens on definitions alone, before the user’s actual message gets read.
It gets worse once the agent starts chaining calls. Say it pulls a two-hour meeting transcript from Google Drive, then writes a summary into Salesforce. That transcript passes through the model’s context twice (once coming in, once going back out), which Anthropic estimates at roughly 50,000 extra tokens for that single workflow. None of it is the model “thinking.” It’s just data in transit, paying full context price both ways.

Where Those Tokens Actually Go
Here’s a real-shaped example, lifted from the MCP architecture docs, of what one tool definition costs:
{
"name": "salesforce_updateRecord",
"description": "Updates a record in Salesforce",
"inputSchema": {
"type": "object",
"properties": {
"objectType": { "type": "string", "description": "Salesforce object type" },
"recordId": { "type": "string", "description": "Record ID to update" },
"data": { "type": "object", "description": "Fields to update" }
},
"required": ["objectType", "recordId", "data"]
}
}That’s one tool. Multiply it by every tool on every connected server, and you start to see the shape of the problem. I didn’t want to just take Anthropic’s word for the scale of it, so I measured it myself with tiktoken:
import json
import tiktoken
TOOLS = [
{
"name": "salesforce_updateRecord",
"description": "Update fields on a Salesforce object",
"inputSchema": {
"type": "object",
"properties": {
"objectType": {"type": "string", "description": "Salesforce object type"},
"recordId": {"type": "string", "description": "Record ID to update"},
"data": {"type": "object", "description": "Fields to update"},
},
"required": ["objectType", "recordId", "data"],
},
},
{
"name": "calculator_arithmetic",
"description": "Perform mathematical calculations including basic arithmetic, trigonometric functions, and algebraic operations",
"inputSchema": {
"type": "object",
"properties": {
"expression": {"type": "string", "description": "Mathematical expression to evaluate"}
},
"required": ["expression"],
},
},
]
enc = tiktoken.get_encoding("cl100k_base")
counts = [len(enc.encode(json.dumps(t))) for t in TOOLS]
print(counts, sum(counts) / len(TOOLS))That gave me roughly 96 tokens per tool definition, on the cheap end: these two are simple, single-level schemas. Project that across 40 connected servers with 15 tools each, a completely ordinary setup for a team-wide agent, and you’re at close to 58,000 tokens before the model reads your question. Nested schemas, enums, and longer descriptions push that number up fast.
Anthropic’s own example (thousands of tools, hundreds of thousands of tokens) isn’t an outlier. It’s just further along the same curve.
Progressive Discovery: Catalog, Inspect, Execute
The fix the MCP client best-practices docs recommend is called progressive discovery, and the idea is almost embarrassingly simple once you see it: stop injecting every tool definition upfront. The host still calls tools/list against every connected server, same as before, but it holds those definitions back instead of dumping them into context. In their place, it gives the model one lightweight meta-tool: search_tools.
The pattern has three layers. First, catalog: the model calls search_tools({query: "update salesforce record"}) and gets back names and one-line descriptions only, not full schemas. Second, inspect: once it’s picked a candidate, it calls something like get_tool_details({name: "salesforce_updateRecord"}) to pull the complete input schema for that one tool. Third, execute: now it calls the tool, having loaded only what it actually needed.
Which discovery strategy powers that first search_tools call is up to you. Keyword matching (think BM25, or even basic substring overlap) works surprisingly well when tool names and descriptions are written clearly. Embedding-based search handles synonyms better (“update” vs. “modify” vs. “patch”) at the cost of standing up a vector index.
Some hosts hand the decision to a smaller, cheaper model like Claude Haiku or Gemini Flash to pick tools on the fly. And both OpenAI and Anthropic now ship built-in tool search in their own APIs, which is worth checking before you build your own version of any of this.

Anthropic’s Built-In Tool Search (No Custom Host Required)
Before you build any of this yourself, check whether your model provider already ships it. Anthropic’s tool search tool does the catalog/inspect/execute pattern natively, including for tools that come through the MCP connector. You add a tool_search_tool_bm25_20251119 (natural-language queries) or tool_search_tool_regex_20251119 (regex patterns) entry to your tools array, mark the tools you want deferred with "defer_loading": true, and Claude only loads the full definition for a tool once it’s actually found it through search:
tools=[
{"type": "tool_search_tool_bm25_20251119", "name": "tool_search_tool_bm25"},
{
"name": "get_weather",
"description": "Get the weather at a specific location",
"input_schema": {
"type": "object",
"properties": {"location": {"type": "string"}},
"required": ["location"],
},
"defer_loading": True,
},
# ...hundreds more deferred tools...
]Anthropic’s own numbers here are worth sitting with: a typical five-server setup (GitHub, Slack, Sentry, Grafana, Splunk) burns roughly 55,000 tokens in tool definitions before Claude does anything, and tool search cuts that by more than 85%. That’s suspiciously close to the ~58,000 tokens I extrapolated earlier from my own tiny test, which I honestly didn’t expect to line up that cleanly. It’s a good sign the napkin math holds up.
Their docs also name a harder failure mode than token cost: tool-selection accuracy measurably degrades once you pass 30–50 available tools, search or no search. If you’re already on the Claude API and pulling tools through MCP, this is the lowest-effort fix available: you don’t stand up a search layer at all, you just stop loading everything up front.
One MCP-specific detail worth knowing: when your tools come from MCP servers through Anthropic’s MCP connector, you don’t set defer_loading tool-by-tool. You set it once on the server’s mcp_toolset entry, and it applies to that whole server’s tools. Keep that in mind if you’re mixing a few MCP servers with a handful of native tools: the deferral granularity is different for each.
Rolling Your Own search_tools (When You Need Custom Logic)
Not on the Claude API, or need retrieval logic the built-in search doesn’t cover (access-control filtering, domain-specific ranking)? Then you’re back to implementing the catalog layer yourself. The keyword version is the cheapest place to start. Here’s one I actually ran, not just sketched:
from dataclasses import dataclass
@dataclass
class ToolDef:
name: str
description: str
schema: dict
CATALOG = [
ToolDef("salesforce_updateRecord", "Update fields on a Salesforce object", {}),
ToolDef("salesforce_upsertRecord", "Insert or update a Salesforce record based on an external ID", {}),
ToolDef("gdrive_getFile", "Fetch a file's contents from Google Drive by file ID", {}),
ToolDef("gdrive_searchFiles", "Search Google Drive for files matching a query", {}),
ToolDef("calculator_arithmetic", "Evaluate a mathematical expression", {}),
]
def search_tools(query: str, catalog: list[ToolDef], top_k: int = 3) -> list[dict]:
"""Naive keyword overlap. Swap for BM25 or embeddings once this stops being good enough."""
terms = set(query.lower().split())
scored = []
for tool in catalog:
haystack = f"{tool.name} {tool.description}".lower()
score = sum(1 for term in terms if term in haystack)
if score:
scored.append((score, tool))
scored.sort(key=lambda pair: pair[0], reverse=True)
return [{"name": t.name, "description": t.description} for _, t in scored[:top_k]]Run search_tools("update salesforce record", CATALOG) and you get back the two Salesforce tools, correctly ranked above the Drive and calculator ones. Here’s where it breaks, though: ask for “modify a record” instead of “update a record” and you get nothing, because there’s zero literal word overlap with either tool’s description. That’s the ceiling of keyword matching. Past a few dozen tools, or tools whose names don’t match how people actually phrase requests, you’ll want embeddings or a subagent picking for you.
When to Flip the Switch: The 1–5% Rule
Progressive discovery isn’t free. Every tool call now costs an extra round trip or two (search, then inspect, then finally execute), and that latency is real. The MCP client docs suggest a threshold: once tool definitions cross somewhere between 1% and 5% of your available context window, switch to progressive discovery.
Below that, just load everything; the overhead of the search layer isn’t worth it. Anthropic’s own tool search docs give a more concrete version of the same rule: reach for it at 10 or more tools, or once your definitions pass roughly 10k tokens; skip it under 10 tools, or when every tool gets used on every request, or when your total definitions stay under 100 tokens.
Honestly, I wouldn’t bother with any of this under roughly fifteen to twenty total tools. You’ll see people recommend progressive discovery as a default architecture decision from day one. I’d skip that advice: it adds real complexity and a few hundred milliseconds of latency per tool call, and for a small, focused agent, the definitions were never the bottleneck to begin with. Measure your own tool list with the token-counting script above before you build a search layer you didn’t need.
The Prompt Caching Trap Progressive Discovery Creates
Here’s the part that catches most teams off guard the first time they implement this. Most model providers cache the prompt prefix, and that includes the tools array. Progressive discovery, done carelessly, injects newly-fetched tool definitions into that array mid-conversation, which invalidates the cache.
The resulting cache miss can cost more tokens than the definitions you were trying to save in the first place. You end up worse off than if you’d just loaded everything upfront.
The fix is structural, not clever: append newly discovered definitions after the existing cache breakpoint instead of re-sorting the array, or better, route every tool invocation through a single stable call_tool({name, args}) meta-tool so the array itself never changes shape. Treat connecting or disconnecting a server as something that happens at a conversation boundary, not something you do mid-turn. It’s a small implementation detail that determines whether progressive discovery actually saves you money or quietly costs you more.
Programmatic Tool Calling: The Bigger Hammer
Progressive discovery fixes the tool-definition half of context bloat. It does nothing for the second half: intermediate results. If your agent reads a thousand log entries to find the unique errors and file a ticket for each, every one of those log entries still flows through the model’s context under plain tool calling, even though the model never needed to see most of them.
Programmatic tool calling, sometimes called “code mode,” solves this by having the model write code instead of calling tools directly. The host converts each MCP tool’s schema into a typed function stub, the same idea as turning messy output into a validated shape I wrote about when covering structured outputs and Pydantic validation for local LLMs, and the model writes a short script that calls those stubs, which runs inside a sandbox with no direct network access. Only the final output, usually a single console.log line, comes back to the model. In Anthropic’s log-deduplication example, a task that would have pushed 100,000+ tokens through context under direct tool calling collapses to roughly a 200-token script and a 15-token summary.
The catch: you need a sandbox. The MCP docs list a few real options: Deno or isolated-vm for JavaScript, the experimental Monty interpreter for Python, Wasmtime if you’re willing to compile to WebAssembly. None of them are a five-minute setup.
You also inherit a new security surface: the sandbox needs network isolation, the host has to broker every function call back to the right MCP server, and per-call authorization still applies to anything the generated code does on your behalf. That last point matters if your agent is already touching anything a person would normally approve by hand, which is exactly the territory I’ve covered before when it comes to keeping a human in the loop for automated workflows. A script running inside a sandbox doesn’t get a free pass on that.
Skip this one unless you’re already chaining multiple tool calls with large intermediate payloads flowing between them. If most of your agent’s tasks are a single tool call with a small result, programmatic tool calling is a lot of sandbox plumbing for a problem you don’t have.

Progressive Discovery vs. Programmatic Tool Calling
| Approach | Fixes | Setup cost | Best for |
|---|---|---|---|
| Load everything upfront | Nothing (this is the naive default) | None | Under ~15–20 total tools |
| Progressive discovery | Tool definition overload | A search/catalog layer, cache-aware tool array handling | Dozens to hundreds of tools across many servers |
| Programmatic tool calling | Intermediate result duplication | A sandboxed code runtime + broker | Multi-step chains with large intermediate data |
| Both combined | Both problems at once | Highest (you need both of the above) | General-purpose agents hitting both ceilings |
Pros and Cons of Progressive Discovery
Pros of Progressive Discovery
- Cuts tool-definition token usage by 90%+ once you’re past a few dozen tools (Anthropic’s own measurement: 98.7% in their worked example)
- Often improves tool-selection accuracy too: the model reasons over a handful of relevant candidates instead of scanning hundreds
- Works alongside dynamic server management, so you can connect servers on demand and disconnect them when a task’s done, freeing context further, handy for the same long-running agents where you’d otherwise lean on LangGraph checkpointing to resume state after a crash
Cons of Progressive Discovery
- Every tool call now costs one or two extra round trips (search, inspect), which adds latency
- Keyword search misses synonyms; you’ll eventually need embeddings or a subagent for messier tool catalogs
- Implemented carelessly, it breaks prompt caching and can cost more tokens than it saves
- It’s host-side behavior, not a protocol requirement: your MCP client has to actually implement it, and the spec doesn’t enforce it
Common Pitfalls
A few mistakes show up repeatedly once teams start implementing this:
- Stale search catalogs. If you’re not subscribing to
notifications/tools/list_changedand re-indexing when it fires, your search layer keeps recommending tools that no longer exist, or misses new ones entirely. - Swallowed tool errors. MCP tool failures come back as a normal response with
isError: true, not a transport error. If your code-mode wrappers don’t convert that into a thrown exception, the model’s generated script can silently continue past a failed step. - Over-engineering small catalogs. Building an embedding index for a dozen tools is wasted effort; see the threshold rule above. This is the same instinct that leads people to over-build retry logic for agents that get stuck in loops when a simpler bounded-retry check would’ve done the job.
- Disconnecting servers mid-turn. Treat server connect/disconnect as a conversation-boundary operation. Doing it per-turn thrashes your prompt cache for no real benefit.
Conclusion
If you’re already past a handful of connected MCP servers, don’t wait for the slowdown to get annoying before you deal with MCP context bloat. Run the token-counting script above against your own tool list first — you want a real number, not a guess, before deciding whether you even have the problem. If you’re under the threshold, leave it alone; adding a search layer to a small agent just adds latency for nothing.
If you’re over it and already on the Claude API, flip on the native tool search tool before building anything custom: it’s a few lines of config, not a new service to run. Only build your own catalog layer if you need retrieval logic the built-in version doesn’t offer, and only reach for programmatic tool calling once intermediate results, not tool definitions, are the thing actually filling up your context.
Most teams need the first fix. Fewer need the second.
FAQ
Does every MCP client support progressive discovery?
No. It’s a recommended pattern in the MCP client best-practices documentation, not a protocol requirement. Your host application has to implement the search_tools meta-tool and the deferred-loading logic itself; a server that answers tools/list normally works fine with either a naive host or a progressive-discovery one.
Will progressive discovery work with local LLMs through Ollama?
The mechanics are identical: it’s host-side, not model-side. In my experience, though, smaller local models are noticeably weaker at the “decide whether to search again or just answer” judgment call than frontier hosted models, so a subagent-based discovery strategy may underperform a straightforward keyword search when you’re running agents on local LLMs with Ollama. Keyword or embedding-based search is usually the safer default locally.
Is this the same thing as RAG?
Conceptually close, mechanically different. RAG retrieves document chunks to answer a question; progressive discovery retrieves tool metadata to decide what action to take. Both use the same retrieve-then-use shape, and both can use embeddings, but they’re solving different problems.
Does this slow down simple, single-tool tasks?
Yes, slightly. That’s exactly why the threshold rules exist (1–5% of context window, or Anthropic’s rougher “10+ tools / 10k+ tokens” cutoff). A search-then-inspect-then-execute flow adds round trips that a small, focused agent doesn’t need. Below the threshold, loading tools normally is faster end to end.
Does the same bloat problem apply to MCP resources and prompts, not just tools?
In principle, yes: resources and prompts are discoverable the same way tools are, through their own */list methods. In practice, tool definitions are almost always the bigger offender, since every tool carries a full input schema while resources are usually just referenced by URI until actually read.


