Agent Framework Token Bloat: A 500:1 Input-to-Output Ratio Is Normal

A Reddit user running a self-hosted Telegram-based AI agent with multi-provider routing noticed extreme input-to-output token ratios: ~21k input tokens per message vs 50-200 output tokens, yielding ratios of 100:1 to 500:1. Breakdown: tool definitions ~13k tokens, system prompt ~5k, memory/context files ~3k, user message <100 tokens.

Is This Normal?

Community response confirms that 15-25k baseline context is standard for agent frameworks like LangChain and AutoGPT. The high ratio is structural to having real tool access. Key recommendations:

• Cheap primary model — costs stay bounded even with bloat
• Prompt caching — saves in active sessions but has a 5-minute TTL, limiting effectiveness across idle periods
• Spending caps — essential guardrail even with cheap models

Mitigation Strategies

Users debate two approaches: trim tool definitions per-message based on intent (dynamic tool selection) vs. accepting the bloat and relying on caching. Benchmarking suggests forking the framework to reduce overhead is rarely necessary unless building at scale. The consensus: 21k context is “the cost of doing business” with agent frameworks.