AI Coding Agents Struggle with Context Management in Large Codebases

The Execution Bottleneck Isn't the Problem

Observations from real codebase usage show AI coding agents consistently spend significant time on discovery rather than execution. Each time an agent tackles a new task, it makes 15-20 tool calls for orientation activities including:

• Grepping for routes
• Reading middleware
• Checking types

By the time the agent starts writing code, it has already consumed a substantial portion of its context window on discovery work.

Evidence from Simplified Approaches

Vercel demonstrated this problem from the opposite direction by removing 80% of tools from their agent and giving it bash access instead. This approach resulted in 100% accuracy, suggesting execution capability isn't the limiting factor.

Similarly, Pi (the minimal coding agent) proves the same point with just 4 tools and a system prompt containing fewer than 1,000 tokens.

The Real Challenge: Context Management

If execution is effectively solved, the actual difficult problem becomes context management. Several factors contribute to this challenge:

• Large codebases don't fit within any current context window
• Long tasks accumulate tool outputs that push early reasoning out of the attention window
• Dynamic environments change between sessions
• The "Lost in the Middle" research shows models reason best at the start of their context window — exactly when agents are still searching

The author has published a more detailed analysis exploring these issues and their implications for AI coding agent development.