ETH Zurich Study: Excessive Context Reduces AI Coding Agent Performance

A recent study from ETH Zurich provides concrete evidence that more context doesn't necessarily mean better performance for AI coding agents. The research tested four coding agents across 138 real GitHub tasks, with clear quantitative results.

Key Findings

The study revealed that LLM-generated context files actually reduced task success rates by 2-3% while inference costs increased by 20%. Even human-written context files only improved success by approximately 4%, while still significantly increasing costs.

The Core Problem

Researchers discovered that agents treated every instruction in context files as something that must be executed. In one experiment, when they stripped repositories down to only the generated context file, performance improved again. This indicates that agents struggle to distinguish between essential instructions and irrelevant historical information.

Practical Recommendations

The study recommends only including information that the agent genuinely cannot discover on its own, keeping context minimal. This is particularly relevant for communication data like email threads, which might look like context but are often interpreted as instructions when they're actually historical noise.

Context API Solution

To address this issue, researchers developed a context API (iGPT) that focuses on email processing. The API:

• Reconstructs email threads into conversation graphs before context reaches the model
• Deduplicates quoted text
• Detects who said what and when
• Returns structured JSON instead of raw text

This approach ensures agents receive filtered context rather than entire conversation histories, improving their ability to focus on relevant information.