Why Coding Agents Prefer Grep Over LSP for Most Tasks

AgentConnect engineer Pengcheng Xu ran a pilot study comparing how coding agents use grep versus LSP-backed semantic navigation for code retrieval. The surprising result: agents often choose grep even when a more precise semantic tool is available — and forcing the semantic path can hurt task success.

Task-shaped tool routing

Across three Claude models (Opus 4.8, Sonnet 4.6, Haiku 4.5) and multiple repos, the models picked the LSP tool only 0–6% of the time for simple code-location tasks when both tools were available. On reference-completeness tasks (find every caller of a function), that jumped to 45–57% unprompted.

Forcing a semantic-first path on a localization task dropped success from 100% to 89%. The model's tool choice is task-shaped, not a blanket preference.

Semantic tools win only on noisy codebases

On reference-completeness tasks, LSP-backed paths hit 1.00 precision vs grep's 0.76 — but recall stayed around 0.66 with both. The bottleneck isn't retrieval noise; it's how thoroughly the agent traverses callers.

The deciding factor for accuracy gain was lexical noise, not language type. On a clean TypeScript repo (remeda), LSP added zero F1 gain and burned 16% more tokens. On a noisy TypeScript repo (hono), LSP improved F1 by 0.246 and used 12% fewer tokens.

LLM-friendliness matters more than precision

A tool isn't automatically model-friendly just because its results are precise. It must return enough context for the next step and present it in a shape the model can use directly. Familiarity matters too — models may have learned grep-style action paths during training, though that's a hypothesis, not a proven cause.

The results highlight a broader engineering problem: models don't use tools in isolation. They work through a harness that defines action names, inputs, and returned context. Your tool loop is part of the model's capability surface.