Replacing complex retrieval pipelines with simple git shell commands for LLM agents

From complex pipeline to simple shell access

The team originally built DiffMem, a git-backed memory system for AI agents with version history as context. Their retrieval layer used sentence-transformers for cosine similarity scoring, rank-bm25 for keyword search, and a two-pass LLM pipeline to distill queries and synthesize results. This resulted in a 3GB Docker image (due to PyTorch dependencies), 10% timeout rates on heavy users, and cold starts that rebuilt an in-memory BM25 index each time.

The realization: LLMs already know git

The insight came from recognizing that Unix commands are densely represented in LLM training data through billions of README files, CI scripts, and Stack Overflow answers. The team realized they were extracting information from git with their own code and feeding it to a model that already understands git commands.

The solution: One tool function

They replaced everything with a single tool:

{
  "name": "run",
  "description": "Execute a read-only command in the memory repository",
  "parameters": {
    "command": "Shell command (supports |, &&, ||, ; chaining)"
  }
}

How the agent works

The agent follows a fixed protocol: read the entity manifest, run a temporal probe against the commit log, batch investigation into a single tool call, output a retrieval plan, then stop. It returns pointers, not content, keeping context lean.

The agent reads lightweight signals during turns:

• head -30 for structure
• grep -n for keywords
• git diff HEAD~3.. for recent changes

Real example: Finding connections through commit history

When a user sent a birthday message mentioning feeling isolated, the agent ran:

git log --format='%h %ad' --date=relative --name-only -15

This revealed that wife.md and company.md changed in the same session, and a key colleague appeared in 2 of the last 3 sessions. Keyword search (BM25) would never have found company.md from "feeling isolated on my birthday," but the temporal connection in git history was what mattered.

In turn 3, the agent composed a single tool call with nine commands chained with semicolons:

git diff HEAD~2.. -- memories/people/wife.md; git log --stat -5 -- memories/people/wife.md; head -30 memories/people/wife.md; grep -n "birthday|surgery|stress" memories/people/wife.md; tail -50 timeline/2026-03.md; git diff HEAD~3.. -- timeline/2026-03.md; grep -n "project|deliverable" memories/contexts/company.md; git diff HEAD~2.. -- memories/contexts/company.md; git diff HEAD~1.. -- memories/people/colleague.md

Results

The final output was a JSON retrieval plan with specific git diffs, priority levels, and token estimates. This allowed deletion of rank-bm25, sentence-transformers, scikit-learn, and numpy. The Docker image dropped ~3GB, server starts faster, uses less memory, and the 10% timeout rate disappeared. What remains: requests, openai, and gitpython.