Code retrieval for AI agents: Why vector embeddings fail and per-file LLM graphs win
A year-long experiment building a code indexing system for AI coding tools yielded clear results: vector embeddings on code chunks and Tree-sitter AST parsing both have critical flaws, while per-file LLM analysis stored in a Neo4j graph with semantic fulltext search works best. The findings echo recent papers like RepoGraph (ICLR 2025) and Code-Craft.
Approaches tested
- Vector embeddings on code chunks – discarded entirely. A function named
process()in a payments service and one in an image pipeline embed to similar vectors, despite having nothing to do with each other. Vectors flatten call graphs, inheritance, imports — all structural relationships. Retrieval precision was unacceptable. - Tree-sitter AST parsing – precise and fast, but structural-only. It can tell you a function exists and what it calls, but cannot answer the question “this function handles webhook retries for failed Stripe payments.” Falls short when developers phrase questions in business language.
- Per-file LLM analysis → graph – works. Every file gets an LLM call generating
purpose,summary, andbusinessContext, stored as nodes in Neo4j with edges to classes, functions, keywords, and imports. Retrieval uses fulltext search across those semantic fields instead of vector similarity. SHA-256 diffing limits reindexing to changed files, making the upfront cost manageable.
Benchmarks from literature
RepoGraph (ICLR 2025) showed +32.8% improvement on SWE-bench with graph approaches. Code-Craft achieved +82% top-1 retrieval precision using bottom-up LLM summaries from code graphs.
Comparison to existing tools
The team published a side-by-side in comparison.md. Key differences:
- Bytebell: per-file LLM → purpose + summary + businessContext + entities; Neo4j + MongoDB storage; SHA-256 diff-aware reindex.
- PageIndex: TOC reasoning tree for long PDFs/docs; no code-specific semantics.
- GitNexus: Tree-sitter AST + community detection; optional per-symbol semantics; uses LadybugDB.
- GraphRAG: per-chunk LLM entities + community clustering for general text, not code.
- Sourcegraph/Cody: LSIF/SCIP search index; no per-node semantics; deployment is self-hosted or SaaS.
- Augment: proprietary semantic index with embeddings; SaaS-only; continuous indexing managed.
Open source
The system is open source at github.com/ByteBell/bytebell-oss.
📖 Read the full source: r/LocalLLaMA
👀 See Also
DAUB MCP Server Lets Claude Generate and Render UIs via JSON Specs
DAUB is an MCP server that enables Claude to generate UI interfaces directly from natural language prompts, producing structured JSON specs that render as live interfaces without code generation or compilation. It exposes four tools including generate_ui, render_spec, validate_spec, and get_component_catalog.
Local 35B MoE Model Drops Agent OS Code Failure Rate to 0%
A developer reports that switching a multi-agent system's runtime to Qwen 3.6 35B A3B (MoE, 3B active params) eliminated code failures, achieving 100% success rate through a 5-layer validation gate.
VTCode: A Rust TUI Coding Agent That Aggressively Trims Context with AST-Level Chunking
VTCode is an open-source Rust TUI coding agent that aggressively trims context using AST-level chunking via ripgrep and ast-grep. It supports custom OpenAI-compatible providers, sandboxing with macOS Seatbelt and Linux Landlock, and tree-sitter-bash validation on generated commands.
Meera: A Fully Offline AI Assistant for Linux Gnome Built on Qwen3.5-2B
Meera is an offline AI assistant for Gnome Desktop that uses Qwen3.5-2B-Q4_K_M (1.2 GB) and llama-cpp with Vulkan support. It leverages a second tiny embedding model for tool selection and RAG, avoiding prompt embedding bloat. Works on Ubuntu 24.04 with RTX 5090 and Fedora Silverblue on Intel i3.