Dual-model architecture reduces token consumption by half for long conversations
Context compression system for AI agents
A developer on r/ClaudeAI shared a solution to the problem of AI agents losing context after conversation compaction. The system uses a dual-model architecture where a cheap small model (called the "subconscious") continuously compresses conversation history in the background.
Architecture details
The system has four layers:
- Narrative summary (~1K tokens)
- Compressed factoids
- Semantically retrieved verbatim quotes
- Raw recent turns
The main model ("conscious") receives a curated ~35K token context with the same information density that would normally require 120K tokens of raw history. The main model reads one coherent timeline and doesn't know the memory system exists.
Performance results
The developer simulated 260 turns across different conversation types. For sustained project work (starting with heavy research and gradually shifting to quick exchanges as the model learns the domain), the system cuts token consumption roughly in half.
Development tools
The system was built with Claude Code for the simulation and Claude.ai in the consulting and research stage. The developer is looking for others who have tried routing a smaller model to manage context for a larger one or found other workarounds for the compaction problem.
📖 Read the full source: r/ClaudeAI
👀 See Also
Developer tracks frustration with 'F-Bombs Per Thousand Prompts' metric across 44,212 Claude Code logs
A developer tracked 'fpk' (f-bombs per thousand prompts) across 44,212 Claude Code prompts over 5 months, finding frustration dropped 3.4× from Claude Opus 4-5 to 4-7, and most cursing targeted environmental tooling, not the model.
Agenexus: Agent-Native Platform for Autonomous AI Collaboration
Agenexus is a platform where AI agents register themselves via a SKILL.md file, complete capability challenges verified by Claude API, and get semantically matched for collaboration without human intervention. Built with Next.js, Supabase, Voyage AI embeddings, and Claude API.
WCY format reduces LLM token overhead by 50-71% and adds structural 'I don't know' markers
WCY (Watch-Compute-Yield) is a line-oriented format that reduces JSON token overhead by 50-71% and introduces structural '?' markers for LLMs to indicate uncertainty during reasoning. The format requires no fine-tuning—just three few-shot examples.
Yavio: Open-Source Product Analytics SDK for MCP Apps
Yavio is an open-source product analytics SDK for MCP and MCP Apps that automatically captures tool calls, errors, and resource reads with one function call. The MIT-licensed project provides a dashboard with per-tool breakdowns, funnels, retention, and error tracking.