SMELT compiler reduces OpenClaw workspace token usage by up to 95%
OpenClaw workspace token optimization tool
SMELT is a Python compiler that processes OpenClaw workspace markdown files to reduce token usage when sending content to AI models like Claude or GPT. The tool addresses a specific inefficiency: OpenClaw resends USER.md, SOUL.md, MEMORY.md, and AGENTS.md on every message, not just at startup.
Performance benchmarks
Testing on a Qwen 3.5 122B model on M3-Ultra hardware revealed:
- Startup bundle: 7,268 tokens reprocessed on every inference call
- 50-message session: Over 350,000 tokens of static workspace files reprocessed
- Query-specific token reductions:
- "Who is Sally?": 1,373 tokens raw → 73 tokens SMELT (94.7% savings)
- "When was John born?": 1,374 tokens raw → 62 tokens SMELT (95.5% savings)
- Broad "Tell me about Alex": 1,373 tokens raw → 328 tokens SMELT (76.1% savings)
- Startup TTFT: 14,121ms raw → 13,273ms SMELT (6% faster)
Technical implementation
SMELT uses a four-layer architecture:
- Archive: Original files are never touched
- Compile: Schema-aware structural compression
- Compress: Dictionary replacement
- Select: Query-conditioned retrieval that only sends relevant records with parent context
The fourth layer (Select) is where the 95% token reduction occurs. The compiler is schema-aware and built specifically for OpenClaw workspace file conventions.
Key findings from development
- Naive JSON conversion (a common optimization attempt) is 30% worse than raw markdown
- Heading stripping provides minimal benefit (7-8% improvement)
- Byte compression and token compression are different - measurements must use the actual tokenizer
- 11 of 13 test files achieved 100% fidelity, with two dense archival files having documented failures
Current limitations and availability
The schema is hand-built for OpenClaw workspace conventions. Support for arbitrary markdown requires schema learning (planned). The tool is free for personal use, with code available on GitHub under TooCas/SMELT and research published on Zenodo with DOI.
The project was built with GPT, Claude, and Codex as collaborators.
📖 Read the full source: r/openclaw
👀 See Also
Homebutler: MCP Server for Multi-Server Homelab Management via Claude
Homebutler is a Go binary with a built-in MCP server that lets Claude manage multiple servers over SSH without installing agents on remote machines. It provides 9 tools including system status monitoring, Docker container management, port scanning, and alert rules.
GLM 5 on Mac M3: Performance Observations for Agentic Coding
A user reports running GLM 5 via MLX 4-bit quantization on a Mac M3 with 512GB RAM, finding it usable for agentic coding with context under 50k tokens but noting significant slowdowns beyond that threshold.
ComfyUI Skill Enables AI Agents to Queue and Batch Image Renders via Natural Language
A new open-source skill allows OpenClaw agents to construct ComfyUI workflows, submit jobs, and manage renders through natural language commands like 'Make 50 variations of this concept with different seeds' or 'Compare these 4 prompts side by side at 1024x1024'.
Calmkeep: An External Continuity Layer to Counter LLM Drift in Extended Sessions
Calmkeep is an external continuity layer designed to counteract LLM drift in extended sessions, showing 85% integrity vs 60% for standard Claude in a 25-turn backend build test and 100% vs 50% in a legal session.