oMLX introduces SSD KV caching for Apple Silicon, reducing OpenClaw response times from 30-90 seconds to 5 seconds
What oMLX solves
Running OpenClaw locally typically means sending the same massive system prompt (20-30k tokens covering tools, skills, workspace context) on every request. While Ollama and LM Studio cache KV state, they invalidate the entire cache and recompute from scratch when context shifts mid-session, resulting in 30-90 second response times.
oMLX fixes this by persisting KV cache blocks to SSD in safetensors format. When a previously seen prefix returns, it's restored from disk instead of recomputed - working across requests and server restarts. Since OpenClaw's system prompt is mostly static (only timestamps and runtime metadata shift), SSD caching means only changed parts get recomputed.
Performance benchmarks
Tested with Qwen3.5-122B-A10B-4bit on M3 Ultra 512GB:
- Single request benchmarks:
- 1k context: 768 tok/s prompt processing, 56.6 tok/s generation, 65.5 GB peak memory
- 8k context: 940 tok/s prompt processing, 51.4 tok/s generation, 69.3 GB peak memory
- 32k context: 764 tok/s prompt processing, 42.4 tok/s generation, 73.4 GB peak memory
- Continuous batching (pp1024/tg128):
- 1x batch: 56.6 tok/s, 1.00x speedup
- 2x batch: 92.1 tok/s, 1.63x speedup
- 4x batch: 135.1 tok/s, 2.39x speedup
- 8x batch: 190.2 tok/s, 3.36x speedup
Setup with OpenClaw
- Download the DMG from releases and drag to Applications
- Point it at your model directory (reuses LM Studio models, no re-download needed)
- Add oMLX as a custom provider in openclaw.json
- The web dashboard generates the exact config - no terminal needed
Additional features
- Multi-model serving: LLM + embedding + reranker simultaneously
- Tool calling for all major formats (JSON, Qwen, Gemma, GLM) + MCP
- Tool result trimming - truncates oversized tool outputs
- OpenAI + Anthropic /v1/messages drop-in compatibility
- Native macOS menu bar app (not Electron)
- Apache 2.0 license, 100% open source
📖 Read the full source: r/openclaw
👀 See Also
MCP-India-Stack: Offline-first server for Indian financial data in AI agents
MCP-India-Stack is an offline-first MCP server that provides Indian financial and government API functionality without authentication or external API calls. It bundles datasets locally for tax calculations, validation tools, and lookups.
Mia: Local AI Workspace Daemon with Native Android App and P2P Streaming
Mia is a daemon that runs on your machine and pairs with a native Android app over P2P, allowing you to kick off and monitor long-running AI coding tasks from your phone. It supports OpenCode, Claude Code, Gemini CLI, and Codex agents, streaming output directly to your device in real time.
JetBrains Introduces Plugin for Modern Go Code with AI Agents Junie and Claude Code
JetBrains has released a plugin for AI agents Junie and Claude Code, enhancing their ability to generate modern Go code by adhering to the latest Go features and best practices.
vllm-mlx fork adds tool calling and prompt cache for local AI coding agents
A developer has modified vllm-mlx to fix tool calling issues and add prompt caching, reducing TTFT from 28s to 0.3s for OpenClaw on Apple Silicon. The fork supports Qwen3-Coder-Next at 65 tok/s on M3 Ultra with working function calling.