PinchBench Results: First OpenClaw-Specific AI Coding Agent Benchmark
PinchBench is the first benchmark specifically designed for evaluating AI coding agents in the OpenClaw ecosystem, ranking models by success rate, cost, and speed.
Key Results
The benchmark tested 32 models. Top performers by success rate:
- 1. google/gemini-3-flash-preview: 95.1% success, $0.72 cost, 254.50s speed
- 2. minimax/minimax-m2.1: 93.6% success, $0.14 cost, 239.79s speed
- 3. moonshotai/kimi-k2.5: 93.4% success, $0.20 cost, 291.67s speed
- 4. anthropic/claude-sonnet-4.5: 92.7% success, $3.07 cost, 304.53s speed
- 5. google/gemini-3-pro-preview: 91.7% success, $1.48 cost, 239.55s speed
Notable Findings
- Flash models beat Pro models at lower cost: Gemini-3-Flash-Preview (95.1%, $0.72) outperforms Gemini-3-Pro-Preview (91.7%, $1.48)
- More expensive models don't necessarily perform better
- Minimax 2.5 ranked 31st with 35.5% success rate, 105.96s speed (cost not listed)
- Several models show high success rates above 90% while keeping costs under $1
Performance Range
Success rates range from 95.1% (top) to 35.2% (bottom). Cost-effective options include:
- openai/gpt-5-nano: 85.8% success for $0.03
- google/gemini-2.5-flash-lite: 83.2% success for $0.05
- mistralai/devstral-2512: 81.7% success for $0.10
Several models at the bottom of the ranking (positions 23-32) show success rates around 40% or lower, with costs not listed in the provided data.
📖 Read the full source: r/openclaw
👀 See Also
Open-source CLI uses Claude Haiku to automate Xero expense auditing
A developer has released an open-source Python CLI tool that uses Claude Haiku 4.5 to automate Xero expense auditing. The tool follows a 'deterministic code first, then AI to fill in the gaps' approach, keeping costs to a few cents per audit run.
Meeting Summarization on a 6GB GPU: qwen3.5:0.8B Works at 57s, Granite 4 350M Hallucinates
VoiceFlow v1.6.0 adds local meeting recording and summarization. Benchmarking sub-1B models on a 6GB RTX 3060: qwen3.5:0.8B produces structured summaries in 57s at 2.2GB VRAM, while Granite 4 350M hallucinates badly.
Approach to Self-Improving Memory in Local AI Agents
A developer shares their approach to persistent memory for local AI agents using markdown files as source of truth, episode scoring with confidence-based rules, and trust escalation based on approval patterns.
Running Qwen3.6-35B-A3B-UD-Q5_K_XL Locally with VS Code Copilot on AMD R9700
A user shares their working llama.cpp setup for Qwen3.6-35B-A3B-UD-Q5_K_XL on a single AMD R9700 with Vulkan, achieving full website and Playwright test generation from scratch with minimal nudging.