LLM Council Analysis Reveals Practical Claude Code Token Optimization Strategies

Problem and Experiment Setup

A developer experiencing daily Claude Code usage limits conducted an experiment using LLM Council (https://github.com/karpathy/llm-council). The setup involved 5 different personas that were forced to critique, challenge, and refine solutions, followed by a peer review round.

Key Findings

The analysis revealed that the biggest token drain wasn't complexity, but using "thinking mode" by default. This alone was burning tokens almost like Opus.

Practical Optimization Habits

• Turn OFF extended thinking by default
• /clear after every git commit (non-negotiable)
• Stop writing "yes / continue" prompts
• /compact every ~40 messages
• Keep CLAUDE.md lean or you pay tax every session

Mental Shift and Results

The core insight: Stop treating intelligence as default. Treat it like a resource you deploy intentionally. This shift enables:

• 30-50% token savings instantly
• Ability to actually use Opus without fear
• Predictable daily workflow instead of random limit hits

The council emphasized one rule: If you don't track /cost, you're not optimizing... you're guessing.

Outcome

With the full playbook implementation:

• ~60-70% reduction in token usage
• Same or better output quality
• Opus becomes usable for high value work

The developer noted this approach was more effective than any single prompt hack.