Running Claude Code Offline on an M3 Pro with Qwen3.6: 4 Fixes That Made It Work

Claude Code connects to a local model on an Apple M3 Pro (18 GPU cores, 36GiB unified memory, ~150 GB/s bandwidth) running qwen3.6:35b-a3b-coding-nvfp4 — a 35.1B-parameter MoE model with ~3B active per token, NVFP4 quantized, ~21GB on disk and ~20GiB resident. The setup took a Kubernetes incident from investigation to PR: found root cause, wrote patch, pushed branch, filed PR via gh — all air-gapped. Four fixes turned a model that timed out in 10 minutes into one that closes the loop. Speed is hardware-bound; capability is not.

Stack and Environment

• Hardware: Apple M3 Pro, 18 GPU cores, 36 GiB unified memory, ~150 GB/s memory bandwidth
• Model: qwen3.6:35b-a3b-coding-nvfp4
• Runtime: Ollama 0.24.0, MLX runner (Apple Silicon-native)
• Client: Claude Code v2.1.84 pointed at local Ollama endpoint

Key environment variables (set in a launchd plist for persistence):

OLLAMA_MLX=1
OLLAMA_CONTEXT_LENGTH=32768
OLLAMA_FLASH_ATTENTION=1
OLLAMA_MULTIUSER_CACHE=1
OLLAMA_KEEP_ALIVE=24h
OLLAMA_NO_CLOUD=1

Setup Steps

1. Install Ollama 0.24.0+
2. ollama pull qwen3.6:35b-a3b-coding-nvfp4 (~21GB one-time)
3. Start server with the env vars above
4. Launch Claude Code:
   ANTHROPIC_BASE_URL=http://localhost:11434 MAX_THINKING_TOKENS=0 claude --model qwen3.6:35b-a3b-coding-nvfp4
5. Smoke test: Run kubectl get pods -A and tell me if anything appears unhealthy

Performance Notes

First tool call: seconds (thinking disabled). Prefill (loading ~25K tokens) takes ~60s. Subsequent turns are faster due to prefix caching (OLLAMA_MULTIUSER_CACHE). The model stays loaded via OLLAMA_KEEP_ALIVE=24h. Burst of 404s in Ollama log during prefill is normal (fix #4).

The MoE architecture is key: only ~3B active per token, so runtime cost resembles a 14B dense model while answers approach 35B. A dense 35B doesn't fit 36GiB.