RTX 5000 PRO 48GB Delivers 4400 tok/s Precision Caching for Qwen3.6-27B
One developer took a gamble on the RTX 5000 Pro 48GB ($4300 including taxes) against a Mac Studio — and the numbers justify the leap: up to 4400 tokens/second in prompt processing (PP) and 50–80 tok/s in text generation (TG) with Qwen3.6-27B-FP8 and a full-precision BF16 KV cache.
Hardware and Cost Breakdown
- GPU cost: $4300 (incl. taxes)
- Total build: $5600 with 64GB RAM
- Context limit: 200K tokens at full precision (BF16 KV cache)
Performance Benchmarks
- Prompt processing: 4400 tok/s
- Text generation: 50–60 tok/s for very large prompts, up to 80 tok/s for smaller ones
- Model: Qwen3.6-27B-FP8 with full-precision cache
- Power draw: Roughly half of a dual RTX 5090 setup
Key Observations
The user built the PC from zero experience, relying on Claude Code (burning 50% of weekly Claude Code Max limits on vLLM/Linux setup). A Reddit post detailing exact vLLM settings for Qwen3.6-27B-FP8 with BF16 cache was the primary reference. The author notes that two RTX 5090s would outperform but at significantly higher cost, noise, and power consumption.
📖 Read the full source: r/LocalLLaMA
👀 See Also
Claude Tops App Store Charts Amid Government Standoff
Anthropic's Claude app jumped from 42nd to 1st place on the US App Store's Top Downloaded charts, with ChatGPT and Gemini taking second and third. The surge follows a public disagreement between Anthropic and the US government over military and surveillance use of AI technology.
Analyzing Claude's 1M Context Window Token Burn: Data Shows Unbounded Growth and Cache Miss Compounding
Analysis of Claude's 1M context window reveals two compounding factors causing rapid token consumption: unbounded context growth without auto-compaction and expensive cache misses at larger context sizes. The author provides a Python script to analyze personal token usage from JSONL session files.
Apple Using Google Gemini Access for On-Device AI Model Distillation
Apple has full access to Google's Gemini model for distillation, creating smaller on-device AI models for Siri and other features in iOS 27 without internet connectivity.
Qwen3.6 27B FP8 Runs 200k Tokens BF16 KV Cache at 80 TPS on RTX 5000 PRO 48GB
A Reddit user shares a vLLM setup for Qwen3.6 27B FP8 with BF16 KV cache at 200k tokens, achieving 60-90 TPS on a single RTX 5000 PRO 48GB. Full environment variables, config, and benchmark results are provided.