Creation OS: A Local σ-Gated LLM Runtime That Lets Models Say ‘I Don’t Know’ Instead of Hallucinating
Creation OS is a local-first AI runtime that wraps local LLMs with a σ-gate — a measurement layer that scores each output across multiple uncertainty channels and decides ACCEPT, RETHINK, or ABSTAIN. The goal is to let local models refuse answers when uncertain instead of hallucinating.
Key Features and Setup
- Supports BitNet b1.58 2B-4T, Qwen3-8B Q4_K_M, Gemma 3 4B, and any GGUF model.
- Runs on a MacBook Air M4 8GB as primary machine — no cloud, no API, nothing leaves the device.
- Install:
git clone https://github.com/spektre-labs/creation-osthencd creation-os && bash scripts/quickstart.sh - Full path with local weights:
./scripts/install.shthen./cos chat
σ-Gate Measurements
The gate combines logprob, entropy, perplexity, consistency, semantic σ, conformal τ, session coherence, and meta-cognitive channels into a single verdict:
- ACCEPT → show answer
- RETHINK → regenerate
- ABSTAIN → refuse
Benchmark Results
TruthfulQA (same prompts and seeds):
|Mode |Accuracy|Coverage| |-------------|--------|--------| |BitNet only |0.261 |0.136 | |σ-pipeline |0.336 |0.171 |
+28.7% accuracy from selective regeneration on uncertain rows. LSD probe AUROC: 0.982 on TruthfulQA holdout, 0.960 on TriviaQA. ECE: 0.043. Wrong+confident: 0. Conformal bound: P(error | ACCEPT) ≤ α at α=0.80.
Negative results documented: σ is not dominant on HellaSwag or MMLU. Full details in CLAIM_DISCIPLINE.md.
Formal Verification
Lean 4: 6/6 sorry-free. Frama-C WP: 15/15 tier-1 discharged.
Example Command
./cos chat --once --prompt "What is 2+2?" --multi-sigma --verbose yields output like σ_peak=0.06 action=ACCEPT route=LOCAL σ_combined=0.184 conformal@α=0.80.
MCP Integration
Run python3 -m cos.mcp_sigma_server to expose σ on every response to any MCP-compatible client.
Limitations
σ is not a universal hallucination detector — strongest on factual QA; long-form needs more evaluation. Local model quality still depends on the base model.
📖 Read the full source: r/LocalLLaMA
👀 See Also
ClaudeDesk v4.2–4.3 Introduces Agent Teams Visualization and Repository Atlas Engine
ClaudeDesk v4.2–4.3 provides a desktop GUI for managing Claude Code CLI's Agent Teams and optimizes session setup with the Repository Atlas Engine.
Noren AI: Voice Extraction Tool Identifies Writing Patterns from Samples
Noren AI analyzes 5-10 writing samples to automatically generate a voice guide based on actual patterns, matching 90% of manually identified patterns and discovering additional ones.
Bypassing NemoClaw Sandbox Isolation for Local Nemotron 9B Agent
A developer bypassed NemoClaw's sandbox isolation to run a fully local agent using Nemotron 9B with tool calling on a single RTX 5090. The approach involved iptables configuration, a custom TCP relay, and real-time tool call translation.
Android CLI and Skills for AI Agent Development Workflows
Google released Android CLI with commands like android create and android sdk install, plus Android Skills GitHub repository with modular instruction sets. Internal benchmarks show 70% reduction in LLM token usage and 3x faster task completion.