Running Google Gemma 4 26B-A4B Locally with LM Studio 0.4.0 Headless CLI
What LM Studio 0.4.0 Adds for Local AI
LM Studio 0.4.0 fundamentally changes the architecture by extracting the core inference engine into llmster, a standalone server. This enables running LM Studio entirely from the command line using the new lms CLI, eliminating the need for the GUI. The update makes it usable on headless servers, in CI/CD pipelines, SSH sessions, or for terminal-focused developers.
Key Features in 0.4.0
- llmster daemon: A background service that manages model loading and inference without the desktop app
- lms CLI: Full command-line interface for downloading, loading, chatting, and serving models
- Parallel request processing: Continuous batching instead of sequential queuing, allowing multiple requests to the same model to run concurrently
- Stateful REST API: A new /v1/chat endpoint that maintains conversation history across requests
- MCP integration: Local Model Context Protocol support with permission-key gating
Why Gemma 4 26B-A4B for Local Use
Google's Gemma 4 26B-A4B uses a mixture-of-experts architecture with 128 experts plus 1 shared expert, but only activates 8 experts (3.8B parameters) per token. This means it runs well on hardware that couldn't handle a dense 26B model. On a 14" MacBook Pro M4 Pro with 48GB unified memory, it fits comfortably and generates at 51 tokens/second.
The model scores 82.6% on MMLU Pro and 88.3% on AIME 2026, close to the dense 31B variant (85.2% and 89.2%) while running dramatically faster. It achieves an Elo score of ~1441, competing with models like Qwen 3.5 397B-A17B (~1450 Elo) that require 100-600B total parameters.
Key capabilities include 256K max context, vision support for analyzing screenshots and diagrams, native function/tool calling, and reasoning with configurable thinking modes.
Practical Setup
The article walks through installing the lms CLI and setting up Gemma 4 26B-A4B for local inference that can be used with Claude Code. The author notes significant slowdowns when used within Claude Code from their experience.
📖 Read the full source: HN AI Agents
👀 See Also
ClawsifyAI Agent Handles Email, Research, and Brainstorming Tasks
A developer tested ClawsifyAI, an AI agent-style claw bot, for a week and found it handled emails, research, repetitive work, and brainstorming. The agent provides clear feedback, practical solutions, and sometimes better ideas than originally planned.
MCP-Loci: Local Persistent Memory Server for Claude and MCP-Compatible AI
MCP-Loci is a persistent memory server that solves Claude's session-based memory limitation with five tools: remember, recall, forget, synthesize, and health. It uses hybrid BM25 keyword matching and semantic embeddings for accurate recall without requiring API keys.
Marmy: A Self-Hosted Framework for Managing AI Coding Agents Remotely
Marmy is an open-source, MIT-licensed framework built with Claude Code that lets developers manage AI coding agents and tmux sessions from a mobile app. It includes a Rust agent for development machines and a React Native app for remote control.
Claude Academy: A Free Coding Bootcamp That Runs Inside Claude Desktop
A developer has built Claude Academy, a free coding bootcamp that operates entirely within Claude Desktop's Code tab. The system uses three commands to deliver 64 structured lessons across web development fundamentals, with progress tracking and real project building.