boxBot: An Open-Source Smart Speaker Powered by Claude and Hailo AI
A developer known as FunScore645 built boxBot, a smart speaker that uses Claude for agent-driven hardware control instead of rigid rules. The project is open-sourced at github.com/dv-hart/boxbot and is still a work in progress but already useful for daily family tasks.
Hardware and Build
The hardware stack includes:
- Raspberry Pi (model not specified)
- Hailo AI accelerator (for edge inference, presumably)
- Respeaker microphone array
- Raspberry Pi camera
- Waveshare display
- Speakers
The enclosure is handcrafted from walnut, with passive cooling—the developer used Claude to calculate thermal loads and dissipation rates for the design.
Agent-Driven Architecture
Instead of complex conversation rules, the agent (likely Claude-based) controls the hardware directly through a custom Python SDK. Key capabilities:
- Dynamic muting: The agent decides when to mute the mic to ignore background noise or side conversations.
- Barge-in handling: Agent-driven logic manages interruptions without explicit rules.
- Multi-channel response: Can reply via voice or WhatsApp, chosen by the agent.
- Widget-based display: The screen uses a JSON widget framework—the agent reads and manipulates widgets via JSON without screenshots.
Skill System and Flexibility
The agent uses a small set of tools, with most logic routed through Python scripts and the boxBot SDK. Skills are defined in Markdown files. Example: when the developer added a family calendar event, his wife wanted a text notification. He added one line in a .md file, and the agent updated the calendar skill to send a WhatsApp message automatically on new events.
Memory and Security
The developer mentions a custom memory system and security considerations (camera+mic in a living room) but deferred details to comments. The agent is tightly coupled with hardware for security.
Who It's For
Developers interested in building AI-driven hardware projects with agent-controlled peripherals and open-source foundations.
📖 Read the full source: r/ClaudeAI
👀 See Also
Lemonade by AMD: Open Source Local LLM Server for GPU and NPU
Lemonade is an open source local AI server that runs text, image, and speech models on GPUs and NPUs. It's OpenAI API compatible, supports multiple models simultaneously, and has a 2MB native C++ backend.
CC-Wiki: Turn Claude Code Sessions into a Shareable Quartz Knowledge Base
CC-Wiki converts your ~/.claude session history into a Quartz-based knowledge base. One command installs it; running /cc-wiki inside a Claude Code session packages the conversation.
Persistent Side Panel for Claude Code with Autonomous Content Management
A developer built a TUI panel that sits in an iTerm2 split pane next to the terminal, featuring three fixed panels that Claude autonomously manages to show relevant content like code, diagrams, and status updates.
Brunnfeld Agentic World: Multi-Agent Medieval Economy Simulation Without Behavioral Prompts
A TypeScript simulation where 20 LLM agents autonomously trade in a medieval village economy with no behavioral instructions, goals, or trading strategies. Agents receive ~200 token perceptions each tick and interact through a deterministic engine handling physics, recipes, and market mechanics.