ClawVibe Hands-Free iOS Voice Assistant for AI Agents with On-Device STT/TTS

ClawVibe is a native iOS voice assistant that lets you talk to your AI agent completely hands-free — designed for use while driving. The developer built it after hitting the problem of a 45-minute commute where phone-in-hand solutions weren't safe and web UIs required tapping. After two months of work, the app is now in TestFlight beta and will be free on the App Store.

Key Technical Decisions

The core architectural choice: move all audio processing on-device. The initial version streamed voice audio to a server for transcription, but ran into latency spikes and packet loss on mobile networks. The fix was to handle speech recognition and text-to-speech on the phone, sending only the plain text transcript over the network. This eliminates connectivity issues; it even works in areas with barely any reception.

On-device processing posed a backend challenge: most ML models use Apple's Metal framework (GPU acceleration), which Apple doesn't allow to run in the background. So the app falls back to standard CoreML without GPU acceleration to keep the voice detection and processing alive while the screen is off — a necessary trade-off for hands-free use.

Background noise (GPS directions, conversations, radio) was another pain point. The solution is voice biometrics: the app learns your voice profile and only sends recognized speech to the AI. GPS announcements and other noise are ignored.

Features

On-device speech recognition: Transcribed locally; no audio sent to Apple or Google. Only the text goes to your AI backend.
On-device TTS: Multiple voice options, output through phone or CarPlay speakers.
Always-on voice detection: No button press or wake word; it knows when you're talking.
CarPlay integration: Full hands-free in the car — the original reason for building it.
Primary voice recognition: Only your registered voice triggers the AI; background sounds are filtered out.
GPS context: Location data sent with each message for location-aware responses.

Backend & Availability

You need your own AI backend to connect ClawVibe to. It supports several setups, with more integrations in progress. If you already run an OpenClaw instance, you're good. If not, you can trial the app with a local on-device model.

The app is currently in TestFlight beta. A free App Store version will provide the full core voice experience. A Premium tier (coming later) will add more voices, multiple voice profiles, expanded CarPlay UI, Apple Watch support, and more.