Architecture for OpenClaw Health-Tracking Agent with Telegram

A Reddit user is planning to build an OpenClaw agent to track health goals (prediabetes, diet, exercise, sleep) via Telegram. The agent would handle daily prompts, photo analysis, feedback, and trend reports. Below is a technical breakdown based on the source post and practical knowledge of similar workflows.

Architecture Recommendation

The core pattern: Telegram Bot → OpenClaw Agent (with LLM backend) → Database → Reporting cron job.

• Telegram Bot: Use python-telegram-bot or node-telegram-bot-api for receiving messages and sending proactive reminders. Schedule reminders via a background cron (e.g., APScheduler or CronJob).
• OpenClaw Agent: The agent wraps an LLM (GPT-4o or Gemini 2.0 Flash for cost) with system prompt that:
  • For meal photos: extracts dish name, portion estimate, and Mediterranean diet compliance.
  • For exercise/sleep screenshots: parses structured data (e.g., steps, sleep duration).
  • Returns 3-4 sentence feedback per update.
• Storage: PostgreSQL or SQLite for simplicity. See DB schema below.
• Reporting: Weekly cron runs an aggregation query + calls LLM to generate natural language report.

Database Structure

-- Users table
CREATE TABLE users (
  user_id INTEGER PRIMARY KEY,
  telegram_id TEXT UNIQUE,
  goals TEXT,          -- JSON: diet, exercise, sleep, water targets
  blood_test_results TEXT -- JSON, optional
);

-- Daily entries (one per type per day)
CREATE TABLE entries (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  user_id INTEGER,
  entry_date DATE,
  type TEXT CHECK(type IN ('sleep', 'water', 'exercise', 'meal')),
  raw_text TEXT,
  raw_image BLOB,       -- or S3 URL
  structured_data JSON, -- e.g., {"hours":7.5,"quality":"good"}
  feedback TEXT,
  created_at TIMESTAMP
);
-- Weekly/monthly summaries (precomputed)
CREATE TABLE reports (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  user_id INTEGER,
  period_type TEXT CHECK(period_type IN ('weekly', 'monthly')),
  period_start DATE,
  report_text TEXT,
  generated_at TIMESTAMP
);

Image Interpretation Reliability

OpenClaw agents rely on the underlying LLM's vision capabilities. GPT-4o and Gemini 2.0 Flash are good at identifying Mediterranean diet compliance (recognizing vegetables, legumes, olive oil) but struggle with precise portion sizes. For screenshots from gym/sleep apps, text extraction via OCR works well if the image is clear. Budget tip: Use Gemini 2.0 Flash ($0.10/1M input tokens) for images and GPT-4o-mini for text.

Budget Feasibility ($50/month)

Realistic with optimization. Example breakdown:

• LLM inference: ~40-50 calls/day (daily check-ins, photos, reports). Using Gemini 2.0 Flash ($0.10/1M input, $0.40/1M output) → ~$5/month.
• Database: SQLite on a $5 VPS (e.g., Hetzner CX22) → $5/month.
• Telegram bot hosting: Same VPS → included.
• Total: ~$10-15/month. Leaves room for GPT-4o fallback (20 calls, ~$8) or occasional image analysis with Gemini Pro Vision.

Risks & Missing Features

• Medical advice boundary: System prompt must strictly forbid diagnosis or changing professional advice. Store disclaimer per user.
• Privacy: Encrypt health data at rest (AES-256). Consider HIPAA compliance if serving others.
• Feedback loop: User may ignore reminders. Design escalating notifications (e.g., after 3 ignored water prompts, send a funny GIF).
• Weekly report generation: LLM hallucination risk. Prefer deterministic aggregation for metrics, use LLM only for narrative.

Existing components: You can adapt OpenClaw's built-in Telegram connector and PostgreSQL template. No out-of-box health agent exists — you'll need to build the prompt and DB schema from scratch.