Kelet: Automated Root Cause Analysis for AI Agents
What Kelet Does
Kelet is a service that continuously monitors AI agents and LLM applications in production to automatically identify why they fail. Instead of agents crashing with clear errors, they often give wrong answers quietly, requiring manual trace analysis. Kelet automates this investigation by clustering failure patterns across thousands of sessions.
How It Works
- You connect your traces and signals (user feedback, edits, clicks, sentiment, LLM-as-a-judge, etc.)
- Kelet processes those signals and extracts facts about each session
- It forms hypotheses about what went wrong in each case
- It clusters similar hypotheses across sessions and investigates them together
- It surfaces a root cause with a suggested fix you can review and apply
The key insight: individual session failures look random, but when you cluster the hypotheses, failure patterns emerge.
Integration Options
Three ways to integrate:
- Kelet Skill for coding agents: Scans your codebase, discovers where signals should be collected, and sets everything up automatically
- Python SDK:
pip install kelet - TypeScript SDK:
npm install kelet
Manual setup requires adding two lines to your agent code. Kelet is fully OpenTelemetry-compliant, so any OTEL-instrumented agent works out of the box.
Supported Frameworks and Platforms
Works with: OpenTelemetry, Langfuse, Mixpanel, OpenAI, Anthropic, LangChain, pydantic AI SDK, CrewAI, Strands, Agno, Mastra, PostHog, LangGraph, AutoGen, LlamaIndex, Haystack, Semantic Kernel, and Gemini APIs.
Works with any agent or LLM application where you own the code: agentic loops, multi-step workflows, RAG pipelines, chatbots, autonomous agents.
Two situations where Kelet isn't the right fit:
- If you use AI tools built by others (Cursor, Claude Code, Copilot as a developer)
- If you're building a skill or plugin inside an existing agentic platform
Technical Details
- Runs on Kelet's servers (SOC 2 certified)
- Continuously ingests traces 24/7
- LLM tokens for analysis are covered by Kelet (don't touch your model API bill)
- Pricing based on usage (see kelet.ai/pricing)
- Currently free during beta (no credit card required)
Performance Metrics
From pilot cohort data:
- 73% of teams had failures nobody noticed (Kelet found them)
- 14.3 minutes median time from trace ingestion to prompt patch
- 33K+ sessions analyzed across design partner deployments
📖 Read the full source: HN AI Agents
👀 See Also
MemAware Benchmark Tests AI Memory Beyond Keyword Search
MemAware is a benchmark with 900 questions across 3 difficulty levels that tests whether AI assistants with memory can surface relevant context when queries don't hint at it. Results show BM25 search scored 2.8% vs 0.8% with no memory, while vector search drops to 0.7% on cross-domain connections.
YourMemory: AI memory with biological decay hits 59% recall on LoCoMo-10
YourMemory gives AI agents persistent memory using Ebbinghaus forgetting curve and graph-enhanced retrieval. Benchmarked at 59% Recall@5 on LoCoMo-10, 2× better than Zep Cloud.
Browser CLI: A Token-Efficient Browser Automation Tool for AI Coding Agents
Browser CLI is a persistent headless Chromium daemon that provides browser automation via plain Bash commands, achieving ~95% token savings compared to Playwright MCP by reducing calls from ~1,500 tokens to ~75 tokens.
Developer Builds Scheme Compiler to WASM Using AI in 4 Days
A developer created Puppy Scheme, a Scheme compiler that targets WebAssembly, in about 4 days using AI assistance. The compiler supports 73% of R5RS and R7RS, uses WASM GC, and achieved compilation time improvements from 3½ minutes to 11 seconds overnight.