Building a Self-Improving Knowledge System with Claude Code and Obsidian

Architecture Overview

A developer created a self-improving knowledge system that runs 25 automated tools hourly to solve Claude Code's session amnesia problem. The system connects Claude Code to an Obsidian vault (~350 notes) with local semantic search, knowledge graphs, and automated processing.

Technical Stack

• Obsidian vault as the knowledge store
• Claude Code (Opus) as the AI that reads/writes the vault
• Ollama + bge-m3 (1024-dim embeddings, RTX 3080) for local semantic search
• SQLite (better-sqlite3) for search index, graph DB, codebase index
• Express server for a React dashboard
• 2 MCP servers giving Claude native vault + graph access
• Windows Task Scheduler running everything hourly

Tool Layers and Functions

Layer 1: Data Collection

• vault-live-sync.mjs: Watches Claude Code JSONL sessions in real-time, converts to Obsidian notes
• vault-sync.mjs: Hourly sync of Supabase stats, AutoPost status, git activity, project context
• vault-voice.mjs: Voice-to-vault with Whisper transcription + Sonnet summary of audio files
• vault-clip.mjs: Web clipping from RSS feeds + Brave Search topic monitoring + AI summary
• vault-git-stats.mjs: Git metrics including commit streaks, file hotspots, hourly distribution

Layer 2: Processing & Intelligence

• vault-digest.mjs: Daily digest aggregating all sessions into one readable page
• vault-reflect.mjs: Uses Sonnet to extract key decisions from sessions, auto-promotes to MEMORY.md
• vault-autotag.mjs: AI auto-tagging with Sonnet suggesting tags + wikilink connections
• vault-schema.mjs: Frontmatter validator with 10 note types, compliance reporting, auto-fix mode
• vault-handoff.mjs: Generates machine-readable handoff.json (survives compaction better than markdown)
• vault-session-start.mjs: Assembles optimal context package for new Claude sessions

Layer 3: Search & Retrieval

• vault-search.mjs: FTS5 + chunked semantic search (512-char chunks, bge-m3 1024-dim). Flags include --semantic, --hybrid, --scope, --since, --between, --recent. Includes retrieval logging + heat map.
• vault-codebase.mjs: Indexes 2,011 source files: exports, routes, imports, JSDoc
• vault-graph.mjs: Knowledge graph with 375 nodes, 275 edges, betweenness centrality, community detection, link suggestions
• vault-graph-mcp.mjs: Graph as MCP server with 6 tools (search, neighbors, paths, common, bridges, communities) Claude can use natively

Layer 4: Self-Improvement

• vault-patterns.mjs: Weekly patterns including momentum score (1-10), project attention %, velocity trends, token burn ($), stuck detection, frustration/energy tracking, burnout risk
• vault-spaced.mjs: Spaced repetition (FSRS) with 348 notes tracked, priority-based review scheduling
• vault-prune.mjs: Hot/warm/cold decay scoring, auto-archives stale notes, flags never-retrieved notes
• vault-contradict.mjs: Contradiction detection with rule-based (stale references, metric drift, date conflicts) + AI-powered (Sonnet compares related docs)
• vault-research.mjs: Autonomous research with Brave Search + Sonnet, scheduled topic monitoring

Layer 5: Visualization & Monitoring

• vault-canvas.mjs: Auto-generates Obsidian Canvas files from knowledge graph (5 modes: full map, per-project, hub-centered, communities, daily)
• vault-heartbeat.mjs: Proactive agent that gathers state from all services, uses Sonnet to reason about what needs attention

The system was built by a solo dev agency owner who runs 4 interconnected projects, manages 64K business leads, and conducts hundreds of Claude Code sessions per week. The tools are all Node.js ES modules with zero external dependencies beyond what's already in the repo.