Fixing OpenClaw's Blind Spots: Building a Sitemap to Fetch All Anthropic Blogs
A reader noticed that OpenClaw's summary of Anthropic's tech blogs was missing some posts — specifically one from April 8th. The root cause: Anthropic hosts their blogs at multiple distinct URLs, and OpenClaw's browser tool doesn't crawl all of them. The fix involved generating a sitemap by having the agent click every link to build a complete list, then feeding that sitemap into the summarization pipeline. The solution is packaged as a skill for easy reuse, though it does cost extra tokens to generate the sitemap due to the link-clicking process.
Key Takeaway
- Anthropic's blog URLs are not unified — they spread content across different hostnames, making naive scraping incomplete.
- OpenClaw's browser tool for RSS/blog aggregation doesn't auto-discover all pages; it needs a complete URL list upfront.
- Manually generating a sitemap by traversing all links works, but it consumes tokens proportional to the site's size.
This is a pragmatic workaround rather than a deep fix, but it's a useful pattern for any site with fragmented content. If you need reliable coverage of multiple-blog-host setups, pre-seeding with a sitemap is the simplest approach.
📖 Read the full source: r/openclaw
👀 See Also
Open-Sourced the-vibe-stack: Markdown Rules to Maintain Claude Code Consistency
A developer has open-sourced 'the-vibe-stack' — a set of Markdown rules designed to keep Claude Code on track during long sessions by enforcing a rigid schema. The approach aims to reduce logic drift and token waste while ensuring predictable output.
Browser-native real-time coherence control system for Claude with SDE bands and Kalman filtering
A developer has built a real-time coherence control harness that runs entirely as a Claude artifact in the browser, treating conversation as a stochastic process with live Monte Carlo SDE paths, dual Kalman filtering, and behavioral signal detection.
AI Roundtable: Tool for Comparing 200+ AI Models on Structured Questions
AI Roundtable is a free tool that lets users pose questions with defined answer options, select up to 50 models from a pool of 200+, and get structured responses under identical conditions. It also includes a debate feature where models can see each other's reasoning and a reviewer model that summarizes transcripts.
Claude Code skill combines DeepMind Aletheia and Anthropic harness approaches
A Claude Code skill implements a Planner→Generator→Evaluator→Reviser pipeline that synthesizes DeepMind's Aletheia math research agent with Anthropic's multi-agent coding architecture, adding blind pre-analysis where the evaluator reasons about correct approaches before seeing candidate code.