Fixing OpenClaw's Blind Spots: Building a Sitemap to Fetch All Anthropic Blogs
A reader noticed that OpenClaw's summary of Anthropic's tech blogs was missing some posts — specifically one from April 8th. The root cause: Anthropic hosts their blogs at multiple distinct URLs, and OpenClaw's browser tool doesn't crawl all of them. The fix involved generating a sitemap by having the agent click every link to build a complete list, then feeding that sitemap into the summarization pipeline. The solution is packaged as a skill for easy reuse, though it does cost extra tokens to generate the sitemap due to the link-clicking process.
Key Takeaway
- Anthropic's blog URLs are not unified — they spread content across different hostnames, making naive scraping incomplete.
- OpenClaw's browser tool for RSS/blog aggregation doesn't auto-discover all pages; it needs a complete URL list upfront.
- Manually generating a sitemap by traversing all links works, but it consumes tokens proportional to the site's size.
This is a pragmatic workaround rather than a deep fix, but it's a useful pattern for any site with fragmented content. If you need reliable coverage of multiple-blog-host setups, pre-seeding with a sitemap is the simplest approach.
📖 Read the full source: r/openclaw
👀 See Also
Exploring Clawe: Open-source Multi-agent Coordination System
Clawe is an open-source tool allowing for efficient multi-agent coordination, offering features like scheduling, task management, and real-time notifications.
Claude Growth Skill: Structured B2B SaaS Growth Playbooks for AI Coding Agents
A developer built a Claude Growth Skill containing 6 battle-tested playbooks based on 5 SaaS case studies, $90M ARR partnerships, and 1,800 user interviews. It structures B2B SaaS growth work from PMF validation to channel ecosystem and sales.
pop-pay MCP server adds payment guardrails for Claude Code agents
pop-pay is an MCP server that lets Claude Code agents handle purchases without exposing credit card numbers. It uses CDP injection to place virtual card credentials directly into payment iframes, with Claude only receiving masked confirmation numbers.
sourcecode: Open-Source CLI to Compress Large Java/Spring Monorepos for Claude
sourcecode CLI reduces a ~4k-file Java/Spring monorepo from ~3M tokens to 1.7k tokens (compact mode). Currently focuses on context compression, git hotspot detection, and symbol lookup.