Scrapling integrated as OpenClaw's scraping backbone
Scrapling, an open-source scraping library, has been integrated into OpenClaw as its core scraping backbone. This addresses common issues with AI agents struggling with real-world web data, broken scrapers, Cloudflare walls, and selectors that fail when sites update.
Key features and capabilities
Unlike most scraping tools that rely on hardcoded selectors that break when websites redesign, Scrapling learns the structure of a page and adjusts automatically when things change without requiring code rewrites.
- 774x faster than BeautifulSoup with Lxml
- Works across HTTP and full browser automation
- Supports CSS, XPath, text, and regex selectors
- Async sessions for parallel scraping
- Includes a CLI so non-developers can use it without writing code
Getting started
Installation is straightforward:
pip install "scrapling[ai]"The library is fully open source under a BSD-3 license and has been gaining traction on GitHub.
📖 Read the full source: r/openclaw
👀 See Also
Fullerenes: Open-source persistent memory layer for coding agents cuts tokens by 64% on SWE-bench
Fullerenes uses a local SQLite knowledge graph built via Tree-sitter to give coding agents like Claude Code persistent memory, reducing token usage by 64% on SWE-bench and up to 96.6% on internal benchmarks.
Marmy: A Self-Hosted Framework for Managing AI Coding Agents Remotely
Marmy is an open-source, MIT-licensed framework built with Claude Code that lets developers manage AI coding agents and tmux sessions from a mobile app. It includes a Rust agent for development machines and a React Native app for remote control.
Exploring macOS's sandbox-exec for Secure Application Execution
sandbox-exec is a macOS command-line utility that allows applications to run in a restricted environment. Learn how to utilize it with custom sandbox profiles.
Clawmates: OpenClaw, but for Teams
New project brings multi-user OpenClaw deployment with shared knowledge, cost visibility, and admin controls.