OpenClaw

scrapling-mcp

Advanced web scraping with Scrapling — MCP-native guidance for extraction, crawling, and anti-bot handling. Use via mcporter (MCP) to call the `scrapling` MCP server for execution; this skill provides strategy, recipes, and best practices.

devbd10 来源安装数☆ 0 仓库 Stars

查看来源 ↗安装与使用 SKILL.md ↗

安装与使用

请根据来源仓库和你的 AI 工具说明安装此技能。先阅读技能文件和依赖要求，再添加到工作流。

SkillHub CLI

skillhub install devbd1/scrapling-mcp

打开来源仓库，查看说明和兼容环境后，按来源提供的命令或步骤安装。

将来源链接提供给你的 Agent，让它先读取说明、核对依赖，并向你说明将执行的安装步骤。

查看来源 ↗使用指南

技能说明

Scrapling MCP — Web Scraping Guidance

Source repo: https://github.com/DevBD1/openclaw-skill-scrapling-mcp

Guidance Layer + MCP Integration
Use this skill for strategy and patterns. For execution, call Scrapling's MCP server via mcporter.

Quick Start (MCP)

1. Install Scrapling with MCP support

pip install scrapling[mcp]
# Or for full features:
pip install scrapling[mcp,playwright]
python -m playwright install chromium

2. Add to OpenClaw MCP config

{
  "mcpServers": {
    "scrapling": {
      "command": "python",
      "args": ["-m", "scrapling.mcp"]
    }
  }
}

3. Call via mcporter

mcporter call scrapling fetch_page --url "https://example.com"

Execution vs Guidance

Task	Tool	Example
Fetch a page	mcporter	`mcporter call scrapling fetch_page --url URL`
Extract with CSS	mcporter	`mcporter call scrapling css_select --selector ".title::text"`
Which fetcher to use?	This skill	See "Fetcher Selection Guide" below
Anti-bot strategy?	This skill	See "Anti-Bot Escalation Ladder"
Complex crawl patterns?	This skill	See "Spider Recipes"

Fetcher Selection Guide

┌─────────────────┐     ┌──────────────────┐     ┌──────────────────┐
│   Fetcher       │────▶│ DynamicFetcher   │────▶│ StealthyFetcher  │
│   (HTTP)        │     │ (Browser/JS)     │     │ (Anti-bot)       │
└─────────────────┘     └──────────────────┘     └──────────────────┘
     Fastest              JS-rendered               Cloudflare, 
     Static pages         SPAs, React/Vue          Turnstile, etc.

Decision Tree

Static HTML? → Fetcher (10-100x faster)
Need JS execution? → DynamicFetcher
Getting blocked? → StealthyFetcher
Complex session? → Use Session variants

MCP Fetch Modes

fetch_page — HTTP fetcher
fetch_dynamic — Browser-based with Playwright
fetch_stealthy — Anti-bot bypass mo...

来源信息

Source: github
Repository: openclaw/skills
Path: skills/devbd1/scrapling-mcp
Version: —
Commit: 7ecc23afd22d9fffef8896f83af5b820194116a1
分类来源: rules
状态: active
可见性: public

这些信息来自收录时的来源分析，可能滞后；使用前请核对最新来源文件。

来源内容

SKILL.md

# Scrapling MCP — Web Scraping Guidance

Source repo: https://github.com/DevBD1/openclaw-skill-scrapling-mcp

> **Guidance Layer + MCP Integration**  
> Use this skill for **strategy and patterns**. For execution, call Scrapling's MCP server via `mcporter`.

## Quick Start (MCP)

### 1. Install Scrapling with MCP support
```bash
pip install scrapling[mcp]
# Or for full features:
pip install scrapling[mcp,playwright]
python -m playwright install chromium
```

### 2. Add to OpenClaw MCP config
```json
{
  "mcpServers": {
    "scrapling": {
      "command": "python",
      "args": ["-m", "scrapling.mcp"]
    }
  }
}
```

### 3. Call via mcporter
```
mcporter call scrapling fetch_page --url "https://example.com"
```

## Execution vs Guidance

| Task | Tool | Example |
|------|------|---------|
| Fetch a page | **mcporter** | `mcporter call scrapling fetch_page --url URL` |
| Extract with CSS | **mcporter** | `mcporter call scrapling css_select --selector ".title::text"` |
| Which fetcher to use? | **This skill** | See "Fetcher Selection Guide" below |
| Anti-bot strategy? | **This skill** | See "Anti-Bot Escalation Ladder" |
| Complex crawl patterns? | **This skill** | See "Spider Recipes" |

## Fetcher Selection Guide

```
┌─────────────────┐     ┌──────────────────┐     ┌──────────────────┐
│   Fetcher       │────▶│ DynamicFetcher   │────▶│ StealthyFetcher  │
│   (HTTP)        │     │ (Browser/JS)     │     │ (Anti-bot)       │
└─────────────────┘     └──────────────────┘     └──────────────────┘
     Fastest              JS-rendered               Cloudflare, 
     Static pages         SPAs, React/Vue          Turnstile, etc.
```

### Decision Tree
1. **Static HTML?** → `Fetcher` (10-100x faster)
2. **Need JS execution?** → `DynamicFetcher`
3. **Getting blocked?** → `StealthyFetcher`
4. **Complex session?** → Use Session variants

### MCP Fetch Modes
- `fetch_page` — HTTP fetcher
- `fetch_dynamic` — Browser-based with Playwright
- `fetch_stealthy` — Anti-bot bypass mo...

搜索