Webclaw

webclaw https://example.com --format markdown

# Example Domain

This domain is for use in illustrative examples in documents.

You may use this domain in literature without prior coordination or asking for permission.

npx create-webclaw

brew tap 0xMassi/webclaw
brew install webclaw

docker run --rm ghcr.io/0xmassi/webclaw https://example.com

cargo install --git https://github.com/0xMassi/webclaw.git webclaw-cli
cargo install --git https://github.com/0xMassi/webclaw.git webclaw-mcp

webclaw https://stripe.com --format markdown

webclaw https://docs.anthropic.com --format llm

webclaw https://example.com/blog/post --only-main-content

webclaw https://example.com \
  --include "article, main, .content" \
  --exclude "nav, footer, .sidebar, .ad"

webclaw https://docs.rust-lang.org --crawl --depth 2 --max-pages 50

webclaw https://github.com --brand

webclaw https://example.com/pricing --format json > pricing-old.json
webclaw https://example.com/pricing --diff-with pricing-old.json

npx create-webclaw

{
  "mcpServers": {
    "webclaw": {
      "command": "~/.webclaw/webclaw-mcp"
    }
  }
}

Scrape these competitor pricing pages and summarize the differences.

Crawl this documentation site and prepare clean context for a RAG index.

Extract the brand colors, fonts, and logos from this company website.

npm install @webclaw/sdk
pip install webclaw
go get github.com/0xMassi/webclaw-go

import { Webclaw } from "@webclaw/sdk";

const client = new Webclaw({ apiKey: process.env.WEBCLAW_API_KEY! });

const page = await client.scrape({
  url: "https://example.com",
  formats: ["markdown"],
  only_main_content: true,
});

console.log(page.markdown);

from webclaw import Webclaw

client = Webclaw(api_key="wc_your_key")

page = client.scrape(
    "https://example.com",
    formats=["markdown"],
    only_main_content=True,
)

print(page.markdown)

curl -X POST https://api.webclaw.io/v1/scrape \
  -H "Authorization: Bearer $WEBCLAW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "formats": ["markdown"],
    "only_main_content": true
  }'

export WEBCLAW_API_KEY=wc_your_key

webclaw https://example.com --cloud

webclaw/
  crates/
    webclaw-core     HTML to markdown, text, JSON, and LLM-ready output
    webclaw-fetch    Fetching, crawling, batching, and mapping
    webclaw-llm      Local and hosted LLM provider support
    webclaw-pdf      PDF text extraction
    webclaw-mcp      MCP server for AI agents
    webclaw-cli      Command-line interface

URL:
Command or API request:
Expected output:
Actual output:
Format used: markdown / llm / text / json / html
CLI, MCP, SDK, or API: