On this page
You want your AI to scrape the web. Here is how to make that happen in under a minute.
Prerequisites
- Node.js 18+ -- check with
node --version - Claude Code -- install with
npm install -g @anthropic-ai/claude-code
That is it. Let us go.
Step 1: Install CrawlForge
Step 2: Get Your Free API Key
- Go to crawlforge.dev/signup
- Create your account (no credit card required)
- Copy your API key (starts with
cf_live_)
You now have 1,000 free credits to use.
Step 3: Configure Claude Code
Run the setup wizard:
Enter your API key when prompted. The wizard verifies your connection and configures Claude Code automatically.
Alternative: Set the environment variable manually:
You Are Ready. Try These Examples.
Open Claude Code and paste any of these prompts:
Fetch a Web Page (1 credit)
Fetch https://news.ycombinator.com and show me the top 5 stories
Claude uses fetch_url to grab the page, then parses the HTML to extract story titles and links.
Search the Web (5 credits)
Search the web for "best MCP servers for web scraping 2026" and summarize the top 3 results
Claude uses search_web to query Google, then summarizes each result.
Extract Structured Data (2 credits)
Extract all product names and prices from https://books.toscrape.com
Claude uses scrape_structured with CSS selectors to pull structured data into a clean table.
Deep Research (10 credits)
Research the current state of the MCP ecosystem: key players, adoption stats, and where it is heading
Claude uses deep_research to query multiple sources, cross-reference claims, and produce a cited report.
Stealth Scrape (5 credits)
Use stealth mode to fetch the pricing page from cloudflare.com
Claude uses stealth_mode with residential proxies and browser fingerprint rotation to bypass anti-bot protections.
Credit Costs at a Glance
| Tool | Credits | Use Case |
|---|---|---|
| fetch_url, extract_text, extract_links | 1 | Basic page fetching |
| scrape_structured, extract_content | 2 | Data extraction |
| analyze_content, map_site | 3 | Analysis and discovery |
| search_web, crawl_deep, batch_scrape | 5 | Search and bulk operations |
| deep_research | 10 | Multi-source AI research |
Your 1,000 free credits give you 1,000 basic fetches, 200 searches, or 100 deep research operations.
Next Steps
- Read the launch announcement for the full feature overview
- Browse the API reference for all 18 tools
- Check out the complete MCP guide for advanced patterns
- Set up Claude Desktop if you prefer the GUI
Questions? Join our Discord or open an issue on GitHub. Happy scraping!
