From the author
Expert guide for building web scrapers and crawlers using Crawlee (JavaScript/TypeScript and Python). Use this skill whenever the user wants to: scrape a website, build a web crawler, extract data from web pages, automate browser navigation, handle anti-bot blocking, manage proxies or sessions for scraping, use Playwright/Puppeteer/Cheerio/BeautifulSoup for web data extraction, crawl sitemaps, download files from URLs, or deploy a scraper to the cloud. Trigger even for loosely related phrases like "get data from a website", "automate browser", "scrape prices", "extract links", "crawl URLs", or "bypass bot detection". Covers CheerioCrawler, PlaywrightCrawler, PuppeteerCrawler, HttpCrawler, JSDOMCrawler (JS), and BeautifulSoupCrawler, ParselCrawler, PlaywrightCrawler (Python).
Install
npx skills add LeoYeAI/agent-master-skills --skill crawlee -a claude-code
Installs to .claude/skills/crawlee/ · Each command fetches the skill straight from GitHub with the open skills CLI.
Byte-identical to the scanned commit, with the repository’s licence and a SOURCE.txt.
Want your agent to suggest skills while you type? See how to use skills from your IDE.
Safety checks on SKILL.md
No warnings.
Why 89
aiqbench computes the trust score from public signals. It is not a security audit, so read a skill before you install it.
Similar skills
Closest to this one by what they do and when they are used.
- scrapy-web-scrapingMindrally/skillsExpert guidance for building web scrapers and crawlers using the Scrapy Python framework with best practices for spider development, data extraction, and pipeline management.Browser★ 289
- browser-automationalirezarezvani/claude-skillsUse for browser automation, scraping, form filling or structured extraction from web pages. Builds scrapers with scraping_toolkit.py and form flows with form_automation_builder.py. Browser testing: playwright-pro.Browser★ 29.1K
- web-scrapinga5c-ai/babysitterBuild robust web scrapers — HTML parsing, JavaScript rendering, rate limiting, anti-bot evasion, and structured data extraction.Browser★ 1.9K
- website-scrapingmoonlight-lupin/agent-skillsGeneric playbook for extracting structured data from any website — hotel prices, flight fares, e-commerce listings, real estate, jobs, competitor product catalogues, anything where the goal is to turn one or more URLs into clean records on disk. Use this skill whenever the user mentions scraping, harvesting, extracting, or pulling data from a website; names a specific site or competitor they want data from; asks how to handle JavaScript-rendered pages, Cloudflare blocks, bot detection, Turnstile challenges, or Playwright; wants to monitor prices over time; needs to automate a copy-paste research task; or says things like "just get the data from X" or "I need a script that grabs N from site Y". Covers recon, picking the lightest extraction tool that works, surviving anti-bot defences, and writing clean JSONL output with a run manifest. Scope is scraping only — landing data in a database, scheduling, and downstream pipeline work are explicitly out of scope and live in other tools.Browser★ 114
- Puppeteer Web Scraperagentskillexchange/skillsHeadless Chrome scraping via Puppeteer with automatic cookie handling, JavaScript rendering, and Cheerio-based DOM extraction. Handles infinite scroll and lazy-loaded content.Browser★ 60
- puppeteer-automationMindrally/skillsExpert guidance for browser automation using Puppeteer with best practices for web scraping, testing, screenshot capture, and JavaScript execution in headless Chrome.Browser★ 289
Ratings & reviews
—
no reviews
Tried this skill? Rate it to help others. Sign in
Discussion
Join the discussion. Sign in