Playwright Web Scraper

Playwright Web Scraper

Extract structured data from multiple web pages with respectful, ethical crawling practices.

When to Use This Skill

Use when extracting structured data from websites with "scrape data from", "extract information from pages", "collect data from site", or "crawl multiple pages".

Do NOT use for testing workflows (use playwright-e2e-testing), monitoring errors (use playwright-console-monitor), or analyzing network (use playwright-network-analyzer). Always respect robots.txt and rate limits.

Quick Start

Scrape product listings from an e-commerce site:

// 1. Validate URLs
python scripts/validate_urls.py urls.txt

// 2. Scrape pages with rate limiting
const results = [];
for (const url of urls) {
  await browser_navigate({ url });
  await browser_wait_for({ time: Math.random() * 2 + 1 }); // 1-3s delay

  const data = await browser_evaluate({
    function: `
      Array.from(document.querySelectorAll('.product')).map(el => ({
        title: el.querySelector('.title')?.textContent?.trim(),
        price: el.querySelector('.price')?.textContent?.trim(),
        url: el.querySelector('a')?.getAttribute('href')
      }))
    `
  });