Data Scraper Agent

This is a complete starter kit for building scheduled scrapers that collect public data and let an LLM score or summarize the results. It's architected around GitHub Actions for free hosting, Gemini Flash for free inference with automatic model fallback when you hit rate limits, and storage adapters for Notion, Sheets, or Supabase. The batch processing approach keeps you inside free tier quotas, and there's a feedback loop so the agent learns from your decisions over time. Good for job monitoring, price tracking, or any repetitive data collection task where you want AI enrichment without paying for infrastructure. The codebase includes concrete source connectors, a learning memory system, and backfill scripts for existing data.

skills.sh installs: 5.6K
GitHub stars: 300.3K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add affaan-m/ecc --skill data-scraper-agent --agent claude-code

Installs into .claude/skills of the current project.

Files

Data Scraper Agent

Build a production-ready, AI-powered data collection agent for any public data source. Runs on a schedule, enriches results with a free LLM, stores to a database, and improves over time.

Stack: Python · Gemini Flash (free) · GitHub Actions (free) · Notion / Sheets / Supabase

When to Activate

User wants to gather or monitor any public website or API
User says "build a bot that checks...", "monitor X for me", "collect data from..."
User wants to track jobs, prices, news, repos, sports scores, events, listings
User asks how to automate data collection without paying for hosting
User wants an agent that gets smarter over time based on their decisions

Core Concepts

The Three Layers

Every data collection agent has three layers:

COLLECT → ENRICH → STORE
  │           │        │
Scraper    AI (LLM)  Database
runs on    scores/   Notion /
schedule   summarises Sheets /
           & classifies Supabase