Data Engineer

This one gives you solid patterns for building ETL pipelines and data infrastructure in TypeScript. You get three main approaches: daily batch syncs with full extract/transform/load cycles, incremental delta syncs that only grab what changed since last run, and real-time event processing with Kafka. The transformers are practical too, covering the usual data cleaning (emails, phones, names), enrichment with calculated fields like lifecycle stages, and SQL-based aggregations for analytics. It's built around Next.js with Prisma and includes actual cron job setup. Good if you're moving data between APIs and databases without wanting to spin up Airflow.

skills.sh installs: 181
GitHub stars: 39
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add daffy0208/ai-dev-standards --skill data-engineer --agent claude-code

Installs into .claude/skills of the current project.

Files

Data Engineer Skill

I help you build robust data pipelines, ETL processes, and data infrastructure.

What I Do

Data Pipelines:

Extract, Transform, Load (ETL) processes
Data ingestion from multiple sources
Batch and real-time processing
Data quality validation

Data Infrastructure:

Database schema design
Data warehousing
Caching strategies
Data replication

Analytics:

Data aggregation
Metrics calculation
Report generation
Data export

ETL Patterns

Pattern 1: Simple ETL Pipeline

Use case: Daily sync from external API to database

// lib/etl/daily-sync.ts

interface RawCustomer {
  id: string
  full_name: string
  email_address: string
  signup_date: string
}

interface Customer {
  id: string
  name: string
  email: string
  signupDate: Date
}

export async function syncCustomers() {
  console.log('Starting customer sync...')

  // EXTRACT: Fetch data from external API
  const response = await fetch('https://api.example.com/customers', {
    headers: {
      Authorization: `Bearer ${process.env.API_KEY}`
    }
  })

SKILL.md on GitHub