Airflow Dag Patterns

If you're building data pipelines with Airflow, this skill gives you production-ready DAG patterns that actually work in the real world. It covers TaskFlow API decorators for cleaner XCom passing, dynamic DAG generation from config files, proper branching with quality checks, and sensor patterns for external dependencies. The examples show concrete retry strategies, error handling, and testing approaches that prevent the common DAG antipatterns that break pipelines at 3am. Solid reference for both building new workflows and refactoring messy DAGs.

skills.sh installs: 11.3K
GitHub stars: 42.4K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add wshobson/agents --skill airflow-dag-patterns --agent claude-code

Installs into .claude/skills of the current project.

Files

Apache Airflow DAG Patterns

Production-ready patterns for Apache Airflow including DAG design, operators, sensors, testing, and deployment strategies.

When to Use This Skill

Creating data pipeline orchestration with Airflow
Designing DAG structures and dependencies
Implementing custom operators and sensors
Testing Airflow DAGs locally
Setting up Airflow in production
Debugging failed DAG runs

Core Concepts

1. DAG Design Principles

Principle	Description
Idempotent	Running twice produces same result
Atomic	Tasks succeed or fail completely
Incremental	Process only new/changed data
Observable	Logs, metrics, alerts at every step

2. Task Dependencies

# Linear
task1 >> task2 >> task3

# Fan-out
task1 >> [task2, task3, task4]

# Fan-in
[task1, task2, task3] >> task4

# Complex
task1 >> task2 >> task4
task1 >> task3 >> task4