Tracing Upstream Lineage

When you're staring at a table wondering where the hell the data actually comes from, this walks you through the investigation. It shows you how to find the DAG that populates your table, trace back through the SQL to spot upstream sources, and recursively build the full chain from external systems like Salesforce or S3 all the way to your target. The column-level tracing is genuinely useful for tracking down specific field origins through transformations. If you're on Astro you get a visual lineage tab that does some of this automatically, but on OSS Airflow you're reading DAG source code and task definitions manually. The lineage report format with diagrams and data quality implications is solid for documenting what you find.

skills.sh installs: 1.1K
GitHub stars: 459
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add astronomer/agents --skill tracing-upstream-lineage --agent claude-code

Installs into .claude/skills of the current project.

Files

Upstream Lineage: Sources

Trace the origins of data - answer "Where does this data come from?"

Lineage Investigation

Step 1: Identify the Target Type

Determine what we're tracing:

Table: Trace what populates this table
Column: Trace where this specific column comes from
DAG: Trace what data sources this DAG reads from

Step 2: Find the Producing DAG

Tables are typically populated by Airflow DAGs. Find the connection:

Search DAGs by name: Use af dags list and look for DAG names matching the table name
- load_customers -> customers table
- etl_daily_orders -> orders table
Explore DAG source code: Use af dags source <dag_id> to read the DAG definition
- Look for INSERT, MERGE, CREATE TABLE statements
- Find the target table in the code
Check DAG tasks: Use af tasks list <dag_id> to see what operations the DAG performs

On Astro

If you're running on Astro, the Lineage tab in the Astro UI provides visual lineage exploration across DAGs and datasets. Use it to quickly trace upstream dependencies without manually searching DAG source code.