Databricks Pipelines

databricks/databricks-agent-skillsOfficial

This is your go-to for building data pipelines on Databricks using Lakeflow (formerly Delta Live Tables). It handles the critical decision of streaming table versus materialized view, which trips up most people. The decision tree alone is worth having open while you work. Covers Auto Loader for ingestion, Auto CDC for change data capture, Append Flows for multi-source patterns, and expectations for data quality. One thing that stands out: it explicitly calls out common traps like trying to aggregate over streaming sources (use an MV with batch read instead) and misusing unions when you need Append Flows. Requires databricks CLI and assumes you're already familiar with core Databricks operations.

skills.sh installs: 1K
GitHub stars: 393
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add databricks/databricks-agent-skills --skill databricks-pipelines --agent claude-code

Installs into .claude/skills of the current project.

Files

Lakeflow Spark Declarative Pipelines Development

FIRST: Use the parent databricks-core skill for CLI basics, authentication, profile selection, and data discovery commands.

SDP, LDP, Lakeflow Declarative Pipelines, and DLT (former name) all refer to the same product.

Decision Tree

Use this tree to choose the dataset type first, then layer on applicable features. Multiple features can apply to the same dataset — e.g., a Streaming Table can use Auto Loader for ingestion, Append Flows for fan-in, and Expectations for data quality.

SKILL.md on GitHub

Sponsors

Promote your product hereReach 125k+ visitors monthly Promote your product hereReach 125k+ visitors monthly