Enterprise Data Engineering Pipeline Ssis Pyspark

This is a full-stack enterprise data warehousing setup that combines SSIS for ETL orchestration, SQL Server with star schema design (fact and dimension tables), and PySpark for scaled analytics. You get the complete pipeline: ingesting raw CSVs, transforming through SSIS packages with lookup validations and error logging, loading into a dimensional model, then running Python audits for data quality checks and PySpark aggregations for heavy workloads. The source includes actual SQL DDL for dimensions and facts, SSIS component mappings, and Python scripts with SQLAlchemy connections. If you're building a classic Microsoft-stack data warehouse but need to scale beyond what T-SQL handles comfortably, this shows how to wire it all together with concrete error handling and BI views.

skills.sh installs: 1.5K
GitHub stars: 2
last updated: Jul 14, 2026

View on GitHub

Install

$ npx -y skills add aradotso/data-skills --skill enterprise-data-engineering-pipeline-ssis-pyspark --agent claude-code

Installs into .claude/skills of the current project.

Files

Enterprise Data Engineering Pipeline (SSIS + PySpark)

Skill by ara.so — Data Skills collection.

Overview

This project provides a complete enterprise data engineering solution that combines:

SSIS (SQL Server Integration Services) for ETL orchestration
SQL Server with Star Schema data warehouse design (fact and dimension tables)
Python (Pandas) for data quality audits and visualization
PySpark for big data analytics and aggregation

The pipeline ingests raw CSV files (Sales, Products, Customers), transforms them through SSIS, loads into a dimensional model, and performs analytics at scale.