Harvard Art Museums Data Engineering

This is a complete data engineering starter project that walks you through building ETL pipelines with real museum data from Harvard's API. You get the full stack: API pagination with rate limiting, transformation of nested JSON into normalized MySQL tables with proper foreign keys, 20+ analytical SQL queries, and a Streamlit dashboard with Plotly visualizations. The repo includes ready-to-use code for extracting artifact metadata, images, and color data, then loading it into relational tables. It's genuinely useful if you're learning data engineering patterns or need a working example of museum/cultural heritage data pipelines to adapt for your own projects.

skills.sh installs: 536
GitHub stars: 2
last updated: Jul 14, 2026

View on GitHub

Install

$ npx -y skills add aradotso/data-skills --skill harvard-art-museums-data-engineering --agent claude-code

Installs into .claude/skills of the current project.

Files

Harvard Art Museums Data Engineering & Analytics

Skill by ara.so — Data Skills collection.

This skill enables you to build production-ready data engineering pipelines using the Harvard Art Museums API. The project demonstrates real-world ETL patterns, relational database design, SQL analytics, and interactive visualization with Streamlit.

What It Does

API Integration: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting
ETL Pipeline: Transforms nested JSON into normalized relational tables
Database Design: Creates properly structured MySQL/TiDB tables with foreign keys
SQL Analytics: Executes 20+ analytical queries for insights
Visualization: Builds interactive Streamlit dashboards with Plotly charts

Installation

# Clone the repository
git clone https://github.com/Manali0711/Harvard-Artifacts-Collection-Data-Engineering-Analytics-App.git
cd Harvard-Artifacts-Collection-Data-Engineering-Analytics-App

# Install dependencies
pip install -r requirements.txt

# Required packages
pip install streamlit pandas requests mysql-connector-python plotly python-dotenv