Harvard Artifacts Collection Etl Analytics

This is a complete ETL reference implementation for working with the Harvard Art Museums API. It shows you how to handle paginated API requests with rate limiting, transform nested JSON into normalized SQL tables (metadata, media, colors), and build a Streamlit dashboard with 20+ analytical queries and Plotly visualizations. The code is practical and well-structured, covering the full pipeline from extraction to visualization. Use this when you need to see how a museum or cultural heritage data pipeline actually fits together, or as a template for similar API-to-database projects. The main value is seeing all the pieces (API pagination, DataFrame transformations, MySQL schema design, dashboard queries) in one working example rather than scattered across documentation.

skills.sh installs: 1.4K
GitHub stars: 2
last updated: Jul 14, 2026

View on GitHub

Install

$ npx -y skills add aradotso/data-skills --skill harvard-artifacts-collection-etl-analytics --agent claude-code

Installs into .claude/skills of the current project.

Files

Harvard Artifacts Collection ETL Analytics

Skill by ara.so — Data Skills collection.

This project provides a complete data engineering and analytics solution for the Harvard Art Museums API. It demonstrates ETL pipeline construction, relational database design, SQL analytics, and interactive visualization using Streamlit. The application extracts artifact metadata, transforms nested JSON into structured tables, loads data into SQL databases, and provides 20+ analytical queries with auto-generated visualizations.

Installation

# Clone the repository
git clone https://github.com/Manali0711/Harvard-Artifacts-Collection-Data-Engineering-Analytics-App.git
cd Harvard-Artifacts-Collection-Data-Engineering-Analytics-App

# Install dependencies
pip install -r requirements.txt

# Set up environment variables
export HARVARD_API_KEY="your_api_key_here"
export DB_HOST="your_database_host"
export DB_USER="your_database_user"
export DB_PASSWORD="your_database_password"
export DB_NAME="your_database_name"