Claude MarketplacesSubmit
aradotso avatar

Google Cloud Data Engineering Hub

aradotso/data-skills

This is a comprehensive reference hub with 54+ production-ready GCP data engineering projects built by a Google Developer Expert. You get working code for BigQuery ML models, Dataflow pipelines, Pub/Sub streaming, Cloud Composer DAGs, and Dataproc jobs. Each project includes architecture diagrams, sample data, and deploy.sh scripts tested against live GCP environments. It's structured as modular Python packages, not scripts, which makes it easier to adapt patterns into your own codebase. If you're building data pipelines on GCP and want proven examples beyond the official docs, this gives you a solid starting point with real UPSERT patterns, signed URLs, lifecycle policies, and Beam transforms already implemented.

skills.sh installs
662
GitHub stars
2
last updated
Jul 14, 2026
View on GitHub

Install

$ npx -y skills add aradotso/data-skills --skill google-cloud-data-engineering-hub --agent claude-code

Installs into .claude/skills of the current project.

Files

Google Cloud Data Engineering Hub Skill

Skill by ara.so — Data Skills collection

This skill enables AI coding agents to help developers build production-grade Google Cloud Platform (GCP) data engineering solutions using this comprehensive reference repository of 54+ working projects covering BigQuery, Dataflow, Apache Beam, Cloud Composer, Pub/Sub, Dataproc, Gemini AI, and Vertex AI.

What This Project Provides

A curated collection of complete, runnable GCP data engineering projects. Each project includes:

  • Modular Python code (not scripts)
  • ASCII architecture diagrams
  • Sample data fixtures
  • deploy.sh with GCP setup automation
  • End-to-end working examples tested against live GCP environments

Built by Vishal Bulbule (Google Developer Expert, 12x GCP Certified).

Installation & Setup

Clone the Repository

git clone https://github.com/vishal-bulbule/google-cloud-data-engineering-hub.git
cd google-cloud-data-engineering-hub

Prerequisites

  • Python 3.10+
  • Google Cloud SDK (gcloud CLI)
  • Active GCP project with billing enabled
  • Appropriate IAM permissions

Environment Setup

SKILL.md on GitHub

Sponsors

Promote your product hereReach 125k+ visitors monthlyPromote your product hereReach 125k+ visitors monthlyAgent CaffeineSponsoredClose your lid, come back to finished workSnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthlyWaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthlyknobsSponsoredSee your site on any phone, any day, in any language

Details

Repository
github.com/aradotso/data-skills
Category
Cloud & Infrastructure
First seen
Jul 14, 2026

More from aradotso/data-skills

All 60

Recommended

More Cloud & Infrastructure
1shipshitdev avataraws-infrastructureshipshitdev/libraryaws infrastructure151skills.sh installs
2awslabs avataraws-lambda-durable-functionsawslabs/agent-plugins>0skills.sh installs
3google avatargoogle-cloud-recipe-onboardinggoogle/skillsGuides a developer's first steps on Google Cloud, covering account creation, billing setup, project management, and deploying a first resource. Use when a new d16.4Kskills.sh installs
4google avatargoogle-cloud-waf-reliabilitygoogle/skillsGenerates guidance for reliability, resilience, availability, redundancy, fault-tolerance, and disaster recovery (DR) for Google Cloud workloads based on the de15.1Kskills.sh installs
5aws avataraws-cloudformationaws/agent-toolkit-for-awsAuthor, validate, and troubleshoot AWS CloudFormation templates. Covers template authoring with secure defaults, pre-deployment validation (cfn-lint, cfn-guard,9.4Kskills.sh installs
6aws avatarquerying-aws-s3aws/agent-toolkit-for-awsQueries S3 object metadata, tracks bucket activity, audits object changes, searches annotations, and analyzes storage metrics using S3 Metadata system tables (j6Kskills.sh installs