Claude MarketplacesSubmit
annasmazhar avatar

PySpark MCP

annasmazhar/pyspark_mcp

This is a PySpark migration and optimization toolkit built on SQLGlot. It converts SQL between dialects (PostgreSQL, Oracle, Redshift, MySQL, Snowflake) and generates PySpark DataFrame API code from SQL queries. The AWS Glue integration generates complete job templates, handles DynamicFrame conversions, and analyzes S3 partitioning strategies. You also get code review tools that scan existing PySpark for performance issues, suggest join strategies, and detect duplication across hundreds of files with concurrent batch processing. Reach for this when migrating legacy SQL workloads to Spark or when you need to generate Glue jobs without writing boilerplate. It won't handle recursive CTEs natively but provides Spark SQL equivalents and guidance for edge cases.

tools
3
transport
STDIO
last updated
Oct 18, 2026
View on GitHub

Tools

3 tools

Public tool metadata for what this MCP can expose to an agent.

convertSQL → PySpark (including `mode=batch_dir`).

SQL → PySpark (including `mode=batch_dir`).

No parameter schema in public metadata yet.

glue_jobGlue 5.0 job *template* strings.

Glue 5.0 job *template* strings.

No parameter schema in public metadata yet.

reviewCode review, patterns, duplicates.

Code review, patterns, duplicates.

No parameter schema in public metadata yet.

Sponsors

Promote your product hereReach 125k+ visitors monthlyPromote your product hereReach 125k+ visitors monthlyAgent CaffeineSponsoredClose your lid, come back to finished workSnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthlyWaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthlyknobsSponsoredSee your site on any phone, any day, in any language

Readme

PySpark MCP Server

SQL migration assistance, AWS Glue job template generation, and Spark code optimization — as an MCP server.

Not the live-Spark pyspark-mcp package. This project is SQL → PySpark / Glue source generation, published as pyspark-tools. SemyonSinchenko/pyspark-mcp introspects a running SparkSession. A deprecated pyspark-mcp console script remains here so old configs keep working; it prints a warning, then starts this server.

CI Pipeline Python 3.11+ License: MIT

What It Does

View the full README on GitHub

Details

Repository
github.com/annasmazhar/pyspark_mcp
Categories
Databases, Cloud & Infrastructure
Package
pyspark-tools
Transport
STDIO
Registry
MCP registry, active

Related Databases MCP servers

View all
1arpe-io avatarMigratorxpress Mcparpe-io/migratorxpress-mcpMCP server for MigratorXpress — cross-platform database migration with parallel transfer0stars
2aryanduntley avatarAifparyanduntley/aifpDatabase-driven FP enforcement and project management for AI-maintained codebases0stars
3lutolabs avatarMyAITwinlutolabs/mytwin-mcpA personal RAG database you build from chat, so AI creates work that sounds like you.0stars
4cyanheads avatarfaostat-mcp-servercyanheads/faostat-mcp-serverUN FAOSTAT global food & agriculture statistics over a local SQLite mirror, via MCP.0stars
5daedalus avatarMcp Server Mssqldaedalus/mcp-server-mssqlMCP server for Microsoft SQL Server using mssql-python0stars
6dbsectrainer avatarMcp Data Pipeline Connectordbsectrainer/mcp-data-pipeline-connectorUniversal data connector for CSV, Postgres, and REST APIs via DuckDB0stars