Modal Serverless Gpu
Modal gives you GPU access without managing infrastructure, and this skill covers how to use it. You write Python code to define your compute needs (T4 through H100), and Modal handles scaling, cold starts, and billing by the second. The guide walks through deploying ML models as APIs, running batch jobs, and using persistent storage for model caching. It's honest about when to use alternatives like RunPod for long-running workloads or SkyPilot for multi-cloud setups. The real value is in the practical patterns: dynamic batching, keeping containers warm, and using the class-based approach with lifecycle hooks to avoid reloading models on every request.
- skills.sh installs
- 1K
- GitHub stars
- 14.4K
- last updated
- Aug 21, 2026
Install
$ npx -y skills add orchestra-research/ai-research-skills --skill modal-serverless-gpu --agent claude-codeInstalls into .claude/skills of the current project.
Files
Modal Serverless GPU
Comprehensive guide to running ML workloads on Modal's serverless GPU cloud platform.
When to use Modal
Use Modal when:
- Running GPU-intensive ML workloads without managing infrastructure
- Deploying ML models as auto-scaling APIs
- Running batch processing jobs (training, inference, data processing)
- Need pay-per-second GPU pricing without idle costs
- Prototyping ML applications quickly
- Running scheduled jobs (cron-like workloads)
Key features:
- Serverless GPUs: T4, L4, A10G, L40S, A100, H100, H200, B200 on-demand
- Python-native: Define infrastructure in Python code, no YAML
- Auto-scaling: Scale to zero, scale to 100+ GPUs instantly
- Sub-second cold starts: Rust-based infrastructure for fast container launches
- Container caching: Image layers cached for rapid iteration
- Web endpoints: Deploy functions as REST APIs with zero-downtime updates
Use alternatives instead:
- RunPod: For longer-running pods with persistent state
- Lambda Labs: For reserved GPU instances
- SkyPilot: For multi-cloud orchestration and cost optimization
- Kubernetes: For complex multi-service architectures
Sponsors
Promote your product hereReach 125k+ visitors monthlyPromote your product hereReach 125k+ visitors monthly
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any language
Agent CaffeineSponsoredClose your lid, come back to finished workSnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthlyWaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthlyknobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/orchestra-research/ai-research-skills
- Categories
- AI & Agent Building, Data Science & ML, Automation & Workflows, Cloud & Infrastructure
- First seen
- Jun 3, 2026
More from orchestra-research/ai-research-skills
All 95- Moe Training1K
- Sglang1K
- Chroma1K
- Instructor1K
- Speculative Decoding1K
- Llama Factory1K
- Model Merging1K
- Nemo Curator1K
- Transformer Lens Interpretability1K
- Distributed Llm Pretraining Torchtitan1K
- Openrlhf Training1K
- Evaluating Code Models1K
- Gptq1K
- Sentencepiece1K
- Outlines1K
- Llamaindex1K
- Pyvene Interventions1K
- Simpo Training1K
- Slime Rl Training1K
- Training Llms Megatron1K
- Guidance1K
- Mamba Architecture1K
- Model Pruning1K
- Sparse Autoencoder Training1K
Recommended
More AI & Agent Building1
moe-trainingorchestra-research/ai-research-skillsTrain Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense mode1Kskills.sh installs
moe-trainingorchestra-research/ai-research-skillsTrain Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense mode1Kskills.sh installs2llama-factoryorchestra-research/ai-research-skillsExpert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support1Kskills.sh installs
llama-factoryorchestra-research/ai-research-skillsExpert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support1Kskills.sh installs3model-mergingorchestra-research/ai-research-skillsMerge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific exp1Kskills.sh installs
model-mergingorchestra-research/ai-research-skillsMerge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific exp1Kskills.sh installs4transformer-lens-interpretabilityorchestra-research/ai-research-skillsProvides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation 1Kskills.sh installs
transformer-lens-interpretabilityorchestra-research/ai-research-skillsProvides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation 1Kskills.sh installs5openrlhf-trainingorchestra-research/ai-research-skillsHigh-performance RLHF framework with Ray+vLLM acceleration. Use for PPO, GRPO, RLOO, DPO training of large models (7B-70B+). Built on Ray, vLLM, ZeRO-3. 2× fast1Kskills.sh installs
openrlhf-trainingorchestra-research/ai-research-skillsHigh-performance RLHF framework with Ray+vLLM acceleration. Use for PPO, GRPO, RLOO, DPO training of large models (7B-70B+). Built on Ray, vLLM, ZeRO-3. 2× fast1Kskills.sh installs6evaluating-code-modelsorchestra-research/ai-research-skillsEvaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding 1Kskills.sh installs
evaluating-code-modelsorchestra-research/ai-research-skillsEvaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding 1Kskills.sh installs