Moe Training
If you're looking to train larger language models without burning through your compute budget, this is the template you want. It's built around Mixture of Experts architecture, which activates only a subset of parameters per token (think 13B out of 47B in Mixtral). The skill includes DeepSpeed integration and covers the key MoE patterns used in production models like Mixtral 8x7B and DeepSeek-V3. You're looking at roughly 5x cost reduction versus dense models at similar performance. The real win is being able to scale capacity without linear compute scaling, which matters a lot if you're training on a realistic budget. It's practical infrastructure code, not just theory.
- skills.sh installs
- 357
- GitHub stars
- 33.2K
- last updated
- Aug 21, 2026
Install
$ npx -y skills add davila7/claude-code-templates --skill moe-training --agent claude-codeInstalls into .claude/skills of the current project.
Files
If you're looking to train larger language models without burning through your compute budget, this is the template you want. It's built around Mixture of Experts architecture, which activates only a subset of parameters per token (think 13B out of 47B in Mixtral). The skill includes DeepSpeed integration and covers the key MoE patterns used in production models like Mixtral 8x7B and DeepSeek-V3. You're looking at roughly 5x cost reduction versus dense models at similar performance. The real win is being able to scale capacity without linear compute scaling, which matters a lot if you're training on a realistic budget. It's practical infrastructure code, not just theory.
Browse the repository on GitHubSponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/davila7/claude-code-templates
- Category
- Data Science & ML
- First seen
- Jun 3, 2026
More from davila7/claude-code-templates
All 618- Ab Test Setup356
- Codex356
- Javascript Mastery356
- Model Merging356
- Artifacts Builder355
- Blip 2 Vision Language355
- Programmatic Seo355
- Biopython354
- Production Code Audit354
- Model Pruning353
- Fda Consultant Specialist352
- Voice Ai Development351
- Evaluating Code Models350
- Geo Database350
- Notion Spec To Implementation350
- Training Llms Megatron350
- Changelog Generator349
- Performance Profiling349
- Server Management349
- Tensorrt Llm349
- Deslop348
- Algorithmic Art347
- Email Systems347
- Faiss347
Recommended
More Data Science & ML
ln-510-quality-coordinatorlevnikolaevich/claude-code-skillsUse when coordinating story quality evaluation with mandatory research, worker summaries, agent review, regression evidence, and bounded refinement.357skills.sh installs
dx-data-navigatorpskoett/pskoett-ai-skillsQuery Developer Experience (DX) data via the DX Data MCP server PostgreSQL database. Use this skill when analyzing developer productivity metrics, team performa355skills.sh installs
alibabacloud-resourcecenter-searchaliyun/alibabacloud-aiops-skillsAlibaba Cloud Resource Center - Global Resource Inventory, Search & Statistics Skill. Provides cross-region, cross-product, and cross-account resource inventory354skills.sh installs
grant-proposal-assistantlyndonkl/claudeGuides creation and review of competitive grant proposals (NIH R01/R21/K, NSF, foundations) by applying reviewer-perspective thinking to ensure clear hypotheses353skills.sh installs
sales-call-prep-assistantonewave-ai/claude-skillsGenerate pre-call research briefs with company news, stakeholder backgrounds, and custom discovery question sets.353skills.sh installs
custom-code-managementwebflow/webflow-skillsAdd, review, or remove inline custom scripts on a Webflow site (up to 10,000 chars). Use for analytics, tracking pixels, chat widgets, or any custom JavaScript.353skills.sh installs