Model Merging

Mergekit is quietly powering some of the best models on HuggingFace leaderboards, and this skill walks you through actually using it. You can blend fine-tuned models without retraining to combine capabilities like math reasoning with coding or chat skills. The guide covers linear averaging, SLERP for smooth two-model interpolation, and fancier methods like TIES and DARE that handle conflicts when merging many specialized models. It includes real YAML configs for each approach and explains when to use what. The layer-wise merging patterns are especially useful if you want different model strengths at different depths. Runs on CPU, so no GPU costs, and you can experiment with new model variants in minutes instead of days of training.

skills.sh installs: 1K
GitHub stars: 14.4K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add orchestra-research/ai-research-skills --skill model-merging --agent claude-code

Installs into .claude/skills of the current project.

Files

Model Merging: Combining Pre-trained Models

When to Use This Skill

Use Model Merging when you need to:

Combine capabilities from multiple fine-tuned models without retraining
Create specialized models by blending domain-specific expertise (math + coding + chat)
Improve performance beyond single models (often +5-10% on benchmarks)
Reduce training costs - no GPUs needed, merges run on CPU
Experiment rapidly - create new model variants in minutes, not days
Preserve multiple skills - merge without catastrophic forgetting

Success Stories: Marcoro14-7B-slerp (best on Open LLM Leaderboard 02/2024), many top HuggingFace models use merging

Tools: mergekit (Arcee AI), LazyMergekit, Model Soup

Installation

# Install mergekit
git clone https://github.com/arcee-ai/mergekit.git
cd mergekit
pip install -e .

# Or via pip
pip install mergekit

# Optional: Transformer library
pip install transformers torch