Fine Tuning Expert

This is what you reach for when you need to fine-tune an LLM without burning through GPU budget or guessing at hyperparameters. It walks you through the full pipeline from dataset validation to deployment, with working examples for LoRA and QLoRA using Hugging Face PEFT. The checkpoint-driven workflow is smart: it forces you to validate your data before training and actually measure metrics before deploying. Includes concrete configs (rank 16, alpha 32, cosine scheduling) and the merge-and-unload pattern for production. The constraint list calling out common mistakes like skipping warmup or merging incompatible adapters shows this comes from someone who's debugged those failures. Covers instruction tuning, RLHF, and DPO if you need them.

skills.sh installs: 4.5K
GitHub stars: 12.3K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add jeffallan/claude-skills --skill fine-tuning-expert --agent claude-code

Installs into .claude/skills of the current project.

Files

Fine-Tuning Expert

Senior ML engineer specializing in LLM fine-tuning, parameter-efficient methods, and production model optimization.

Core Workflow

Dataset preparation — Validate and format data; run quality checks before training starts
- Checkpoint: python validate_dataset.py --input data.jsonl — fix all errors before proceeding
Method selection — Choose PEFT technique based on GPU memory and task requirements
- Use LoRA for most tasks; QLoRA (4-bit) when GPU memory is constrained; full fine-tune only for small models
Training — Configure hyperparameters, monitor loss curves, checkpoint regularly
- Checkpoint: validation loss must decrease; plateau or increase signals overfitting
Evaluation — Benchmark against the base model; test on held-out set and edge cases
- Checkpoint: collect perplexity, task-specific metrics (BLEU/ROUGE), and latency numbers
Deployment — Merge adapter weights, quantize, measure inference throughput before serving