Knowledge Distillation

davila7/claude-code-templates

This is a template for compressing large language models down to smaller ones while keeping most of their performance. The techniques here (temperature scaling, soft targets, logit distillation) let you take a 70B parameter model and shrink it to 7B while retaining over 90% accuracy, which is huge for deployment costs. It's based on solid research, referencing the original Hinton paper from 2015 and newer work like MiniLLM. Honestly most useful if you're trying to move from expensive proprietary models to something you can actually run in production, or if you need to create a specialized model for a specific domain without training from scratch. The 478 installs suggest people are actively experimenting with this.

skills.sh installs

497

GitHub stars

33.2K

last updated

Aug 21, 2026

Sponsors

Promote your product hereReach 125k+ visitors monthlyPromote your product hereReach 125k+ visitors monthlyAgent CaffeineSponsoredClose your lid, come back to finished workSnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthlyWaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthlyknobsSponsoredSee your site on any phone, any day, in any language

Recommended

More Debugging

1longbridge-risk-analysislongbridge/skillsRisk measurement and stress testing via Longbridge — computes VaR (historical simulation / parametric), CVaR (expected shortfall), max drawdown, Sharpe ratio, C496skills.sh installs

2api-paginationaj-geddes/useful-ai-promptsImplement efficient pagination strategies for large datasets using offset/limit, cursor-based, and keyset pagination. Use when returning collections, managing l495skills.sh installs

3longbridge-company-tearsheetlongbridge/skillsCompany tear sheet / one-pager via Longbridge Securities — generates a high-density 1–2 page company snapshot: business overview, key financials (revenue / net 495skills.sh installs

4longbridge-profit-analysislongbridge/skillsAccount-level profit and loss analysis via Longbridge Securities — simple return, time-weighted return (TWR), per-symbol P&L breakdown, and P&L by market. Suppo494skills.sh installs

5longbridge-post-investmentlongbridge/skillsPost-investment monitoring via Longbridge Securities — tracks portfolio holdings vs plan on a quarterly / monthly basis, extracts key KPIs (revenue growth / gro493skills.sh installs

6autonomous-agentsdavila7/claude-code-templatesAutonomous agents are AI systems that can independently decompose goals, plan actions, execute tools, and self-correct without constant human guidance. The chal492skills.sh installs