Cuda Kernels

This walks you through writing optimized CUDA kernels for HuggingFace diffusers and transformers on H100, A100, and T4 GPUs. You get kernel templates for RMSNorm, attention, RoPE, and activation functions, plus benchmarking scripts that compare against PyTorch baselines. The included RMSNorm kernel hits 2.67x speedup in microbenchmarks but only 6% end-to-end improvement in LTX-Video generation because normalization is a small fraction of total compute. Worth using if you're already profiling and know which kernels are your bottleneck. The skill includes working examples for both diffusers and transformers integration, architecture-specific optimization guides, and support for loading pre-compiled kernels from HuggingFace Hub to skip local compilation entirely.

skills.sh installs: 212
GitHub stars: 783
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add huggingface/kernels --skill cuda-kernels --agent claude-code

Installs into .claude/skills of the current project.

Files

CUDA Kernels for Diffusers & Transformers

This skill provides patterns and guidance for developing optimized CUDA kernels targeting NVIDIA GPUs (H100, A100, T4) for use with HuggingFace diffusers and transformers libraries.

Hard Constraints — Read Before Writing Any Code

Kernels MUST build with kernel-builder and meet the Kernel Hub requirements. kernel-builder compiles against the Python limited API (ABI3) so a single binary works for Python 3.9+ across versions. Several patterns that are standard in generic PyTorch-extension tutorials are therefore hard build failures here. Do not use them, even if PyTorch documentation or your training data suggests them.