Nanogpt

This is Andrej Karpathy's minimal GPT implementation in roughly 300 lines of readable PyTorch code. You can train a character-level model on Shakespeare in five minutes on a CPU, or reproduce GPT-2 124M on OpenWebText with multi-GPU if you have the hardware. The entire architecture fits in model.py with zero abstractions, making it genuinely hackable for learning how transformers work under the hood. It's not for production use. If you're trying to understand attention mechanisms or experiment with architecture changes without wading through framework abstractions, this is the cleanest starting point. The code assumes you know PyTorch basics but explains nothing else, which is exactly the point.

skills.sh installs: 1K
GitHub stars: 14.4K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add orchestra-research/ai-research-skills --skill nanogpt --agent claude-code

Installs into .claude/skills of the current project.

Files

nanoGPT - Minimalist GPT Training

Quick start

nanoGPT is a simplified GPT implementation designed for learning and experimentation.

Installation:

pip install torch numpy transformers datasets tiktoken wandb tqdm

Train on Shakespeare (CPU-friendly):

# Prepare data
python data/shakespeare_char/prepare.py

# Train (5 minutes on CPU)
python train.py config/train_shakespeare_char.py

# Generate text
python sample.py --out_dir=out-shakespeare-char

Output:

ROMEO:
What say'st thou? Shall I speak, and be a man?

JULIET:
I am afeard, and yet I'll speak; for thou art
One that hath been a man, and yet I know not
What thou art.

Common workflows

Workflow 1: Character-level Shakespeare

Complete training pipeline:

# Step 1: Prepare data (creates train.bin, val.bin)
python data/shakespeare_char/prepare.py

# Step 2: Train small model
python train.py config/train_shakespeare_char.py

# Step 3: Generate text
python sample.py --out_dir=out-shakespeare-char

SKILL.md on GitHub