Transformer Lens Interpretability

If you're reverse-engineering how transformers work internally, this gives you TransformerLens expertise for mechanistic interpretability research. It's the standard library for inspecting model internals through HookPoints, letting you cache activations, patch them between runs to test causality, and decompose outputs by attention head or layer. The workflow walkthroughs cover activation patching for causal tracing, circuit analysis like the famous IOI paper, and induction head detection. Works with 50+ models including GPT-2, LLaMA, and Mistral. The main limitation is it's transformer-only, so you'll need nnsight or pyvene for other architectures. If you're doing SAE work specifically, SAELens is purpose-built for that instead.

skills.sh installs: 1K
GitHub stars: 14.4K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add orchestra-research/ai-research-skills --skill transformer-lens-interpretability --agent claude-code

Installs into .claude/skills of the current project.

Files

TransformerLens: Mechanistic Interpretability for Transformers

TransformerLens is the de facto standard library for mechanistic interpretability research on GPT-style language models. Created by Neel Nanda and maintained by Bryce Meyer, it provides clean interfaces to inspect and manipulate model internals via HookPoints on every activation.

GitHub: TransformerLensOrg/TransformerLens (2,900+ stars)

When to Use TransformerLens

Use TransformerLens when you need to:

Reverse-engineer algorithms learned during training
Perform activation patching / causal tracing experiments
Study attention patterns and information flow
Analyze circuits (e.g., induction heads, IOI circuit)
Cache and inspect intermediate activations
Apply direct logit attribution

Consider alternatives when:

You need to work with non-transformer architectures → Use nnsight or pyvene
You want to train/analyze Sparse Autoencoders → Use SAELens
You need remote execution on massive models → Use nnsight with NDIF
You want higher-level causal intervention abstractions → Use pyvene