Sglang

SGLang is a serving framework that's worth knowing about if you're building agents or need structured outputs. The killer feature is RadixAttention, which automatically caches shared prefixes across requests. In practice, this means if you're running agents with the same system prompt, you get 5× faster inference than vLLM because it only computes the new tokens. It handles constrained JSON/regex generation natively and is genuinely faster at it (3× on JSON decoding). The codebase powers 300,000+ GPUs at xAI and LinkedIn, so it's battle-tested at scale. Use it when you have repeated context or need guaranteed output formats. Stick with vLLM if you just need simple text generation without the prefix caching overhead.

skills.sh installs: 1K
GitHub stars: 14.4K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add orchestra-research/ai-research-skills --skill sglang --agent claude-code

Installs into .claude/skills of the current project.

Files

SGLang

High-performance serving framework for LLMs and VLMs with RadixAttention for automatic prefix caching.

When to use SGLang

Use SGLang when:

Need structured outputs (JSON, regex, grammar)
Building agents with repeated prefixes (system prompts, tools)
Agentic workflows with function calling
Multi-turn conversations with shared context
Need faster JSON decoding (3× vs standard)

Use vLLM instead when:

Simple text generation without structure
Don't need prefix caching
Want mature, widely-tested production system

Use TensorRT-LLM instead when:

Maximum single-request latency (no batching needed)
NVIDIA-only deployment
Need FP8/INT4 quantization on H100

Quick start

Installation

# pip install (recommended)
pip install "sglang[all]"

# With FlashInfer (faster, CUDA 11.8/12.1)
pip install sglang[all] flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/

# From source
git clone https://github.com/sgl-project/sglang.git
cd sglang
pip install -e "python[all]"

Launch server

# Basic server (Llama 3-8B)
python -m sglang.launch_server \
    --model-path meta-llama/Meta-Llama-3-8B-Instruct \
    --port 30000

SKILL.md on GitHub

Details

Repository: github.com/orchestra-research/ai-research-skills
Categories: AI & Agent Building, Data Science & ML
First seen: Jun 3, 2026