Hqq Quantization

HQQ gives you calibration-free quantization down to 4/3/2-bit precision, which means you can quantize any model in minutes without needing sample datasets. The real advantage here is speed and flexibility: while GPTQ and AWQ require hours and calibration data, HQQ just works immediately with multiple backend options like Marlin, TorchAO, and BitBlas for different hardware. It integrates natively with HuggingFace and vLLM, and you can fine-tune the quantized models with LoRA if needed. The tradeoff is that calibration-based methods like AWQ will generally give you better accuracy for production serving, but if you're experimenting with extreme quantization or need to compress models fast without datasets, this is the pragmatic choice.

skills.sh installs: 1K
GitHub stars: 14.4K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add orchestra-research/ai-research-skills --skill hqq-quantization --agent claude-code

Installs into .claude/skills of the current project.

Files

HQQ - Half-Quadratic Quantization

Fast, calibration-free weight quantization supporting 8/4/3/2/1-bit precision with multiple optimized backends.

When to use HQQ

Use HQQ when:

Quantizing models without calibration data (no dataset needed)
Need fast quantization (minutes vs hours for GPTQ/AWQ)
Deploying with vLLM or HuggingFace Transformers
Fine-tuning quantized models with LoRA/PEFT
Experimenting with extreme quantization (2-bit, 1-bit)

Key advantages:

No calibration: Quantize any model instantly without sample data
Multiple backends: PyTorch, ATEN, TorchAO, Marlin, BitBlas for optimized inference
Flexible precision: 8/4/3/2/1-bit with configurable group sizes
Framework integration: Native HuggingFace and vLLM support
PEFT compatible: Fine-tune quantized models with LoRA

Use alternatives instead: