Llava

LLaVA combines a CLIP vision encoder with Vicuna language models to build conversational image understanding systems. You'd use this for visual question answering, multi-turn image chat, or document understanding where you need natural back-and-forth dialogue about images. The 7B model is the sweet spot at 14GB VRAM, though you can drop to 4GB with quantization. It's genuinely open source with Apache 2.0 licensing and hits competitive benchmarks on VQAv2 and GQA. Be aware it hallucinates details and struggles with spatial reasoning and counting, so verify critical information. If you just need image classification, stick with CLIP. If you need GPT-4V quality and have budget, use the API. This is for when you want capable vision-language chat running on your own infrastructure.

skills.sh installs: 1K
GitHub stars: 14.4K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add orchestra-research/ai-research-skills --skill llava --agent claude-code

Installs into .claude/skills of the current project.

Files

LLaVA - Large Language and Vision Assistant

Open-source vision-language model for conversational image understanding.

When to use LLaVA

Use when:

Building vision-language chatbots
Visual question answering (VQA)
Image description and captioning
Multi-turn image conversations
Visual instruction following
Document understanding with images

Metrics:

23,000+ GitHub stars
GPT-4V level capabilities (targeted)
Apache 2.0 License
Multiple model sizes (7B-34B params)

Use alternatives instead:

GPT-4V: Highest quality, API-based
CLIP: Simple zero-shot classification
BLIP-2: Better for captioning only
Flamingo: Research, not open-source

Quick start

Installation

# Clone repository
git clone https://github.com/haotian-liu/LLaVA
cd LLaVA

# Install
pip install -e .

Basic usage

from llava.model.builder import load_pretrained_model
from llava.mm_utils import get_model_name_from_path, process_images, tokenizer_image_token
from llava.constants import IMAGE_TOKEN_INDEX, DEFAULT_IMAGE_TOKEN
from llava.conversation import conv_templates
from PIL import Image
import torch

SKILL.md on GitHub