nvidia avatar

Tao Train Mask Auto Label

This handles weakly-supervised segmentation when you have point or box annotations instead of full pixel masks. It's built on ViT-MAE and ships with AutoML enabled by default, tuning learning rate and weight decay in a narrow Bayesian range around the fine-tuning defaults. You'll need COCO-style JSON annotations, at least 24GB VRAM per GPU, and patience with the tiny learning rates that come with vision transformers. The crop size lives under dataset.crop_size, not model.crop_size, which trips people up. If you're prototyping masks from minimal labels and already have the annotation infrastructure, it works. Just know the runtime rejects tiny ViT variants, so stick with base or larger.

skills.sh installs
1.6K
GitHub stars
3.9K
last updated
Aug 21, 2026

Install

$ npx -y skills add nvidia/skills --skill tao-train-mask-auto-label --agent claude-code

Installs into .claude/skills of the current project.

Files

MAL

Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery).

MAL (Mask Auto-Label) for weakly-supervised segmentation. Produces segmentation masks from minimal annotations (e.g., point or box annotations). Uses ViT-MAE backbone.

Set train.pretrained_model_path for ViT-MAE pretrained weights.

Quick Start (docker run)

Docker-native launch — no TAO SDK and no Python on the host. Use the local Docker/platform skill instead when it gives a stricter environment-specific command (non-root UID mapping, cache redirects, remote daemons).

TAO_PYT_IMAGE_DEFAULT=nvcr.io/nvidia/tao/tao-toolkit:7.3.0-pyt  # versions-key: images.tao_toolkit.pyt
TAO_PYT_IMAGE="${TAO_PYT_IMAGE:-$TAO_PYT_IMAGE_DEFAULT}"
RUN_ROOT="${RUN_ROOT:-$PWD}"
DOCKER_COMMON=(
  --rm --gpus all --shm-size=8g
  --shm-size=8g
  --ulimit memlock=-1
  --ulimit stack=67108864
  -v "$RUN_ROOT/data:/data:ro"
  -v "$RUN_ROOT/specs:/specs:ro"
  -v "$RUN_ROOT/results:/results"
)

Train:

docker run "${DOCKER_COMMON[@]}" "$TAO_PYT_IMAGE" \
  mal train -e /specs/train.yaml

Evaluate:

Sponsors

Details

Repository
github.com/nvidia/skills
First seen
Jul 14, 2026

More from nvidia/skills

All 236

Recommended

1juliusbrussee avatarcavemanjuliusbrussee/cavemanUltra-compressed communication mode. Cuts output tokens 65% (measured) by speaking like caveman while keeping full technical accuracy. Supports intensity levels618.3Kskills.sh installs
2mattpocock avatargrill-memattpocock/skillsA relentless interview to sharpen a plan or design.1.5Mskills.sh installs
3shadcn avatarimproveshadcn/improveSurvey any codebase as a senior advisor and produce prioritized, self-contained implementation plans for OTHER models/agents to execute. Strictly read-only on s40Kskills.sh installs
4obra avatarsystematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes326.7Kskills.sh installs
5forrestchang avatarkarpathy-guidelinesforrestchang/andrej-karpathy-skillsBehavioral guidelines to reduce common LLM coding mistakes through explicit assumptions, simplicity, and verifiable success criteria.17.9Kskills.sh installs
6vercel-labs avatarfind-skillsvercel-labs/skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express int4.3Mskills.sh installs