awslabs avatar

Hyperpod Node Debugger

When a single node in your HyperPod cluster goes down or starts acting weird, this is your triage tool. It walks you through diagnosing GPU hardware failures (XIDs, ECC errors, row-remap issues), EFA and security group misconfigurations, Slurm node states, disk pressure, SSM agent problems, and pod networking. The workflow is read-only: it runs diagnostics, points you to the relevant section in its reference docs, then shows you the exact commands to fix it. You approve each destructive action yourself. It explicitly won't touch cluster-wide problems, NCCL tuning, or model flop utilization, which have their own dedicated skills. If you're managing p5 or Trn instances at scale and one node is misbehaving, this gives you a structured checklist instead of guessing.

GitHub stars
783
last updated
Jun 11, 2026

Install

$ npx -y skills add awslabs/agent-plugins --skill hyperpod-node-debugger --agent claude-code

Installs into .claude/skills of the current project.

Files

HyperPod Node Debugger

Operating policy. Run read-only diagnostics yourself. Never run a command that changes cluster, node, or workload state — present each one as a Suggested command (run this yourself) block and wait for the customer. Destructive order: investigate → reboot → replace (replace destroys root + secondary volumes; not supported on Slurm controller nodes). Never discard training state, logs, or caches on speculation.

Sponsors

Details

Repository
github.com/awslabs/agent-plugins
Categories
AI & Agent Building, Debugging, Automation & Workflows
License
Apache-2.0
First seen
Jun 11, 2026

More from awslabs/agent-plugins

All 32
1awslabs avatarhyperpod-performance-debuggerawslabs/agent-pluginsDiagnose performance issues on Amazon SageMaker HyperPod clusters — uneven NCCL bandwidth across nodes and poor filesystem throughput. Read-only. Surfaces host-0skills.sh installs
2awslabs avatarhyperpod-slurm-debuggerawslabs/agent-pluginsDiagnostic-only skill for Slurm scheduler and node-daemon issues on Amazon SageMaker HyperPod Slurm clusters. Scope mirrors the HyperPod troubleshooting guide. 0skills.sh installs
3awslabs avatarmodel-selectionawslabs/agent-pluginsSelects a base model for the user's use case by querying SageMaker Hub. Use when the user asks which model to use, wants to select or change their base model, m0skills.sh installs
4awslabs avatarsdk-getting-startedawslabs/agent-pluginsValidates the user's environment for SageMaker AI operations — checks SDK version, AWS region, and execution role. Use when the user says "set up", "getting sta0skills.sh installs
5callstackincubator avatardogfoodcallstackincubator/agent-skillsSystematically explore and test a mobile app on iOS/Android with agent-device to find bugs, UX issues, and other problems. Use when asked to "dogfood", "QA", "e0skills.sh installs
6callstackincubator avatarreact-native-testingcallstackincubator/agent-skills>0skills.sh installs