Investigating Anomalous Results

When something breaks or looks wrong, the natural instinct is to tweak until it passes. This skill enforces the opposite: find the root cause before changing anything. It walks through four phases, from characterizing the anomaly and instrumenting each pipeline stage to forming testable hypotheses and documenting the actual resolution. The iron law is simple: no dropping data, changing parameters, or rerunning with different seeds until you know whether it's a code bug, data issue, or real finding. Honestly, this is most useful when the anomaly is convenient and supports what you wanted to see, because that's when the temptation to skip investigation is strongest. It's structured enough to follow under pressure when your judgment is weakest.

GitHub stars: 202
last updated: Jun 1, 2026

View on GitHub

Install

$ npx -y skills add k-dense-ai/science-superpowers --skill investigating-anomalous-results --agent claude-code

Installs into .claude/skills of the current project.

Files

Investigating Anomalous Results

Overview

Random tweaks waste time and manufacture false findings. Quietly dropping the inconvenient data point, nudging the cutoff, or re-running until it "works" doesn't fix the problem — it fabricates a result.

Core principle: ALWAYS find the root cause before adjusting anything. An adjustment made before you understand the cause is, at best, noise and, at worst, fraud.

Violating the letter of this process is violating the spirit of it.

The first question for any anomaly: is this a code bug, a data issue, or a real finding? You cannot answer by guessing, and you must not "fix" it until you know — because a real finding is not something to fix.

The Iron Law

NO ADJUSTMENTS WITHOUT ROOT-CAUSE INVESTIGATION FIRST

No dropping data, changing a test, transforming a variable, re-running with new parameters, or "cleaning" until Phase 1 is complete. If you haven't found the cause, you cannot justify the adjustment.