Umbraco Skill Test Runner
This is part of Umbraco's skill validation pipeline, specifically the third phase that runs actual tests. It orchestrates unit tests via Web Test Runner, mocked integration tests, and full E2E tests through Playwright, then spits out a consolidated JSON report. The nice touch is that it'll automatically spin up your Umbraco instance if E2E tests need it and shut it down when done. You'd use this after link validation and code analysis to verify your skill examples actually work. It assumes you're working within the Umbraco CMS Skills project structure and have your test infrastructure already set up with the right tooling.
- skills.sh installs
- 384
- GitHub stars
- 27
- last updated
- Aug 21, 2026
Install
$ npx -y skills add umbraco/umbraco-cms-backoffice-skills --skill umbraco-skill-test-runner --agent claude-codeInstalls into .claude/skills of the current project.
Files
Skill Test Runner
Runs all tests from skill examples (unit, mocked, E2E) and outputs a JSON report.
Usage
cd .claude/skills/umbraco-skill-test-runner/scripts
npm install --silent
npx tsx run-tests.ts
Test Types
| Type | Description | Runner |
|---|---|---|
| Unit | Component tests with @open-wc/testing | Web Test Runner |
| Mocked | Integration tests against mocked backoffice | Playwright |
| E2E | Full end-to-end against real Umbraco | Playwright |
Environment Variables
| Variable | Default | Description |
|---|---|---|
UMBRACO_USER_LOGIN | - | Admin email for E2E authentication |
UMBRACO_USER_PASSWORD | - | Admin password for E2E authentication |
UMBRACO_URL | https://localhost:44325 | Umbraco instance URL |
Auto-Start Umbraco
If E2E tests are needed and Umbraco is not running, the script will automatically:
- Start Umbraco from
./Umbraco-CMS.Skills/usingdotnet run - Wait for the instance to be ready
- Run E2E tests
- Stop Umbraco when done (if it was started by the script)
Output
Sponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/umbraco/umbraco-cms-backoffice-skills
- Categories
- Testing & QA, .NET & C#
- First seen
- Jun 3, 2026
More from umbraco/umbraco-cms-backoffice-skills
All 76- Umbraco State Management384
- Umbraco Example Generator383
- Umbraco Skill Code Analyzer383
- Umbraco User Profile App383
- Umbraco Mocked Backoffice382
- Umbraco Tiptap Statusbar Extension382
- Umbraco Tiptap Toolbar Extension382
- Umbraco Validation Checks382
- Umbraco Ufm Component381
- Umbraco Validation Context381
- Umbraco E2e Testing380
- Umbraco Msw Testing379
- Umbraco Unit Testing374
- Umbraco Test Builders370
- Umbraco Testing368
- Umbraco Backoffice447
- Umbraco Controllers418
- Umbraco Dashboard418
- Umbraco Extension Template413
- Umbraco Quickstart406
- Umbraco Openapi Client405
- Umbraco Property Editor Ui405
- Umbraco Extension Registry403
- Umbraco Notifications403
Recommended
More Testing & QA
umbraco-unit-testingumbraco/umbraco-cms-backoffice-skillsUnit and component testing for Umbraco backoffice extensions using @open-wc/testing374skills.sh installsumbraco-test-buildersumbraco/umbraco-cms-backoffice-skillsJsonModels.Builders for creating test data in Umbraco tests370skills.sh installsumbraco-testingumbraco/umbraco-cms-backoffice-skillsRouter skill for choosing the right testing approach for Umbraco backoffice extensions368skills.sh installs
android-testing-unitkrutikjain/android-agent-skillsWrite fast, focused Android unit tests for reducers, use cases, repositories, and lifecycle-safe state holders.339skills.sh installs
javascript-testing-patternssickn33/antigravity-awesome-skillsjavascript testing patterns308skills.sh installs