Pi Computer Use

Semantic computer control for macOS that uses Accessibility API references instead of raw coordinates. You take a screenshot to get tagged elements like @e1 and @e2, then click or fill them without worrying about pixel positions. It falls back to coordinates when AX coverage is weak and batches actions to avoid round trips. The strict mode is clever: prevents focus stealing so you can automate apps in the background without your cursor jumping around. Needs Accessibility and Screen Recording permissions on first run. If you're automating Mac GUI tasks with Pi and want something more reliable than coordinate-based clicking, this is the right approach.

skills.sh installs: 206
GitHub stars: 26
last updated: Jun 3, 2026

View on GitHub

Install

$ npx -y skills add aradotso/trending-skills --skill pi-computer-use --agent claude-code

Installs into .claude/skills of the current project.

Files

pi-computer-use

Skill by ara.so — Daily 2026 Skills collection.

pi-computer-use gives Pi agents a semantic computer-use surface for visible macOS windows. It prefers Accessibility (AX) targets (like @e1) over raw coordinates, returns semantic state after every action, and attaches screenshots only when AX coverage is too weak.

Installation

Via Pi (recommended)

pi install git:github.com/injaneity/pi-computer-use#v0.2.1

Pin to a specific version:

pi install -l git:github.com/injaneity/pi-computer-use#v0.2.1

Via npm

npm install @injaneity/pi-computer-use
# or pin a version
npm install @injaneity/pi-computer-use@0.2.1

Remove

pi remove git:github.com/injaneity/pi-computer-use#v0.2.1
npm remove @injaneity/pi-computer-use

First-Run Permissions

On first session, macOS will prompt for permissions for:

~/.pi/agent/helpers/pi-computer-use/bridge

Grant both:

Accessibility — required for AX ref targeting
Screen Recording — required for screenshots

How It Works

Three components:

SKILL.md on GitHub

Details

Repository: github.com/aradotso/trending-skills
First seen: Jun 3, 2026