Skill

Fail

Audit score 45

peekaboo

steipete/clawdis

Capture and automate macOS UI with CLI commands for screenshots, element targeting, and input control.

What is peekaboo?

Peekaboo is a full macOS UI automation CLI that captures screens, inspects UI elements, drives input (clicks, typing, gestures), and manages apps/windows/menus. Use it when you need to automate macOS interactions, test UI flows, or extract visual information from the screen.

Capture live video and screenshots with frame extraction and optional AI analysis
Inspect and annotate UI elements with snapshot IDs for reliable targeting
Click, drag, type, scroll, and gesture on UI elements or coordinates with human-like motion profiles
Manage apps, windows, menus, Dock items, and Spaces programmatically
Execute `.peekaboo.json` scripts for complex automation workflows
Output results as JSON for scripting and integration with agents

How to install peekaboo

npx skills add https://github.com/steipete/clawdis --skill peekaboo

Prerequisites

macOS system (darwin)
Peekaboo CLI installed via Homebrew: `brew install steipete/tap/peekaboo`
Screen Recording and Accessibility permissions granted to the terminal or OpenClaw app
Optional: OpenClaw app for bridged socket access to shared permissions

Claude Code

Cursor

Windsurf

Cline

How to use peekaboo

1.Run `peekaboo permissions` to verify Screen Recording and Accessibility grants
2.Use `peekaboo see --annotate --path /tmp/output.png` to capture and annotate the current screen
3.Identify target element IDs from the annotated screenshot
4.Execute interactions with `peekaboo click --on <ID>`, `peekaboo type "text"`, or other commands
5.Chain commands together or use `peekaboo run` with a `.peekaboo.json` script for complex workflows
6.Add `--json` flag to any command for machine-readable output suitable for agent parsing

Use cases

Good for

Automate login flows and form filling across macOS applications
Test UI interactions by capturing screenshots, analyzing them, and clicking identified elements
Extract and analyze dashboard or report data from application windows
Automate repetitive tasks like menu navigation and window management
Record and replay complex user interaction sequences for testing or documentation

Who it's for

macOS automation engineers and QA testers
AI agents and coding assistants automating desktop workflows
DevOps and infrastructure teams managing macOS systems
Developers building UI testing frameworks

peekaboo FAQ

How do I target UI elements reliably?

Use `peekaboo see --annotate` to capture and label all interactive elements with IDs, then reference those IDs with `--on` or `--id` flags in interaction commands. This is more reliable than coordinates.

What permissions does Peekaboo need?

Screen Recording and Accessibility permissions. Check status with `peekaboo permissions`. If running via OpenClaw, set `PEEKABOO_BRIDGE_SOCKET` to use the app's existing grants instead of requesting new ones.

Can I use Peekaboo in scripts or with agents?

Yes. All commands support `--json` output for parsing, and you can create `.peekaboo.json` scripts with `peekaboo run`. The CLI is designed for automation and integration.

How do I handle timing and human-like interactions?

Use `--duration`, `--delay`, `--steps`, and `--profile human|linear` flags on motion and typing commands to control speed and smoothness.

What should I do if capture fails in a subprocess?

Diagnose with `peekaboo bridge status --json` and `peekaboo permissions status --json`. Ensure `PEEKABOO_BRIDGE_SOCKET` is set correctly if running via OpenClaw.

Full instructions (SKILL.md)

Source of truth, from steipete/clawdis.

name: peekaboo description: "Capture and automate macOS UI with the Peekaboo CLI." homepage: https://peekaboo.boo metadata: { "openclaw": { "emoji": "👀", "os": ["darwin"], "requires": { "bins": ["peekaboo"] }, "install": [ { "id": "brew", "kind": "brew", "formula": "steipete/tap/peekaboo", "bins": ["peekaboo"], "label": "Install Peekaboo (brew)", }, ], }, }

Peekaboo

Peekaboo is a full macOS UI automation CLI: capture/inspect screens, target UI elements, drive input, and manage apps/windows/menus. Commands share a snapshot cache and support --json/-j for scripting. Run peekaboo or peekaboo <cmd> --help for flags; peekaboo --version prints build metadata. Tip: run via polter peekaboo to ensure fresh builds.

OpenClaw Bridge

The OpenClaw macOS app hosts Peekaboo Bridge at ~/Library/Application Support/OpenClaw/bridge.sock. Before running Peekaboo from OpenClaw, select that socket so the CLI uses the app's Screen Recording and Accessibility grants instead of starting its standalone daemon:

export PEEKABOO_BRIDGE_SOCKET="${PEEKABOO_BRIDGE_SOCKET:-$HOME/Library/Application Support/OpenClaw/bridge.sock}"

Confirm routing with peekaboo bridge status --json; hostKind must be gui and the socket path must end in OpenClaw/bridge.sock.

Features (all CLI capabilities, excluding agent/MCP)

Core

bridge: inspect Peekaboo Bridge host connectivity
capture: live capture or video ingest + frame extraction
clean: prune snapshot cache and temp files
config: init/show/edit/validate, providers, models, credentials
image: capture screenshots (screen/window/menu bar regions)
learn: print the full agent guide + tool catalog
list: apps, windows, screens, menubar, permissions
permissions: check Screen Recording/Accessibility status
run: execute .peekaboo.json scripts
sleep: pause execution for a duration
tools: list available tools with filtering/display options

Interaction

click: target by ID/query/coords with smart waits
drag: drag & drop across elements/coords/Dock
hotkey: modifier combos like cmd,shift,t
move: cursor positioning with optional smoothing
paste: set clipboard -> paste -> restore
press: special-key sequences with repeats
scroll: directional scrolling (targeted + smooth)
swipe: gesture-style drags between targets
type: text + control keys (--clear, delays)

System

app: launch/quit/relaunch/hide/unhide/switch/list apps
clipboard: read/write clipboard (text/images/files)
dialog: click/input/file/dismiss/list system dialogs
dock: launch/right-click/hide/show/list Dock items
menu: click/list application menus + menu extras
menubar: list/click status bar items
open: enhanced open with app targeting + JSON payloads
space: list/switch/move-window (Spaces)
visualizer: exercise Peekaboo visual feedback animations
window: close/minimize/maximize/move/resize/focus/list

Vision

see: annotated UI maps, snapshot IDs, optional analysis

Global runtime flags

--json/-j, --verbose/-v, --log-level <level>
--no-remote, --bridge-socket <path>

Quickstart (happy path)

peekaboo permissions
peekaboo list apps --json
peekaboo see --annotate --path /tmp/peekaboo-see.png
peekaboo click --on B1
peekaboo type "Hello" --return

Common targeting parameters (most interaction commands)

App/window: --app, --pid, --window-title, --window-id, --window-index
Snapshot targeting: --snapshot (ID from see; defaults to latest)
Element/coords: --on/--id (element ID), --coords x,y
Focus control: --no-auto-focus, --space-switch, --bring-to-current-space, --focus-timeout-seconds, --focus-retry-count

Common capture parameters

Output: --path, --format png|jpg, --retina
Targeting: --mode screen|window|frontmost, --screen-index, --window-title, --window-id
Analysis: --analyze "prompt", --annotate
Capture engine: --capture-engine auto|classic|cg|modern|sckit

Common motion/typing parameters

Timing: --duration (drag/swipe), --steps, --delay (type/scroll/press)
Human-ish movement: --profile human|linear, --wpm (typing)
Scroll: --direction up|down|left|right, --amount <ticks>, --smooth

Examples

See -> click -> type (most reliable flow)

peekaboo see --app Safari --window-title "Login" --annotate --path /tmp/see.png
peekaboo click --on B3 --app Safari
peekaboo type "user@example.com" --app Safari
peekaboo press tab --count 1 --app Safari
peekaboo type "supersecret" --app Safari --return

Target by window id

peekaboo list windows --app "Visual Studio Code" --json
peekaboo click --window-id 12345 --coords 120,160
peekaboo type "Hello from Peekaboo" --window-id 12345

Capture screenshots + analyze

peekaboo image --mode screen --screen-index 0 --retina --path /tmp/screen.png
peekaboo image --app Safari --window-title "Dashboard" --analyze "Summarize KPIs"
peekaboo see --mode screen --screen-index 0 --analyze "Summarize the dashboard"

Live capture (motion-aware)

peekaboo capture live --mode region --region 100,100,800,600 --duration 30 \
  --active-fps 8 --idle-fps 2 --highlight-changes --path /tmp/capture

App + window management

peekaboo app launch "Safari" --open https://example.com
peekaboo window focus --app Safari --window-title "Example"
peekaboo window set-bounds --app Safari --x 50 --y 50 --width 1200 --height 800
peekaboo app quit --app Safari

Menus, menubar, dock

peekaboo menu click --app Safari --item "New Window"
peekaboo menu click --app TextEdit --path "Format > Font > Show Fonts"
peekaboo menu click-extra --title "WiFi"
peekaboo dock launch Safari
peekaboo menubar list --json

Mouse + gesture input

peekaboo move 500,300 --smooth
peekaboo drag --from B1 --to T2
peekaboo swipe --from-coords 100,500 --to-coords 100,200 --duration 800
peekaboo scroll --direction down --amount 6 --smooth

Keyboard input

peekaboo hotkey --keys "cmd,shift,t"
peekaboo press escape
peekaboo type "Line 1\nLine 2" --delay 10

Notes

Requires Screen Recording + Accessibility permissions.
In OpenClaw subprocesses, set PEEKABOO_BRIDGE_SOCKET as shown above. Do not pass --no-remote unless the calling process has its own Screen Recording grant.
Diagnose subprocess capture failures with peekaboo bridge status --json, then peekaboo permissions status --json, then a normal Bridge-routed capture such as peekaboo image --mode screen --json.
On macOS 15+, the "bypass private window picker" prompt is separate from the base Screen Recording grant; it can appear even when Bridge permissions are otherwise correct.
Use peekaboo see --annotate to identify targets before clicking.