peekaboo-cli

# Peekaboo CLI macOS UI automation command-line tool providing complete desktop control capabilities. ## Installation Copy this skill to your agent's skills directory: ```bash # Claude Code mkdir -p ~/.claude/skills/peekaboo-cli cp -r . ~/.claude/skills/peekaboo-cli/ # OpenClaw mkdir -p ~/.openclaw/skills/peekaboo-cli cp -r . ~/.openclaw/skills/peekaboo-cli/ ``` ## Prerequisites - [Peekaboo CLI installed](https://github.com/steipete/Peekaboo#install) (`brew install steipete/tap/peekaboo`) - macOS Screen Recording and Accessibility permissions granted ## Pre-execution Validation Before executing any CLI commands, verify permission status first. ### 1. Check Permission Status ```bash peekaboo permissions status ``` ### 2. Grant Permissions (if needed) If permissions are not granted, guide the user to enable them: ``` Peekaboo requires the following macOS permissions: - Screen Recording: For capturing UI screenshots - Accessibility: For controlling UI elements System Settings → Privacy & Security → Screen Recording/Accessibility → Add Terminal app ``` Then verify: ```bash peekaboo permissions grant ``` ### 3. Verify Configuration ```bash peekaboo list apps ``` ## Quick Start ```bash # List running applications peekaboo list apps # Capture current window UI with element annotations peekaboo see --annotate --path /tmp/ui.png # Click specific element (ID from see command) peekaboo click --on elem_42 # Type text peekaboo type "Hello World" --return ``` ## Core Commands ### UI Capture & Analysis ```bash # Capture UI element map (recommended before click/type) peekaboo see --json --annotate --path /tmp/ui.png # Target specific app/window peekaboo see --app "Safari" --window-title "GitHub" # Capture menubar popovers peekaboo see --menubar # Capture entire screen peekaboo see --mode screen ``` ### Element Interaction ```bash # Click element by ID peekaboo click --on elem_42 peekaboo click "Submit" --wait-for 8000 # Coordinate-based click peekaboo click --coords 100,200 # Double-click / Right-click peekaboo click --on elem_42 --double peekaboo click --coords 100,200 --right # Type text peekaboo type "user@example.com" --return peekaboo type --clear "new text" # Clear field first # Key presses and hotkeys peekaboo press return peekaboo hotkey "cmd,w" # Cmd+W ``` ### App Management ```bash # Launch app peekaboo app launch "Safari" peekaboo app launch "Xcode" --open ~/Projects/MyApp.xcodeproj # Quit app peekaboo app quit --app "Safari" peekaboo app quit --all --except "Finder,Terminal" # Switch focus peekaboo app switch --to Safari --verify ``` ### Window Management ```bash # List windows peekaboo list windows --app "Safari" --include-details bounds,ids # Move/Resize window peekaboo window move --app Safari -x 100 -y 100 peekaboo window resize --app Safari -w 1200 --height 800 # Focus/Close/Minimize/Maximize peekaboo window focus --app Terminal --space-switch peekaboo window close --app Safari ``` ### Menu Operations ```bash # Click menu item peekaboo menu click --app Safari --path "File,New Tab" # List menubar status items peekaboo list menubar ``` ### Scroll & Drag ```bash # Scroll peekaboo scroll --direction down --amount 3 # Drag peekaboo drag --from 100,100 --to 300,300 # Move mouse peekaboo move --coords 500,500 ``` ## Common Workflows ### Form Filling ```bash # 1. Capture UI to get element IDs peekaboo see --json --annotate --path /tmp/form.png # 2. Click input field and type peekaboo click --on elem_10 peekaboo type "username" --tab peekaboo type "password" --return ``` ### App Automation ```bash # 1. Launch app peekaboo app launch "Safari" --wait-until-ready # 2. Wait and perform actions peekaboo sleep 2 peekaboo hotkey "cmd,l" # Focus address bar peekaboo type "https://github.com" --return ``` ## Output Format Most commands support `--json` output: ```bash peekaboo see --json | jq '.data.ui_elements[] | {id, label, role}' peekaboo list apps --json | jq '.data[] | select(.frontmost == true)' ``` ## Troubleshooting ### Permission Errors ```bash peekaboo permissions status peekaboo permissions grant ``` ### Element Not Found 1. Re-capture UI: `peekaboo see --json` 2. Use `--wait-for` to increase wait time 3. Use `--coords` for coordinate-based targeting ### Focus Issues 1. Use `--space-switch` for cross-Space windows 2. Use `peekaboo app switch --to <app> --verify` to confirm focus 3. Use `--no-auto-focus` when already focused manually ## Detailed Documentation * **UI Capture** [references/see.md](references/see.md) * **Element Click** [references/click.md](references/click.md) * **Text Input** [references/type.md](references/type.md) * **Key Press** [references/press.md](references/press.md) * **Hotkey** [references/hotkey.md](references/hotkey.md) * **App Management** [references/app.md](references/app.md) * **Window Management** [references/window.md](references/window.md) * **Menu Operations** [references/menu.md](references/menu.md) * **Menubar** [references/menubar.md](references/menubar.md) * **Scroll** [references/scroll.md](references/scroll.md) * **Drag** [references/drag.md](references/drag.md) * **Mouse Move** [references/move.md](references/move.md) * **Swipe Gesture** [references/swipe.md](references/swipe.md) * **Long-running Capture** [references/capture.md](references/capture.md) * **Image Processing** [references/image.md](references/image.md) * **Autonomous Agent** [references/agent.md](references/agent.md) * **MCP Server** [references/mcp.md](references/mcp.md) * **Config Management** [references/config.md](references/config.md) * **Permissions** [references/permissions.md](references/permissions.md) * **Clipboard** [references/clipboard.md](references/clipboard.md) * **List Queries** [references/list.md](references/list.md) * **Open Files** [references/open.md](references/open.md) * **Sleep** [references/sleep.md](references/sleep.md) * **Space Management** [references/space.md](references/space.md) * **Dialog** [references/dialog.md](references/dialog.md) * **Dock** [references/dock.md](references/dock.md) * **Clean Cache** [references/clean.md](references/clean.md) * **Bridge Service** [references/bridge.md](references/bridge.md) * **Daemon** [references/daemon.md](references/daemon.md) * **Tools** [references/tools.md](references/tools.md) * **Learn** [references/learn.md](references/learn.md) * **Run** [references/run.md](references/run.md) * **MCP Capture Meta** [references/mcp-capture-meta.md](references/mcp-capture-meta.md) * **Visualizer** [references/visualizer.md](references/visualizer.md)