agent-desktop
An open-source CLI that lets any AI agent use desktop apps through the operating system's accessibility tree instead of screenshots.
Vercel Labs published agent-desktop v1.0.0 on October 28, 2026 UTC. The latest release on the releases page is v1.0.2, dated October 30, 2026. It is a CLI and C library for macOS, Windows, and Linux. A snapshot returns JSON with a ref for every element an agent can act on. The agent acts on a ref, including click, type, select, and scroll, then snapshots again to check. It does not use pixel matching. It is headless by default: actions go through the accessibility API without moving the pointer or taking focus. A --headed flag allows physical input. Install with npm install -g agent-desktop. The skill for agent harnesses is npx skills add https://github.com/vercel-labs/agent-desktop --skill agent-desktop. macOS needs Accessibility permission. Linux needs the graphical session with at-spi2-core. Windows needs no grant for apps at the terminal's integrity level. The README says snapshots, clipboard reads, and screenshots can contain private desktop content, so keep them out of public logs. The license is Apache 2.0. It is free. Last checked November 2, 2026.
Where it ranks
25 in AI agents
An open-source CLI that drives desktop apps through the accessibility tree.
See the ai agents list25 in Vibe coding
A desktop CLI any coding agent can call without screenshots.
See the vibe coding list
Pricing
Free
Free. Apache 2.0.
Checked
Added November 2, 2026
Last verified November 2, 2026
Dated releases
Also in ai agents
- n8n · Workflow automation you can self-host. Connect apps and add AI steps you can see.
- Dify · Dify is a platform for building, deploying, and managing AI agents and workflows, connecting models, knowledge, and tools in one workspace.
- CrewAI · Framework for multi-agent jobs, plus a hosted product for teams that need control.