一个Windows桌面自动化服务器，可以暴露UI自动化、屏幕截图、合成功能 输入、OCR、屏幕上的视觉和系统控制通过MCP（标准和 Streamable-HTTP） 以及localhost HTTP API，所有这些都位于kill开关和仅附加的审计日志之后。基于 .NET 9 + FlaUI（UIA3）。（完整架构文档请参见 Deskhand.html 。）

两个主机（HTTP 和 MCP）都是覆盖在一个受控后端（IAutomationBackend）上的薄壳，因此它们 暴露相同的能力并记录到相同的审计轨迹。当前表面为**~136 MCP工具和~161个HTTP路由**。

该构建涵盖单机默认桌面路径（会话内UIA + capture + 输入）， 安全桌面捕获（UAC/锁定/登录，通过系统助手），以及治理。有一点是 仅捕获，并标注出现在哪里：驱动输入 在安全桌面上 — 一个硬的Windows 边界（捕获有效;即使是 SYSTEM 的 uiAccess 进程，合成输入也被拒绝）。

安全桌面输入： Deskhand可以查看安全桌面，并附带一个选择加入的尽力而为方案 去驱动它。

Deskhand — Windows desktop automation for AI agents

A Windows desktop-automation server that exposes UI Automation, screen capture, synthetic input, OCR, on-screen vision, and system control over MCP (stdio and Streamable-HTTP) and a localhost HTTP API, all behind a kill switch and an append-only audit log. Built on .NET 9 + FlaUI (UIA3). (See Deskhand.html for the full architecture doc.)

Both hosts (HTTP and MCP) are thin shells over one governed backend (IAutomationBackend), so they expose identical capabilities and record to the same audit trail. The current surface is ~136 MCP tools and ~161 HTTP routes.

The build covers the single-machine Default desktop path (in-session UIA + capture + input), secure-desktop capture (UAC / lock / logon, via a SYSTEM helper), and governance. One thing is capture-only and called out where it appears: driving input on the secure desktop — a hard Windows boundary (capture works; synthetic input is refused even for a SYSTEM, uiAccess process).

Secure-desktop input: Deskhand can view the secure desktop, and ships an opt-in, best-effort path to drive it. Testing showed the UAC consent prompt cannot be driven by synthetic input by any mechanism we can deploy on demand (SYSTEM, a self-granted uiAccess token, or a signed uiAccess binary all hit ACCESS_DENIED) — it’s a deliberate Windows boundary. To automate a UAC approval, move the prompt off the secure desktop with /uac/config {promptOnSecureDesktop:false} (one-click in the dashboard). Full write-up: docs/secure-desktop-input.md.

Deskhand vs. a CUA driver

A CUA driver (computer-using agent — Anthropic Computer Use, OpenAI’s Computer-Using Agent / Operator, trycua/cua, and similar) drives a machine the way a person looks at a screen: the model gets a screenshot, reasons over the pixels, and emits click(x, y) / type(...) actions. It is OS-agnostic and needs no knowledge of the app — but every step costs a vision round-trip, and the click lands wherever the model guessed the pixel was.

Deskhand starts from the opposite end — structure before pixels — and keeps the pixel path as a fallback rather than the whole story. It is not itself an agent loop; it’s the tool surface you point your own agent at (over MCP or HTTP).