yu-tao-li/dsh-computer-use-win
Developer toolsWindows computer use for DeepSeek Harness (dsh): MCP stdio server + PowerShell UIA backend, 22 desktop tools
Overview
：在“设置 → Computer Use”选择“懒启动”，启动只加载 1 项管理工具，桌面工具按需加载，仍允许全部工具。现有默认方案保持“全部”。详见工具加载。 让 DeepSeek Harness 的 agent 读取并操作真实 Windows 应用：找控件、看截图、识别文字、点击、输入和管理窗口。 Windows 交互桌面和已安装的 DSH。默认使用系统自带的 Windows PowerShell 5.1；DSH Desktop 使用自带的 Electron 运行时，独立 Node 宿主需要 Node.js ≥ 22。 2. 完整退出并重新启动 DSH Desktop，打开 。也可点击插件列表中的简介进入具体设置。 3. 默认全部加载，直接开始一个桌面任务；需要减少工具时，在设置里选择“精简”或“自定义”。 升级已有安装也使用同一条安装命令，随后完整重启 Desktop。安装与升级请核对宿主实际使用的 profile；安装到另一个 profile 不会改变当前宿主的插件。 安装后重启 dsh web。使用其他 profile 时，将 web 替换为对应名称。也可在插件市场搜索 ，或使用 GitHub 源： 独立服务默认全部加载；WCU_TOOL_MODE=lite 使用精简集合，WCU_ENABLED_TOOLS 可指定 JSON 数组。DSH 专属的设置页、对话展示和整轮提示需要 DSH 宿主。更多配置见工具加载说明。 以下为 的真实界面。演示在独立会话中完成：新开浏览器窗口，搜索“晴天”，并确认结果中出现“周杰伦”。配图重新截取为原始像素的无损 PNG，仅保留相关对话内容或设置面板；点击图片可查看原图。
README
dsh-computer-use-win
让 DSH 看懂并操作你的 Windows 桌面。
您的 Star ⭐ ↗ 就是我更新的动力，球球了，这对我很重要！如有问题，欢迎去 提 Issue ↗，您来许愿，我来实现！
简体中文
0.5.0 新增懒启动：在“设置 → Computer Use”选择“懒启动”，启动只加载 1 项管理工具，桌面工具按需加载，仍允许全部工具。现有默认方案保持“全部”。详见工具加载。
让 DeepSeek Harness 的 agent 读取并操作真实 Windows 应用：找控件、看截图、识别文字、点击、输入和管理窗口。
23 项桌面能力 + 1 个工具管理入口 · 明确的控制提示 · 按 Esc 停止当前轮 · 中英文设置
快速开始 · 界面预览 · 能做什么 · 按你的习惯设置 · 使用示例 · 工具与进阶 · 常见问题 · English
快速开始
需要： Windows 交互桌面和已安装的 DSH。默认使用系统自带的 Windows PowerShell 5.1；DSH Desktop 使用自带的 Electron 运行时，独立 Node 宿主需要 Node.js ≥ 22。
-
在 DSH Desktop 自带终端中执行：
dsh plugin --profile desktop add dsh-computer-use-win -
完整退出并重新启动 DSH Desktop，打开 设置 → Computer Use。也可点击插件列表中的简介进入具体设置。
-
默认全部加载，直接开始一个桌面任务；需要减少工具时，在设置里选择“精简”或“自定义”。
升级已有安装也使用同一条安装命令，随后完整重启 Desktop。安装与升级请核对宿主实际使用的 profile；安装到另一个 profile 不会改变当前宿主的插件。
界面预览
以下为 DSH Desktop 2.0.18 + 插件 0.4.3 的真实界面。演示在独立会话中完成：新开浏览器窗口，搜索“晴天”，并确认结果中出现“周杰伦”。配图重新截取为原始像素的无损 PNG，仅保留相关对话内容或设置面板；点击图片可查看原图。
对话中的操作记录。 展开工具调用后，可看到带鼠标图标的“输入了文本”“发送了按键”“检查了操作结果”等记录，点击操作行可进入“查看步骤和截图”。
工具加载。 外层先选精简、全部或自定义。下图仅截取暗色设置面板，展示默认加载全部 24 项工具的方案。
能做什么
| 你想完成的事 | 插件提供的能力 |
|---|---|
| 找到窗口、按钮和输入框 | 原生窗口枚举、UIA 无障碍树、控件查询 |
| 读取自绘界面、canvas 或截图中的文字 | Windows OCR、文字坐标、截图；稀疏 UIA 查询可自动补 OCR |
| 点击、输入和填写表单 | UIA 语义动作、鼠标键盘输入、保留焦点的文本粘贴 |
| 等待页面变化、确认操作效果 | 等待窗口、控件或可见文字，再读取或截图确认 |
| 看清 agent 正在做什么 | 屏幕提示、鼠标旁 DSH 标识、对话步骤与结束摘要 |
在 DSH 对话中，桌面工具使用鼠标图标和简洁描述，点击可展开步骤及相关截图，结束后汇总这一轮的操作。已发送、已验证、未完成和已停止会区分展示；重开对话可恢复已有记录。浏览器搜索摘要根据实际输入、提交和后续观察生成。
按你的习惯设置
在 设置 → Computer Use 中调整，点击“保存更改”后生效。界面适配明暗主题，保存结果会向宿主确认并回读；无法读取配置时会明确提示。
| 设置 | 你可以选择什么 |
|---|---|
| 工具加载 | 全部 24 项（默认）、精简 10 项、懒启动初始 1 项（允许全部），或展开“自定义”逐项／分组选取；支持全选和全部关闭 |
| 屏幕提示 | 开关屏幕边缘与提示条，设置文字、顶部／底部位置及结束后 200–3000 毫秒的隐藏延时 |
| 鼠标提示 | 相对屏幕提示独立开关，显示鼠标光环和 DSH 标识；通过叠加层显示，保留系统光标主题 |
| 远程与录屏可见 | 默认开启，支持 UU 虚拟屏幕；随活动显示器、布局和 DPI 变化刷新 |
| 自定义停止方式 | 点击“录制快捷键”，按 F8、Ctrl + Alt + K 等按键或组合键并保存；“恢复默认（Esc）”清除额外快捷键 |
| 预览 | 保存后点击“预览接管提示”，检查当前屏幕和鼠标提示的效果 |
按 Esc 停止当前轮。 物理 Esc 和自定义快捷键会停止这一轮及排队的桌面操作，同一轮不会自动重试。下一次发起任务或发送“继续”即可再次操作，无需去设置里恢复。普通鼠标移动和其他按键不会触发停止；关闭提示仍保留停止监听。
提示覆盖连续操作，也会及时收起。 从首次桌面工具调用开始，提示覆盖截图、读取和短暂思考间隔；轮次完成、取消或失败后按隐藏延时收起。最后一次桌面调用完成后若空闲 30 秒，也会收起提示并释放占用，下一次调用重新显示；正在执行的工具不计为空闲。独立 stdio 客户端按单次调用显示，因为它没有 DSH 的轮次边界。
支持的控件动作优先使用 UIA，文本粘贴不额外移动鼠标。坐标点击和前台输入仍共享当前 Windows 桌面；另一段对话占用桌面时会返回忙碌状态。已送达应用的动作无法撤回，阻塞的应用调用需要先返回。
自定义清单在切换方案时保留，保存后在当前动作结束时应用，并在重启后保留。agent 无法启用用户关闭的工具。完整行为见设置与停止说明。
使用示例
装好后，可以把这样的任务交给 agent：
在记事本中输入“会议纪要”，再读取文本确认写入成功。
在已打开的计算器中计算 25 × 4，并读取结果。
打开浏览器，搜索“Windows UI Automation”，确认出现搜索结果，并总结你做了什么。
打开网易云音乐，搜索“晴天”，确认结果页出现“周杰伦”。如果无障碍树看不到搜索框，用 OCR 定位可见文字。
一次可靠的操作是 观察 → 定位 → 执行 → 确认结果。verification: "sent" 表示输入已发送；wait_for 返回 verification: "observed" 表示观察到了指定条件。确认应针对任务真正产生的效果。目标应用、模型能力和当前页面都会影响结果。
工具与进阶
共 23 项桌面工具 + 1 个管理入口。下表使用短名，完整名称为 windows_computer_use_，在 DSH 中通常再加 mcp__wincu__ 前缀。
常见问题
数据与隐私
UIA、OCR、截图和输入在本机执行，结果交给 DSH 或其他 MCP 调用方。桌面内容是否进一步发送给远端模型，取决于宿主、模型和服务配置。截图及可见文字可能包含当前窗口里的敏感内容，请在分享日志或 Issue 前遮盖账号、路径、凭证和私人对话。
文档与开发
设置与停止 · 工具加载 · 工具参数 · 架构 · 测试记录 · 更新日志
MIT 许可。基于 cgissing/windows-computer-use 的衍生作品，来源与版权见 THIRD_PARTY.md。
English
New in 0.5.0: Lazy startup. Select Lazy in Settings → Computer Use to start with one management tool and load desktop tools on demand. All tools remain allowed, and All remains the default preset. See tool loading.
Give a DeepSeek Harness agent access to real Windows apps: find controls, inspect screenshots, read visible text, click, type and manage windows.
23 desktop tools + 1 management entry · Visible control status · Esc stops the current turn · Chinese and English settings
Quick start · Interface tour · What it can do · Make it yours · Try a task · Tools and options · FAQ · 简体中文
Quick start
Requires: an interactive Windows desktop and DSH. Windows PowerShell 5.1, included with Windows, is the default backend. DSH Desktop uses its bundled Electron runtime; standalone Node hosts require Node.js ≥ 22.
-
Run this in DSH Desktop’s built-in terminal:
dsh plugin --profile desktop add dsh-computer-use-win -
Fully quit and restart DSH Desktop, then open Settings → Computer Use. The introduction in the plugin list also opens the complete settings page.
-
All tools are loaded by default. Start a desktop task, or choose Lite, Lazy or Custom in settings to load fewer tools.
Use the same install command to upgrade, then fully restart Desktop. Check which profile your host actually uses: installing into another profile does not update the active host’s plugin.
Interface tour
These are real screenshots from DSH Desktop 2.0.18 with plugin 0.4.3, using the Chinese interface. In a separate demonstration conversation, the agent opened a new browser window, searched for “晴天” and confirmed “周杰伦” in the results. The images were recaptured as lossless PNGs at native pixel resolution, showing only the relevant conversation content or settings panel. Click an image to view the original.
Actions in the conversation. Expand tool calls to see mouse-icon rows for typing, key presses and result checks. Expand an action row for its Steps and screenshots entry.
Tool loading. Choose Lite, All or Custom first. This image shows only the dark settings panel, with the default All preset and its 24 tools.
What it can do
| Your task | Available capabilities |
|---|---|
| Find windows, buttons and text fields | Native window discovery, UIA accessibility trees and control queries |
| Read custom interfaces, canvas content or screenshots | Windows OCR, text coordinates and screenshots; sparse UIA searches can fall back to OCR |
| Click, type and fill forms | UIA semantic actions, mouse/keyboard input and text pasting that preserves focus |
| Wait for changes and confirm results | Wait for a window, control or visible text, then read or capture the result |
| See what the agent is doing | Screen indicators, a DSH pointer badge, conversation steps and a final summary |
In DSH conversations, desktop calls use a mouse icon and readable descriptions. Expand them for steps and related screenshots; a summary collects the turn’s actions. Sent input, verified effects, incomplete tasks and stopped operations are distinguished, and existing records are restored when reopening a conversation. Browser search summaries use actual input, submission and subsequent observations.
Make it yours
Open Settings → Computer Use, make your choices and click Save changes. The interface supports light and dark themes. Saved values are confirmed and read back from the host; unavailable configuration is shown explicitly.
| Setting | Your choices |
|---|---|
| Tool loading | All: 24 (default), Lite: 10, Lazy: 1 at startup with all allowed, or an expandable Custom checklist with individual/group selection, select-all and disable-all |
| Screen indicators | Toggle the screen edges/banner; customize text, top/bottom placement and a 200–3000ms delay before hiding after completion |
| Pointer indicator | Toggle the pointer halo/DSH badge independently of the screen indicator; overlays preserve your Windows cursor theme |
| Visible in remote sessions and recordings | On by default, including UU virtual displays; refreshes as active monitors, layout and DPI change |
| Custom stop shortcut | Click Record shortcut, press a key or combination such as F8 or Ctrl + Alt + K, and save; Restore default (Esc) clears the additional shortcut |
| Preview | Save, then preview the current screen and pointer indicators |
Press Esc to stop the current turn. Physical Esc or your custom shortcut stops that turn and its queued desktop actions. It cannot automatically retry in the same turn. A new task or a request to continue can use the desktop again without visiting settings. Ordinary mouse movement and other keys do not stop control. Hiding the indicators keeps the stop listener active.
Indicators span a series of operations and clear when idle. They appear with the first desktop call and stay through screenshots, reads and short reasoning gaps. They hide after the configured delay when the turn finishes, is cancelled or fails. After 30 idle seconds following the last completed desktop call, they also hide and release ownership; the next call shows them again. Running tools do not count as idle. Standalone stdio clients use per-call indicators because they provide no DSH turn boundary.
Supported control actions prefer UIA, and pasting text does not reposition the mouse. Coordinate clicks and foreground input still share the Windows desktop. Another conversation receives a busy result while it is owned. Delivered actions cannot be undone, and a blocking application call must return before cancellation can finish.
Switching presets preserves your custom checklist. Saving applies changes after the current action finishes and persists them across restart. Agents cannot enable tools you disabled. See settings and stopping for the complete behavior.
Try a task
After installation, ask the agent to do something like:
Type “Meeting notes” into Notepad, then read the text back to confirm it was entered.
Calculate 25 × 4 in the open Calculator window and read the result.
Open a browser, search for “Windows UI Automation”, confirm that results appeared, and summarize what you did.
Open NetEase Cloud Music, search for “晴天”, and confirm that “周杰伦” appears in the results. If UIA cannot find the search box, use OCR to locate visible text.
A reliable sequence is observe → locate → act → confirm. verification: "sent" means input was dispatched; wait_for returning verification: "observed" means the specified condition was observed. Check the actual effect required by the task. Results depend on the target app, model and current page state.
Tools and options
There are 23 desktop tools + 1 management entry. Short names appear below; complete names use windows_computer_use_, usually prefixed with mcp__wincu__ in DSH.
FAQ
Data and privacy
UIA, OCR, capture and input execute locally. Results are passed to DSH or the MCP caller; whether desktop content is then sent to a remote model depends on the host, model and service configuration. Screenshots and visible text may include sensitive window contents. Redact accounts, paths, credentials and private conversations before sharing logs or issues.
Documentation and development
Settings and stopping · Tool loading · Tool reference · Architecture · Test notes · Changelog
MIT licensed. Derived from cgissing/windows-computer-use; see THIRD_PARTY.md for attribution and copyright.
Plugin marketplace · Releases · Report an issue · Back to English ↑
Install
npx -y dsh-computer-use-winConfiguration
{
"mcpServers": {
"wincu": {
"command": "npx",
"args": ["-y", "dsh-computer-use-win"]
}
}
}