OpenClaw技能：AI代理通过ADB+SSH接管Android/Linux设备——查看屏幕、发送输入、发布内容、自主玩游戏
概览
一个OpenClaw技能，让你的AI代理看到设备的屏幕，发送输入，通过扬声器说话，并通过麦克风监听。通过网络。实时。 你的AI代理连接到网络中的设备，并获得三项功能： 1. — 通过截图捕捉屏幕，用视觉模型分析屏幕内容 2.— 发送点击、滑动、按键、文本输入和手柄指令 3.— 通过设备扬声器进行TTS，麦克风录音通过Whisper转录 这些模式可组合成三种操作模式： - 在您的机器（Mac、Linux或Windows WSL）上运行——在同一网络上（或通过SSH/ADB访问）- adb（Android）、sshpass（Linux设备）、ffmpeg（音频处理）- Whisper API访问（语音输入）、ElevenLabs或本地TTS（语音输出） 连接安卓手机需要一些一次性设置。完整教程请见 SKILL.md，但简短版本如下： 1. — 关于手机的设置→→点击“组装编号7次”2.
概述（AI 翻译）
在任何设备上——Android、Steam Deck、Linux PC，都赋予你的AI眼睛、手和声音。
一个OpenClaw技能，让你的AI代理看到设备的屏幕，发送输入，通过扬声器说话，并通过麦克风监听。通过网络。实时。
它的作用
你的AI代理连接到网络中的设备，并获得三项功能：
- 看见——通过截图捕捉屏幕，用视觉模型分析屏幕上的内容
- Act — 发送点击、滑动、按键、文本输入和手柄指令
- 说话与聆听 — 通过设备扬声器进行语音转语音转，麦克风录音通过Whisper转录
这些模式可组合成三种操作模式：
| 模式 | 发生了什么 |
|---|---|
| 参见 + 行动循环 | 截图→视觉分析→决定动作→发送输入→重复 |
| 语音控制 | 录制麦克风→转录→解析意图→执行→语音回应 |
| 自主游戏玩法完整的游戏循环——AI看屏幕，按键，玩游戏 |
支持设备
| 设备 | 连接 | 截图方法 | 输入法 |
|---|
本页由 AI 翻译，技术细节请以原文为准。
README
以下为原文 README 节选，完整内容请查看原文。
Device Takeover
Give your AI eyes, hands, and a voice on any device — Android, Steam Deck, Linux PC.
An OpenClaw skill that lets your AI agent see a device’s screen, send inputs, speak through its speakers, and listen through its mic. Over the network. In real time.
What It Does
Your AI agent connects to a device on your network and gets three capabilities:
- See — captures the screen via screenshots, analyzes what’s on it with a vision model
- Act — sends taps, swipes, button presses, text input, and gamepad commands
- Speak & Listen — TTS through device speakers, mic recording transcribed via Whisper
These combine into three operating modes:
| Mode | What Happens |
|---|---|
| See + Act Loop | Screenshot → vision analysis → decide action → send input → repeat |
| Voice Control | Record mic → transcribe → parse intent → execute → speak response |
| Autonomous Gameplay | Full gameplay loop — AI watches the screen, presses buttons, plays the game |
Supported Devices
| Device | Connection | Screenshot Method | Input Method |
|---|---|---|---|
| Android phone/tablet | ADB over WiFi | adb exec-out screencap |
adb shell input tap/swipe/keyevent |
| Linux Handheld (SuiPlay, Steam Deck) | SSH | Gamescope DBus TakeScreenshot |
InputPlumber DBus SendButtonChord |
| Linux PC | SSH | import -window root / grim |
xdotool / ydotool |
What You Need
- OpenClaw running on your machine (Mac, Linux, or Windows WSL)
- Target device on the same network (or reachable via SSH/ADB)
- Dependencies:
adb(Android),sshpass(Linux devices),ffmpeg(audio processing) - Optional: Whisper API access (voice input), ElevenLabs or local TTS (voice output)
Android Setup (Quick Start)
Connecting an Android phone requires a few one-time settings. Full walkthrough in SKILL.md, but the short version:
- Enable Developer Options — Settings → About Phone → tap Build Number 7 times
- Enable Wireless Debugging — Settings → Developer Options → Wireless Debugging ON
- Pair once — tap “Pair device with pairing code”, then run
adb pair IP:PORTon your computer - Connect —
adb connect IP:PORT - Test —
adb exec-out screencap -p > /tmp/screen.png
See SKILL.md for the complete step-by-step including troubleshooting.
Use Cases
1. Hands-Free Phone Control
“Read my notifications. Open the camera. Navigate to that app.” Your AI sees the phone screen, navigates the UI, and performs actions — all from a text or voice command.
2. AI Game Assistant
Point your AI at a Steam Deck or SuiPlay running a game. It watches the screen, identifies what’s happening, and either suggests moves or plays autonomously — pressing buttons through the gamepad API.
3. Remote Linux Automation
推荐工具
换一个关键词，或者移除筛选条件。
安装
npx skillfish add oh-ashen-one/device-takeover