Device Takeover

Give your AI eyes, hands, and a voice on any device — Android, Steam Deck, Linux PC.

An OpenClaw skill that lets your AI agent see a device’s screen, send inputs, speak through its speakers, and listen through its mic. Over the network. In real time.

What It Does

Your AI agent connects to a device on your network and gets three capabilities:

1. See — captures the screen via screenshots, analyzes what’s on it with a vision model
2. Act — sends taps, swipes, button presses, text input, and gamepad commands
3. Speak & Listen — TTS through device speakers, mic recording transcribed via Whisper

These combine into three operating modes:

Supported Devices

What You Need

• OpenClaw running on your machine (Mac, Linux, or Windows WSL)
• Target device on the same network (or reachable via SSH/ADB)
• Dependencies: adb (Android), sshpass (Linux devices), ffmpeg (audio processing)
• Optional: Whisper API access (voice input), ElevenLabs or local TTS (voice output)

Android Setup (Quick Start)

Connecting an Android phone requires a few one-time settings. Full walkthrough in SKILL.md, but the short version:

1. Enable Developer Options — Settings → About Phone → tap Build Number 7 times
2. Enable Wireless Debugging — Settings → Developer Options → Wireless Debugging ON
3. Pair once — tap “Pair device with pairing code”, then run adb pair IP:PORT on your computer
4. Connect — adb connect IP:PORT
5. Test — adb exec-out screencap -p > /tmp/screen.png

See SKILL.md for the complete step-by-step including troubleshooting.

Use Cases

1. Hands-Free Phone Control

“Read my notifications. Open the camera. Navigate to that app.” Your AI sees the phone screen, navigates the UI, and performs actions — all from a text or voice command.

2. AI Game Assistant

Point your AI at a Steam Deck or SuiPlay running a game. It watches the screen, identifies what’s happening, and either suggests moves or plays autonomously — pressing buttons through the gamepad API.

3. Remote Linux Automation

SSH into a Linux PC, take screenshots, read what’s on screen, click through UIs, run commands. Monitor a dashboard. Fill out a form. Interact with any GUI application remotely.

Quick Start

# 1. Run the setup script
bash scripts/setup.sh

# 2. Follow the prompts — it installs dependencies, connects to your device, runs a test screenshot

# 3. Start the game loop (or use any mode from SKILL.md)
bash scripts/game-loop.sh android YOUR_DEVICE_IP:PORT

File Structure

├── SKILL.md              # Full skill instructions for OpenClaw agents
├── README.md             # This file
├── EXAMPLES.md           # Worked examples with real commands and output
├── scripts/
│   ├── setup.sh          # One-shot setup and dependency installer
│   └── game-loop.sh      # Automated screenshot → analyze → act loop
└── references/
    ├── android.md        # Full ADB command reference
    └── linux-handheld.md # Gamescope + InputPlumber command reference

Documentation

• SKILL.md — Complete agent instructions (the skill file OpenClaw reads)
• EXAMPLES.md — Real worked examples showing full sessions
• references/android.md — ADB command cheat sheet
• references/linux-handheld.md — Gamescope/InputPlumber reference

License

MIT