Clacky 🧤

Clicky for Windows — open source. Talk to your PC — she sees your screen, points at things, and actually does them.

🎬 Demo & write-up: raynanwuyep.com/clacky · ⬇ Download for Windows

Clacky is a voice-first desktop companion for Windows. Hold a hotkey, talk, and she:

• sees your screen and answers questions about it,
• points — a little buddy flies to whatever you’re asking about (snaps to the real UI element, pixel-accurate),
• acts — opens apps, clicks, types, runs multi-step tasks, using Claude Computer Use,
• remembers you across sessions and learns routines you teach her by voice,
• tours an app — “explain my screen” gives you a spoken, pointing walkthrough.

Her brain is Claude (Sonnet 5 + Computer Use); voice via Deepgram streaming STT and free Edge TTS.

Where this fits

Clicky is Mac-only. Clacky brings the same idea to the majority of desktops that can’t run it — open and free.

⚠️ Early build, and honestly a bit rough. The core loop — talk → see → point → act — works and is genuinely fun. But speech recognition isn’t perfect, and the more advanced features (multi-step tasks, Gmail/Calendar, background research) are lightly tested. This is a “try it and tell me what breaks” release, not a finished product.

---

What works today

The voice companion — clacky run:

• Push-to-talk voice; an on-screen buddy that points at what you ask about.
• “What’s on my screen?” → a spoken answer, buddy points at the relevant thing.
• “Explain my screen” / “walk me through this” → a teaching tour that points out several things, one at a time.
• “Open Notepad and type hello”, “click the Save button”, “go to YouTube” → she acts on your machine.
• “Remember I prefer dark mode”, “save this as my morning routine” → cross-session memory + learned routines.
• “Check my email”, “what’s on my calendar” → opens your logged-in web apps (or an opt-in Google API).
• “Go research X and tell me later” → a background agent works while you keep talking — and leaves you real files, not just words (via an embedded hermes-agent harness, on your same Claude key). Background agents can also use any MCP server you connect — clacky connect notion or clacky connect composio is a browser approve, and Composio alone brings 1000+ apps — so tasks can end in your Notion, Sheets, or Slack, not just on disk. She opens what she delivers, and never claims a delivery she didn’t make.