NoUI

Skip the UI. Turn any website into fast, reliable APIs for agents.
Go beyond Claw. Call the underlying APIs.
Skip Computer-use Agents.

---

🚀 What is NoUI?

NoUI turns any website into an API your agents can call.

Instead of automating clicks and scraping the UI, NoUI:

1. Records how you use a website (through a Tabby browser session)
2. Extracts the underlying APIs
3. Converts them into callable Python functions
4. Exposes them via MCP or as an installable Skill for agents

No clicks. No DOM parsing. No brittle automation.

NoUI ships as a single, agent-agnostic skill bundle (skills/noui/). The only external dependency is a reachable Tabby — there is no separate backend service to run and no LLM API key required.

---

⚡ Why NoUI?

Computer-use agents simulate humans:

• 🐢 Slow (UI loops, page loads)
• 💸 Expensive (token-heavy, step-heavy)
• 🧱 Fragile (break on UI changes)

NoUI executes software directly:

• ⚡ Fast — direct API calls
• 💸 Cheap — fewer steps, fewer tokens
• 🎯 Reliable — uses the same APIs the app uses internally

Stop automating clicks. Execute software.

---

🧠 How it works

1. Record a session in a Tabby browser — manually in a VNC viewer, or with Autopilot (the agent drives POST /execute/browser)
2. Tabby captures HAR traces + interaction events server-side
3. Compile the capture into Python API functions (a FastMCP server and/or a Skill)
4. Execute tools from inside the authenticated Tabby browser session
5. Install the Skill / run the MCP server in any agent
6. Agents call APIs instead of clicking UI

Execution

Generated tools run from inside the authenticated Tabby browser, not from a Python HTTP client. By default (the tabby execution mode) each operation calls Tabby’s POST /execute/fetch endpoint over plain HTTP; the Tabby worker runs fetch(url, {credentials: 'include'}) inside the real authenticated browser, so the browser’s TLS fingerprint and cookies are used — no credential extraction, and no Akamai/Cloudflare false positives. There is no WebSocket or CDP-port access from the NoUI side.

The legacy Python-side path (httpx + resolved credentials) is still available for server-to-server APIs that aren’t reachable from the browser origin; opt in explicitly when compiling with --execution-mode http.

---

🏗️ Architecture

Tabby browser session
    → VNC (human drives) | Autopilot (agent drives POST /execute/browser)
    ↓ (HAR + click/url events — captured server-side by Tabby)
noui_core  (the skill bundle, skills/noui/)
    → capture/   — drain/synthesize the recording bundle
    → compile/   — login → Tabby App Template; workflow → FastMCP server / Skill
    → activate/  — register App Templates, verify auth, install into agents
    ↓
Output (all under skills/noui/workbench/)
    → bundles/                       — saved capture bundles (source for re-generalization)
    → mcp_servers/// — runnable FastMCP packages
    → skills//                  — installable agent Skills
    ↓
Tabby Runtime  (persistent browser sessions + live auth, POST /execute/fetch)
    ↓
MCP / Skill → Claude / Codex / Agents

---

🔑 Core Components

🧩 HAR → API Compiler

• Parses browser network traffic (HAR), deterministically — no LLM calls
• Groups requests into logical workflows
• Generates clean Python functions (MCP server and/or Skill)

🐾 Tabby Runtime

• Keeps browser sessions alive in the cloud
• Handles cookies, headers, auth
• Streams VNC for login / 2FA when needed; runs POST /execute/* for tools

🔌 MCP Server / Skill

• Exposes generated APIs as tools
• Works with Claude, Codex, and MCP-/skill-compatible agents

---

🎯 What you can do

• Automate websites with no public APIs
• Turn internal tools into agent-ready SDKs
• Replace brittle browser automation workflows
• Build production-grade agents that actually scale

---

⚡ Example

def create_invoice(customer_id, amount):
    return call_api(
        method="POST",
        endpoint="/api/invoices",
        headers=session_headers,
        json={"customer_id": customer_id, "amount": amount},
    )

Then your agent simply says:

“Create an invoice for customer X”

NoUI executes it directly.

---

🔐 Authentication Flow

1. NoUI provisions a remote Tabby browser session
2. Streams it via VNC
3. You complete login / 2FA once, in the VNC viewer
4. Tabby maintains the session

No username/password is ever stored (credential_ref: manual:). When the browser reaches the post-login URL, Tabby auto-resolves the login — no repeated “Mark as Resolved” clicks. Agents reuse the authenticated context automatically through POST /execute/fetch.

---

⚔️ NoUI vs Computer-Use Agents

---

🧠 Philosophy

Websites already expose APIs. The UI is just a layer on top.

NoUI removes that layer.

---

🚀 Quick Start

Prerequisites

• Python 3.11+
• A reachable Tabby (cloud, or self-host via Docker + the tabby/ submodule)
• Tabby credentials in one tenant: TABBY_CLIENT_ID / TABBY_CLIENT_SECRET (agent), and TABBY_ADMIN_TOKEN (to register App Templates; Editor role suffices)

Clone with submodules

⚠️ NoUI includes Tabby as a git submodule (for local dev/test and to version-lock the endpoint contract). Clone recursively, or the tabby/ directory will be empty.

git clone --recursive https://github.com/adoptai/noui.git

# Or, if you already cloned without --recursive:
git submodule update --init

Install

cd noui/skills/noui

# 1. Create a venv and install the bundle (noui_core + deps: httpx, python-dotenv, mcp)
python -m venv .venv && . .venv/bin/activate
pip install -e .

# 2. Configure environment
cp .env.example .env
# Edit .env: TABBY_API_URL, TABBY_CLIENT_ID / TABBY_CLIENT_SECRET,
#            and TABBY_ADMIN_TOKEN (for register). All in the SAME tenant.

Scripts are run from skills/noui/ as python scripts/.py ….

---

🤖 Agent Skills

Install the NoUI skill — one self-contained bundle covering the full record → compile → activate pipeline — into your agent (Claude Code, Codex, OpenCode…):

npx skills add https://github.com/adoptai/noui --skill noui

Then invoke /noui in your agent. The bundle’s SKILL.md documents the three pillars and the scripts; references/ holds the deep dives.

Human users who prefer an interactive picker can run npx skills add https://github.com/adoptai/noui (no --skill flag) and tick what they want. AI agents must use the per-skill --skill form — the interactive selector blocks on stdin.

Available skills

Install demo plugins from a clone:

claude plugin install ./skills/travel
claude plugin install ./skills/quickbooks

---

🛠️ Developer Flow

All commands run from skills/noui/.

Unauthenticated apps

# 1. Record a workflow in a Tabby VNC session. --mode workflow is EXPLICIT here: with no
#    login to capture, the combined default would ask you to sign in to nothing.
python scripts/capture_record.py --mode workflow --url "https://example.com"
#    open the printed VNC URL, perform the workflow, click "Finish & export"

# 2. Drain + compile to a FastMCP server (and/or Skill)
python scripts/capture_import.py  --as mcp

Authenticated apps (with Tabby) — the default flow

One recording session captures the login and the workflow, so the human gets a single link and signs in once. NoUI splits the capture at the login boundary.

# 1. Record: sign in AND drive the workflow in ONE VNC session, one "Finish & export".
#    No --mode needed (it defaults to combined); --name also skips the capture entirely
#    if an App Template already covers this app.
python scripts/capture_record.py --url "https://app.example.com/login" --name example

# 2. Drain + compile: registers a tenant-wide login App Template (per-user profiles
#    auto-provision → ACTIVE, no stored credentials) AND compiles the workflow bound
#    to it. No --combined flag: the provision ledger already recorded the mode.
python scripts/capture_import.py  --as skill --name example \
  --post-login-url-pattern ""

# 3. Install the generated Skill into your agent
python scripts/activate_install.py workbench/skills/ claude-code

The first live call against the new profile returns login_required: its per-user session starts LOGIN_NEEDED (nothing is stored, and the recording’s cookies are deliberately not reused for a tenant-wide template). The caller escalates — the Agent Harness shows a Connect card; locally, read vnc_stream.url from GET /agent/session-status/ and hand it to the human. One sign-in, then every later call uses that session. There is nothing to run in advance.

Workflow only — the app already has a profile

# --profile/--from means the auth exists, so the mode defaults to workflow-only.
python scripts/capture_record.py --url "https://app.example.com/app" --profile example
python scripts/capture_import.py  --as both --profile-slug example
# … or let the agent drive it headlessly instead of a human:
python scripts/capture_autopilot.py  --steps steps.json --as both

Static API-key apps (no login to record)

# The app authenticates with a static key on every request — record only the
# workflow and declare the auth model. NoUI emits a ${SECRET:...} placeholder;
# an admin registers the value in the harness secret store (never held by NoUI).
python scripts/capture_record.py --mode workflow --url "https://example.com"
python scripts/capture_import.py  --as skill --execution-mode harness \
  --auth-type api-key --api-key-header Authorization

--auth-type declares the app’s auth model (session default | api-key | auto). Distinct from --auth-mode (agent_token/platform_jwt), which is the Tabby token mode.

---

📦 Generated MCP Output

skills/noui/workbench/
  bundles/.json                 # saved capture bundle (re-generalization source)
  mcp_servers///
    server.py                         # FastMCP entrypoint
    tools.json                        # Tool inventory
    manifest.json                     # Server manifest (lifecycle + auth metadata)
    noui_runtime/
      auth.py                         # Runtime auth adapter (fetches live creds from Tabby)
      execute.py                      # Calls Tabby /execute/fetch from inside the browser
    operations/
      .py                  # One file per generated tool
  skills//                  # Installable agent Skill (SKILL.md + operations/)

---

🖥️ Scripts Reference

The bundle replaces the old CLI with thin, purpose-built scripts (run from skills/noui/):

scripts/capture_record.py    --url  [--mode login|workflow|combined] [--from ]
                             [--profile ] [--auth-type {session|api-key}] [--api-key-header ]
                             # --mode defaults to `combined` (one link, one sign-in), or
                             # `workflow` when --profile/--from supplies the auth
scripts/bundle_inspect.py     | --session  [--json]
scripts/capture_autopilot.py  --steps steps.json --as {mcp|skill|both}
scripts/capture_import.py     [--name ] [--as {mcp|skill|both}] [--execution-mode {tabby|http|harness}]
                             [--combined] [--auth-type {session|api-key|auto}] [--api-key-header ]
                             [--profile-slug ] [--credential-mode {takeover|manual|stored|auto}]
                             [--auth-mode {agent_token|platform_jwt}] [--tenant-id ] [--post-login-url-pattern ]

scripts/compile_workflow.py   --as {mcp|skill|both}
scripts/compile_login.py      --out 

scripts/activate_register.py  [--tenant-id ]
scripts/activate_verify.py   
scripts/activate_install.py       # agent: claude-code|codex|cline|opencode|agents [--project]

---

🔌 Tabby endpoints used

NoUI talks only to Tabby (no NoUI backend service):

---

🔗 Tabby compatibility

NoUI depends on Tabby for authenticated browser sessions. To keep behavior deterministic across releases, NoUI pins a specific SHA on Tabby’s tabby-noui branch via a git submodule.

• Current pin: tabby-noui @ d3c7373
• Branch: tabby-noui
• Submodule path: tabby/ (inside this repository)

Using a different Tabby revision is unsupported. The submodule is for local dev/test; at runtime NoUI talks to whatever Tabby TABBY_API_URL points at. See CONTRIBUTING.md for the submodule-bump workflow.

---

🧩 Capture (no Chrome extension)

Capture happens server-side inside Tabby — there is no Chrome extension to install. Two modes:

• VNC — capture_record.py provisions a recording session; you drive a real browser in the VNC viewer and click “Finish & export”.
• Autopilot — capture_autopilot.py / AutopilotSession drives the browser through POST /execute/browser; NoUI synthesizes the bundle from the inline HAR plus the commands it issued.

Both produce the same {har, click_events, url_events} bundle that the compiler consumes.

---

📚 Project docs

• docs/HANDOFF.md — install & use the skill with Tabby (start here)
• CONTRIBUTING.md — dev setup, PR process, bumping the Tabby submodule
• SECURITY.md — responsible disclosure
• CHANGELOG.md — release history
• CODE_OF_CONDUCT.md

---

🔥 Status

Early open source — expect rough edges. Contributions welcome.

---

📢 Closing

Computer-use agents were step one.

NoUI is what comes next.