ClopinetteAI

Cloudflare-native AI agent with persistent memory, a built-in skills hub, multimodal I/O, and Telegram / WhatsApp / Discord bots out of the box. One Durable Object per user, serverless, BYOK-ready.

Clopinette = French for “next to nothing”. An AI agent that costs clopinettes to run.

Inspired by hermes-agent (Nous Research). Clean-room TypeScript rewrite for Cloudflare.

How it works

        Web / Telegram / WhatsApp / Discord / Evolution / MCP
                              │
                              ▼
                   Hono worker  (clopinette-ai)
                              │
                              ▼
              ClopinetteAgent Durable Object
                (one per user, SQLite + R2)
                              │
                              ▼
         runPipeline() ──► LLM  +  tool layer  +  5 memory layers

Every user gets an isolated Durable Object with its own SQLite, its own MEMORY.md / USER.md, and its own skills. The agent streams over WebSocket on the web UI and replies to Telegram / WhatsApp / Discord via webhooks. Telegram, WhatsApp, and Discord can be linked to a web account with /link so the bot shares memory across channels.

Supporting Cloudflare services (used transparently):

• Queues — usage reports, retry-safe with DLQ
• Workflows — async delegation (sub-agents) and one-shot Vectorize backfill
• Vectorize — hybrid keyword + semantic search on chat history (bge-m3, RRF fusion)
• Cron — the gateway syncs D1 → KV quota every 5 min

Features

• Chat — streaming WebSocket with Markdown, images, voice, drag-and-drop upload
• Tool-driven runtime — web, docs, memory, history, skills, notes, calendar, todo, image, tts, clarify, with optional browser, delegation, and codemode
• 5-layer memory — MEMORY.md / USER.md, hybrid FTS5+Vectorize session search, memory flush, skills index, optional Honcho
• Multi-platform — Telegram, WhatsApp (Meta Cloud API + self-hosted Evolution/Baileys), Discord (slash commands + DM bridge), MCP
• 14 personality presets — helpful, concise, technical, kawaii, pirate, shakespeare, noir, uwu, etc. (/personality)
• Skills Hub — install skills from the built-in catalog (10 curated skills) or trusted GitHub repos like Hermes, Cloudflare, MiniMax, and gstack. Installed skills are refreshed daily when their catalog or GitHub source changes
• Hub hardening — scans imported skills for prompt-injection / exfiltration structure before install, while preserving trusted security and red-team bundles
• BYOK — bring your own key across 13 providers (OpenAI, Anthropic, Google AI Studio, Groq, xAI/Grok, Mistral, DeepSeek, Cohere, Perplexity, OpenRouter, Cerebras, HuggingFace, Replicate) via the Cloudflare AI Gateway. On the hosted SaaS the BYOK plan is free — you only pay your provider
• BYOK media models — image generation, TTS, and voice transcription can run on the user’s own key via the provider’s OpenAI-compatible endpoints (/images/generations, /audio/speech, /audio/transcriptions), configured in Settings → AI Provider
• Cross-provider auxiliary — e.g. primary OpenAI + auxiliary Anthropic for the fast-path / compression
• Self-learning — the agent updates its own memory after N turns
• Async delegation with auto-resume — web-only research sub-agents run in parallel via Cloudflare Workflows. When the last delegate finishes, the agent automatically synthesizes their results and pushes the final reply via the originating gateway (web / Telegram / WhatsApp / Discord). No need to send a follow-up message.
• Multimodal — vision (images), Whisper (voice), PDF / DOCX / XLSX / text extraction

Tools

Primary tools available to the agent:

With codemode active (when the LOADER binding is set), the LLM writes JavaScript that orchestrates internal state tools in one step instead of spending tokens on repetitive tool chatter.

Skills Hub

• Browse installable skills from the built-in catalog or trusted GitHub repos (Hermes, Cloudflare, MiniMax, gstack).
• Built-in catalog: web-research, daily-briefing, document-qa, meeting-notes, code-review, git-workflow, api-design, debug-strategy, research-summary, writing-editor. It ships inside the worker, so a deploy updates the hub for every user.
• Imported skills can carry support files such as references/, templates/, and scripts/ when they are text-based.
• Hub installs are scanned for structural prompt-injection, secret-exfiltration, and dangerous URI patterns before they are written to user storage.
• Daily auto-update: each agent re-checks its catalog and GitHub installs by content hash and re-installs changed ones (URL installs are pinned at install time).

Browser Run observability

• Every successful browser action returns browserRun.sessionId (the Browser Run session behind this user’s browser, one per user DO).
• navigate, live_view, diagnostics and request_human also return browserRun.liveViewUrl (CDP Cloudflare.getLiveView, tab mode, 15 min validity) so a human can watch or take over for login, MFA, CAPTCHA, or sensitive data entry, then tell the agent to continue.
• browser({ action: "diagnostics" }) returns operator guidance; browser({ action: "request_human", reason: "..." }) produces a structured handoff with the Live View link. Fallback without a link:

wrangler browser list
wrangler browser view 

• web responses backed by Browser Run quick actions include browserRun.sessionId, browserRun.browserMsUsed, and browserRun.cfRay when Cloudflare returns them.
• The core worker talks to the PlaywrightMCP Durable Object over DO RPC (callTool, getBrowserRunInfo); external MCP clients use /mcp (streamable HTTP, API_AUTH_KEY bearer) served through the Agents SDK handler.
• Browser Run Session Recordings require launching a browser session with recording: true. The Playwright MCP context factory does not expose a recording toggle yet.

Slash commands

Work on every platform (web, Telegram, WhatsApp, Discord).

Inference

Installation

Prerequisites

• Bun v1.1+
• Wrangler v4+
• Cloudflare account (Workers Paid plan)

Deploy

git clone https://github.com/marceloeatworld/clopinette-ai.git
cd clopinette-ai
cp wrangler.example.jsonc wrangler.jsonc
bun install

# Storage
wrangler r2 bucket create clopinette-memories
wrangler r2 bucket create clopinette-skills
wrangler kv namespace create LINKS

# Queues (retry-safe usage reports)
wrangler queues create clopinette-usage
wrangler queues create clopinette-usage-dlq

# Vectorize (hybrid semantic search, 1024-d for bge-m3)
wrangler vectorize create clopinette-sessions --dimensions=1024 --metric=cosine
wrangler vectorize create-metadata-index clopinette-sessions --property-name=userId --type=string
wrangler vectorize create-metadata-index clopinette-sessions --property-name=sessionId --type=string

# Required secrets
wrangler secret put MASTER_KEY           # openssl rand -base64 32
wrangler secret put CF_ACCOUNT_ID
wrangler secret put CF_GATEWAY_ID
wrangler secret put WS_SIGNING_SECRET    # openssl rand -base64 32 (shared with the gateway)
wrangler secret put API_AUTH_KEY

# Codemode outbound proxy (only if you keep LOADER + CODEMODE_OUTBOUND in wrangler.jsonc)
(cd workers/outbound && wrangler deploy)

# Deploy — Workflows (delegation, vector backfill) are auto-created
bun run deploy

Telegram

wrangler secret put TELEGRAM_BOT_TOKEN
bun run deploy
curl -X POST https://your-worker.workers.dev/api/admin/setup-telegram \
  -H "Authorization: Bearer YOUR_API_AUTH_KEY"

WhatsApp (Meta Cloud API)

wrangler secret put WHATSAPP_ACCESS_TOKEN
wrangler secret put WHATSAPP_VERIFY_TOKEN
wrangler secret put WHATSAPP_APP_SECRET
wrangler secret put WHATSAPP_PHONE_NUMBER_ID
# Set webhook URL in Meta portal: https://your-worker.workers.dev/webhook/whatsapp

Discord

Two channels: Interactions (slash commands, zero setup) and Gateway bridge (DM / @mentions, needs a small external Docker service).

wrangler secret put DISCORD_PUBLIC_KEY
wrangler secret put DISCORD_APPLICATION_ID
wrangler secret put DISCORD_TOKEN
bun run deploy

# Register slash commands + generate bridge secret
curl -X POST https://your-worker.workers.dev/api/admin/setup-discord \
  -H "Authorization: Bearer YOUR_API_AUTH_KEY"
# Returns: { interactionsUrl, bridgeUrl, bridgeSecret }

# For natural DM / @mention chat (optional):
cd workers/discord-bridge
# Set env: DISCORD_TOKEN, BRIDGE_SECRET, WORKER_URL
docker compose up -d

Slash commands (/ask, /status, /memory, etc.) work immediately. The bridge is only needed if you want full conversation in DMs and @mentions.

Optional secrets

# Web search backends (automatic fallback: SearXNG → Brave → Browser Run)
wrangler secret put SEARXNG_URL          # self-hosted SearXNG URL
wrangler secret put BRAVE_API_KEY        # free tier: 2000 queries/month
wrangler secret put CF_BROWSER_TOKEN     # last resort + read/extract/crawl

# Optional GitHub token for Hub indexing / install rate limits
wrangler secret put GITHUB_TOKEN

# Evolution API (self-hosted WhatsApp via Baileys)
wrangler secret put EVOLUTION_API_URL
wrangler secret put EVOLUTION_API_KEY

# Gateway integration (SaaS quota cache refresh; see clopinette-platform)
wrangler secret put GATEWAY_URL           # https://api.example.com
wrangler secret put GATEWAY_INTERNAL_KEY  # optional, falls back to WS_SIGNING_SECRET

# Discord behaviour (plain vars, not secrets)
# DISCORD_REQUIRE_MENTION=false          answer every guild message (default: mention only)
# DISCORD_FREE_RESPONSE_CHANNELS=id,id   channels that never require a mention

Slack: SLACK_SIGNING_SECRET enables signature verification on /webhook/slack, but the adapter is a stub (URL verification only).

Project layout

src/
  index.ts             Hono router — webhooks + MCP + platform setup routes
  agent.ts             ClopinetteAgent DO — sessions, RPC methods, live state
  pipeline.ts          Inference pipeline
  prompt-builder.ts    Block-based system prompt (cached per session + platform)
  compression.ts       Structured context compression + memory flush
  commands.ts          Platform-agnostic slash commands
  delegate-workflow.ts Async sub-agent execution (Cloudflare Workflow)
  backfill-workflow.ts One-shot Vectorize hydration
  playwright-mcp.ts    Browser DO
  config/              Types, constants, personalities
  media/               Vision, Whisper, PDF/DOCX/XLSX
  memory/              Prompt memory, hybrid FTS5+Vectorize search, skills, self-learning
  tools/               Tool implementations + registry
  hub/                 Skills catalog + trusted GitHub / URL sources + install security
  inference/           Per-provider config + smart routing + error classification
  enterprise/          Auth, audit, budget
  gateway/             Telegram / WhatsApp / Evolution / Discord adapters + Slack stub

workers/outbound/       SSRF-safe fetch proxy for codemode sandbox
workers/discord-bridge/ Discord Gateway bridge (Bun + Docker)
test/                   Unit tests + live smoke test

Tests

bun run check    # TypeScript
bun run test     # Unit tests (vitest)
BASE_URL=https://your-worker.workers.dev API_KEY=xxx bun test/live/smoke.ts

Tech stack

Cloudflare Workers, Durable Objects, Workflows, Queues, Vectorize, Workers AI, AI Gateway, KV, R2, Browser Run, AutoRAG. TypeScript 7 + Hono + Agents SDK 0.27 + AI SDK v7.

License

MIT