---

☕ Sponsor this project

🌍 International (card):

(yes — I moved this up here on purpose. Was afraid nobody scrolls past the badges 😅)

---

FLOW KIT

Standalone system to generate AI videos via Google Flow API. Uses a Chrome extension as browser bridge for authentication, reCAPTCHA solving, and API proxying.

Showcase

All outputs below were generated end-to-end by this system — from story concept to final YouTube-ready video with thumbnails, narration, and branding.

Generated YouTube Thumbnails

Visual Consistency Across Scenes

The reference image system keeps characters consistent across an entire video. Each character is generated once as a reference, then the AI uses that reference in every scene — maintaining the same face, clothing, and features.

Doctor character — same face, glasses, white coat across 4 different scenes:

Defector character — same face across ICU, hospital, interview, and Seoul streets:

All frames from a single 50-scene project. Both characters maintain consistent appearance across completely different settings and lighting conditions — powered by the reference image system.

F-15E Rescue — Full Story Arc (25 scenes)

Strategic briefing → pilot departure → formation flight → aircraft hit → CSAR alert → pilot survival.

Hormuz Strait — Naval Scenes

What the Pipeline Produces

Each project goes through: story → entities → reference images → scene images → 8s video clips → narration (TTS) → concat → thumbnails → YouTube upload — all orchestrated via API or AI agent skills.

---

Chrome Extension — Live Dashboard

The Chrome extension runs alongside Google Flow — showing real-time request log (614 total, 328 success), video generation progress, and token status. The Python agent communicates with the extension via WebSocket to automate all API calls.

---

Web Dashboard — Ops Console

A local React dashboard (dashboard/) for monitoring and driving the pipeline — real-time KPIs, per-video stage progress, a scene-level pipeline view with AI review, and a setup guide, all backed by the same FastAPI agent. Supports English, Vietnamese, Hindi, Indonesian, Chinese, Korean, and Japanese.

Architecture

┌──────────────────┐     WebSocket      ┌──────────────────────┐
│  Python Agent    │◄──────────────────►│  Chrome Extension     │
│  (FastAPI+SQLite)│     localhost:9222  │  (MV3 Service Worker) │
│                  │                    │                       │
│  - REST API :8100│  ── commands ──►   │  - Token capture      │
│  - Queue worker  │  ◄── results ──    │  - reCAPTCHA solve    │
│  - Post-process  │                    │  - API proxy          │
│  - SQLite DB     │                    │  (on labs.google)     │
└──────────────────┘                    └──────────────────────┘

Quick Start

One-command setup

./setup.sh

This checks and installs: Python 3.10+, pip, ffmpeg, ffprobe, Chrome, creates venv, installs dependencies, verifies imports.

Windows: Use WSL (wsl --install) or Git Bash. All bash scripts and commands assume a Unix shell.

Manual setup

# Prerequisites: Python 3.10+, ffmpeg, Chrome
pip install -r requirements.txt

Run

# 1. Load Chrome extension: chrome://extensions → Developer mode → Load unpacked → extension/
# 2. Open https://labs.google/fx/tools/flow and sign in
# 3. Start agent
source venv/bin/activate   # if using setup.sh
python -m agent.main

# 4. Verify
curl http://127.0.0.1:8100/health
# {"status":"ok","extension_connected":true}

End-to-End Example: “Pippip the Fish Merchant”

A chubby cat sells fish at a market. 3 scenes, vertical, Pixar 3D style.

How it works (read this first)

The system uses reference images to keep visuals consistent across scenes. Here’s the mental model:

1. Identify every visual element that should look the same across scenes:

• Characters → entity_type: "character" (portrait reference)
• Places → entity_type: "location" (landscape reference)
• Important objects → entity_type: "visual_asset" (detail reference)

2. Describe ONLY appearance in the entity description — this generates the reference image:

• "Chubby orange tabby cat with blue apron, straw hat" (what it looks like)

3. Write scene prompts as ACTION — reference entities by name, describe what they DO:

• "Pippip stands behind Fish Stall, arranging fish..." (what happens)
• NOT: "A chubby orange tabby cat wearing a blue apron stands behind a wooden stall..." (don’t repeat appearance)

4. List all entities that appear in each scene’s character_names array — their reference images get passed to the AI as visual input, ensuring consistency.

Story idea
    ↓
Break into visual elements → characters[] array with entity_type + description
    ↓
Write scene prompts using entity NAMES → character_names lists which refs to use
    ↓
System generates ref image per entity → then composes scenes using those refs

Using Skills (recommended)

Skills handle all the API calls, polling, and verification automatically. Use with Claude Code (/fk-command) or follow the recipe in skills/*.md for any AI agent.

/fk-create-project             ← interactive: asks story, creates entities + scenes
/fk-gen-refs       ← generates all reference images, verifies UUIDs
/fk-gen-images       ← generates scene images with all refs applied
/fk-gen-videos       ← generates videos (2-5 min each, polls automatically)
/fk-concat                ← downloads + merges into final video
/fk-status                ← dashboard: what's done, what's next

Full pipeline in 5 commands. Each skill pre-checks dependencies (e.g. /fk-gen-images verifies all refs exist first).

Manual API (step by step)

---

Core Concepts

Reference Image System

Every visual element that should stay consistent gets a reference image — characters, locations, props. Each reference has a UUID media_id used in all scene generations via imageInputs.

Scene Prompts = Action Only

Scene prompts describe what happens, not character appearance. The reference images maintain visual consistency.

DO:   "Pippip juggling fish at Fish Stall, crowd watching in Open Market"
DON'T: "Pippip the chubby orange tabby cat wearing a blue apron juggling..."

Media ID = UUID

All media_id values are UUID format (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx). Never the base64 CAMS... mediaGenerationId.

Two Prompts per Scene

Each scene has two separate prompts:

• prompt — describes the still image (frame 0): "Luna steps out of rocket onto candy planet. Wide shot, sunrise."
• video_prompt — describes the 8s video motion with sub-clip timing and camera directions:

0-3s: Wide crane down, Luna steps out of rocket onto Candy Planet Surface. Luna gasps "It's beautiful!"
3-6s: Low angle tracking shot, Luna walks across candy ground, shallow DOF. Luna says "Everything is made of candy."
6-8s: Close-up Luna's face, eyes wide with wonder, golden hour backlight. Silence, ambient wind.

Character Voice

Characters can have a voice_description (max ~30 words) for voice consistency:

{"name": "Luna", "entity_type": "character", "description": "Small white cat...", "voice_description": "Soft curious childlike voice with wonder and slight purring"}

Voice descriptions are auto-appended to video prompts before generation.

No Background Music

The worker auto-appends "No background music. Keep only natural sound effects and ambient sounds." to all video prompts. Sound effects from the scene (footsteps, splashing, wind) are preserved.

Pipeline Overview

1. Create project      POST /api/projects (with entities + story)
2. Create video        POST /api/videos
3. Create scenes       POST /api/scenes (chain_type: ROOT → CONTINUATION)
4. Gen ref images      POST /api/requests {type: GENERATE_CHARACTER_IMAGE} per entity
   → Wait ALL complete, verify all have UUID media_id
5. Gen scene images    POST /api/requests {type: GENERATE_IMAGE} per scene
   → Wait ALL complete
6. Gen videos          POST /api/requests {type: GENERATE_VIDEO} per scene
   → Wait ALL complete (2-5 min each)
7. (Optional) Upscale  POST /api/requests {type: UPSCALE_VIDEO} (TIER_TWO only)
8. Download + concat   ffmpeg normalize + concat

Skills (AI Agent Workflows)

Ready-to-use workflow recipes in skills/ (also available as /slash-commands in Claude Code):

Basic Pipeline

Advanced Video

Review & Quality

Reference

TTS & Narration

YouTube

Utilities

AI CLI Compatibility (Skill Consumption)

Skills are .md recipes any AI coding-assistant CLI can read and follow — this is about which agent reads the skill files, not which model does the work:

AI Vision Providers (Video Review)

Separate from the table above — this is about which CLI backend does the vision analysis for /fk-review-video. Three providers are supported and swappable at runtime, no restart required:

# View provider status (installed / version-tested / currently active)
curl -s "http://127.0.0.1:8100/api/providers?live=true" | python3 -m json.tool

# Switch provider — hot-reloaded immediately, no server restart
curl -X PATCH http://127.0.0.1:8100/api/providers \
  -H "Content-Type: application/json" -d '{"active": "agy"}'

Or just run /fk-change-provider for an interactive picker. Full details in skills/fk-change-provider.md.

Video Generation Techniques

API Reference

CRUD Endpoints

Special Endpoints

Request Types

Worker Behavior

• Server handles throttling — worker enforces max 5 concurrent + 10s cooldown automatically. Use POST /api/requests/batch to submit all at once; do NOT manually batch.
• 10s cooldown between API calls (anti-spam, configurable via API_COOLDOWN)
• Reference blocking — scene image gen refuses if any referenced entity is missing media_id
• Skip completed — won’t re-generate already-completed assets
• Cascade clear — regenerating image auto-resets downstream video + upscale
• Retry — failed requests retry up to 5 times
• UUID enforcement — extracts UUID from fifeUrl if response doesn’t provide it directly
• Voice context — auto-appends character voice_description to video prompts
• No background music — auto-appends “no background music, keep sound effects” to all video prompts
• Dual video response schema — Lite/Fast/Ultra models return operations[] and stream URLs; Low Priority models (veo_3_1_*_low_priority, *_ultra_relaxed) return workflows + media with the MP4 inline as base64. The SDK auto-detects, validates the ftyp magic, and saves the binary to output/_workflow_videos/{media_id}.mp4. The scene’s _video_url is then a file:// path which curl and ffmpeg handle natively. _video_media_id always stores the real Flow media UUID, so upscale works for both schemas.

Default Model & Tier Compatibility

The default for PAYGATE_TIER_TWO frame_2_video and start_end_frame_2_video is veo_3_1_i2v_lite_low_priority — the TRUE 0-credit Low Priority that works on every service tier including SERVICE_TIER_ADVANCED.

The *_ultra_relaxed family (Low Priority ultra-quality) silently returns empty operations on SERVICE_TIER_ADVANCED accounts because Google requires SERVICE_TIER_ULTRA for that path. ULTRA-tier users can switch back via /fk-change-model — see skills/fk-change-model.md for the full preset list and tier compatibility matrix.

Material System

Every project must have a material field that controls the visual style of generated images. Set it at project creation.

# List available materials
curl -s http://127.0.0.1:8100/api/materials

# Set on project
curl -X POST http://127.0.0.1:8100/api/projects \
  -d '{"name": "...", "material": "3d_pixar", ...}'

Materials control both entity image_prompt style and scene scene_prefix. Examples: realistic, 3d_pixar, anime, stop_motion, minecraft, oil_painting.

Configuration

Architecture

agent/
├── main.py              # FastAPI app + WebSocket server
├── config.py            # Configuration (loads models.json, providers.json)
├── models.json          # Video/upscale/image model mappings
├── providers.json        # Active AI CLI provider for video review (claude/agy/codex)
├── db/
│   ├── schema.py        # SQLite schema (aiosqlite)
│   └── crud.py          # Async CRUD with column whitelisting
├── models/              # Pydantic models + Literal enums
├── api/                 # REST routes (projects, videos, scenes, characters, requests,
│                         #   flow, models, providers, reviews, materials, music, tts)
├── services/
│   ├── flow_client.py   # WS bridge to extension
│   ├── headers.py       # Randomized browser headers
│   ├── tts.py           # OmniVoice TTS (subprocess-based)
│   ├── scene_chain.py   # Continuation scene logic
│   ├── video_reviewer.py # AI vision review — contact sheet + claude/agy/codex CLI dispatch
│   └── post_process.py  # ffmpeg trim/merge/music
└── worker/
    └── processor.py     # Queue processor + poller

extension/               # Chrome MV3 extension
skills/                  # AI agent workflow recipes (CLI-agnostic)
youtube/
├── auth.py              # OAuth2 multi-channel auth
├── upload.py            # Upload with scheduling + rule validation
└── channels/            # Per-channel config (gitignored)
    └── /
        ├── client_secrets.json  # OAuth2 credentials
        ├── token.json           # Auth token (auto-created)
        ├── channel_rules.json   # Upload rules + SEO defaults
        └── upload_history.json  # Upload log
CLAUDE.md                # AI agent instructions (Claude Code)
AGENTS.md                # AI agent instructions (Codex CLI)
GEMINI.md                # AI agent instructions (Gemini CLI)

TTS Narration (OmniVoice)

Optional narrator voice for scenes. Uses OmniVoice — multilingual zero-shot TTS with voice cloning (600+ languages).

Setup

See skills/fk-gen-tts-template.md for full install guide. Quick version:

pip install torch==2.8.0 torchaudio==2.8.0   # or +cu128 for NVIDIA
pip install omnivoice
python3 -c "from omnivoice import OmniVoice; print('OK')"

If OmniVoice is in a separate venv, point to it:

export TTS_PYTHON_BIN=/path/to/omnivoice-venv/bin/python3

Workflow

1. Create voice template — /fk-gen-tts-template — generates an anchor voice WAV
2. Add narrator text to scenes — PATCH /api/scenes/{id} with narrator_text
3. Generate narration — /fk-gen-narrator — voice-clones the template for each scene
4. Concat with narration — /fk-concat-fit-narrator — trims scene videos to match TTS duration

CPU-only recommended (MPS produces artifacts). ~15-30s per scene.

YouTube Upload Pipeline

Automated upload with per-channel rules, SEO optimization, and brand watermarking.

Setup

# 1. Place OAuth credentials
cp client_secrets.json youtube/channels//

# 2. Authenticate (opens browser)
python3 youtube/auth.py               # Linux / Windows (WSL)
arch -arm64 python3 youtube/auth.py   # macOS Apple Silicon

# 3. Token saved to youtube/channels//token.json (auto-refreshes)

Channel Rules (channel_rules.json)

Each channel has a rules file controlling upload scheduling and SEO:

{
  "shorts": {"max_per_day": 3, "optimal_times": ["07:00", "12:00", "17:00"]},
  "long_form": {"max_per_day": 1, "optimal_times": ["19:00"]},
  "scheduling": {"min_gap_hours": 4, "avoid_hours": [0,1,2,3,4,5]},
  "seo": {"niche": "...", "default_tags": [...], "title_max_chars": 65}
}

Skill Chain

/fk-youtube-seo    → generates title, description, hashtags, tags
/fk-brand-logo     → applies channel icon watermark
/fk-youtube-upload  → validates rules + uploads (auto-detects Short vs Long-form)

Upload validation checks: max per day, min gap between uploads, avoid dead hours. Auto-detects Short (<61s + vertical 9:16) vs Long-form.

Error Handling

Errors can originate from four layers — Google Flow backend, Chrome extension, FastAPI layer, and the worker itself. The worker’s _handle_failure (agent/worker/processor.py:414-481) routes recovery by error_message string content, not HTTP status, because Flow lumps many distinct failures under HTTP 400 with varying details.reason values.

Flow-Native Structured Errors

These arrive in the response body as data.error.details[].reason. The worker appends the reason to error_message as " []".

HTTP Status Codes

Status-code detection logic lives in agent/worker/_parsing.py:_is_error — a result is an error if result.error is set, status >= 400, or data.error is present.

Extension / Transport Errors

String patterns in error_message that the worker recognizes:

Worker Retry Policy

processor.py:_handle_failure decides terminal vs retryable:

1. Auto-recover if message contains "not found" → re-upload media, mark PENDING.
2. Transient WS (reconnected/disconnected/switched) → re-queue PENDING, keep retry_count.
3. CAPTCHA → retry up to 10× without counting toward MAX_RETRIES.
4. Default → increment retry_count; if < MAX_RETRIES (5), schedule retry with 2^retry * 10s backoff (capped 300s). Otherwise mark FAILED.

YouTube Upload Errors

From youtube/upload.py (HTTP errors from YouTube Data API v3):

Common Symptoms → Fix

License

MIT

---

Community & Support

Share anything crazy and useful created with Vibe Code. Drop in to:

• Post the story-video runs and thumbnails you’ve generated
• Share scene templates, prompt recipes, and reference-image setups
• Ask for help when an output isn’t matching what you imagined
• Request features and report bugs you’ve hit in the wild
• Trade tips on Google Flow plan limits, Veo i2v behaviour, and Chrome extension setup
• Facebook Post via Extension MCP
• Right way to build Mobile Application + System

→ facebook.com/groups/vibecodeera