让一个大型语言模型（LLM）观察视频而不破坏上下文窗口：ffmpeg 帧采样在选定分辨率下，按型号系列做标记估计，预算匹配，接触表。
概览
模型不能看视频。它可以看帧，每帧都消耗代币，无论是否重要。这是一项技能加上两个脚本，使帧选择变得有意识：用ffmpeg在你指定的分辨率下采样，在附加任何东西前估算代币成本，并通过帧数与分辨率进行交易来调整预算。 作为Claude Code技能、Codex技能，或者任何能运行脚本的代理的普通指令。需要在PATH上用ffmpeg和ffprobe，别无他用。 每个框架角落都刻有时间戳，并manifest.json列出尺寸和每个模型家族估计代币。接触纸将框架拼贴成一张图片，方便廉价的初次体验。 然后测量你即将安装的设备： 估算使用各供应商发布的尺寸规则：Anthropic的像素在缩放后÷750像素，OpenAI的512像素图块规则（每图块170像素+85像素）和32像素补丁规则（上限为1536个补丁）。它们都是估计值。measure_tokens。
概述（AI 翻译）
模型不能看视频。它可以看帧，每帧都消耗代币，无论是否重要。这是一项技能加上两个脚本，使帧选择变得有意识：用ffmpeg在你指定的分辨率下采样，在附加任何东西前估算代币成本，并通过帧数与分辨率进行交易来调整预算。
可作为Claude Code技能、Codex技能，或任何能运行脚本的代理提供普通指令。需要PATH上的ffmpeg和ffprobe，别无他物。
安装
使用
每个帧的时间戳都刻在角落，并在manifest.json中列出尺寸和每个模型族估计的代币数。接触板将帧拼贴成一张图，方便廉价的初次浏览。
然后测量你即将安装的设备：
旋钮
| 旗帜 | 它的作用 |
|---|---|
--mode interval --every S |
每秒一帧 |
--mode fps N |
每秒 N 帧 |
--mode scene --scene-threshold T |
画面变化的帧;较低的T显示更多切割 |
--mode keyframes |
编码器的I帧，快速粗略传递 |
--resolution low|medium|high|full |
长边512、768、1024或原生;绝不放大 |
本页由 AI 翻译，技术细节请以原文为准。
README
以下为原文 README 节选，完整内容请查看原文。
video-input-to-modern-llm
A model cannot watch a video. It can look at frames, and every frame costs tokens whether or not it mattered. This is a skill plus two scripts that make the choice of frames deliberate: sample with ffmpeg at a resolution you name, estimate the token cost before anything is attached, and fit a budget by trading frame count against resolution.
Works as a Claude Code skill, a Codex skill, or plain instructions for any agent that can run a script. Needs ffmpeg and ffprobe on PATH, nothing else.
Install
git clone
cp -r video-input-to-modern-llm ~/.claude/skills/video-input-to-modern-llm # or ~/.codex/skills/
Use
python3 scripts/sample_video.py demo.mp4 --out /tmp/frames \
--mode scene --resolution low --sheet 4 --token-budget 3000 --model-family claude
demo.mp4: 1920x1080 30.0 fps 00:04:12.40
mode=scene resolution=low -> 15 frames at 512x288 in /tmp/frames/
tokens (frames): claude~2955 openai-tile~3825 openai-patch~2160
tokens (sheets): 1 sheet(s) claude~1842 openai-tile~1445
manifest: /tmp/frames/manifest.json
Every frame has its timestamp burned into the corner and listed in manifest.json with its dimensions and estimated tokens per model family. Contact sheets tile the frames into one image for a cheap first look.
Then meter what you are about to attach:
python3 scripts/measure_tokens.py /tmp/frames/detail
python3 scripts/measure_tokens.py /tmp/frames/detail --exact # real count via Anthropic's endpoint
The knobs
| flag | what it does |
|---|---|
--mode interval --every S |
one frame every S seconds |
--mode fps N |
N frames per second |
--mode scene --scene-threshold T |
frames where the picture changes; lower T finds more cuts |
--mode keyframes |
the encoder’s I-frames, a fast rough pass |
--resolution low|medium|high|full |
long side 512, 768, 1024, or native; never upscales |
--width PX |
explicit long side |
--max-frames N |
cap any mode by thinning evenly |
--token-budget N --model-family claude|openai |
fit the budget: fewer frames first, then lower resolution, never silently over |
--sheet COLS |
build COLSxCOLS contact sheets, capped at 1568 px wide |
--start S --end S |
window the video |
--audio |
also write a mono 16 kHz WAV for transcription |
Token estimates
Estimates use each provider’s published sizing rules: Anthropic’s pixels ÷ 750 after its own downscale, OpenAI’s 512-px tile rule (170 per tile + 85) and 32-px patch rule (capped at 1536 patches). They are estimates. measure_tokens.py --exact asks Anthropic’s token-counting endpoint for the real number without creating a message. OpenAI has no counting endpoint for images, so those columns stay estimates, and some models apply a multiplier on top of the patch count.
The skill
推荐工具
换一个关键词，或者移除筛选条件。
安装
npx skillfish add dreamers-laboratory/video-input-to-modern-llm