stem-shorts

Vertical STEM explainer videos, built from a spec instead of generated animation code.

Eight seconds from an HTTP explainer built with this plugin: a sequence diagram, a section change, then a code block walked through line by line. Narration drives the timing.

▶ Watch a finished one: What is a convolutional neural network? — a full explainer published to YouTube Shorts, built with this pipeline.

Frames from the bundled K-Means example: clustering plots computed from an actual dataset, annotated callouts, a quantization comparison — not illustrations of those ideas.

The difference

Most LLM video tools ask the model to write animation code, then run it and hope it compiles into something that looks right. That is why the ecosystem around them is full of reviewer loops that read execution logs, and why there are public leaderboards scoring how often each model produces a working animation.

stem-shorts asks the model to fill in a spec that gets validated. The layout is a fixed contract, the renderers are deterministic Python, and the spec is checked before anything is drawn. The worst outcome is a chart that could be prettier. It is never a broken video.

Three things follow from that, which the code-generation approach struggles to give you:

• Vertical 1080×1920 for TikTok, Reels and Shorts — not a 16:9 lecture recording.
• Timing derived from the real audio. Each line’s narration is measured with ffprobe after it is generated, and the animation is built around those durations. Nothing is estimated.
• No LaTeX distribution. Formulas render through MathJax, so there is no texlive to install and no 4 GB of dependencies.

Quickstart

# In Claude Code
/plugin marketplace add TUDONG05/stem-shorts
/plugin install stem-shorts@stem-shorts

Install the two external tools the pipeline shells out to, then ask for a video:

pip install edge-tts matplotlib numpy   # narration + renderer
# FFmpeg: apt install ffmpeg  /  brew install ffmpeg

Make a video comparing K-Means and Fuzzy C-Means

The skill scaffolds the project, confirms the script with you, generates the narration, renders the visuals and formulas, and assembles the composition. It stops for your approval at each decision point unless you set AUTO_CREATE_VIDEO=1.

To run the bundled example directly instead, see examples/.

Requirements

No paid service is required. An optional kind: "image" visual calls the Gemini API for the rare line that cannot be computed, and needs a key only if you use it.

How it works

script  →  visuals.spec.json  →  render_visuals.py  →  PNG frames per line
        →  math.spec.json     →  render-math.mjs    →  SVG formulas
        →  generate-vo.mjs    →  edge-tts + ffprobe →  MP3 + real durations
                                                     ↓
                                            index.html, timed to the audio
                                                     ↓
                                                   MP4

Every caption line owns its own time window and its own image. A validator rejects a spec that parks the viewer on one picture for more than two consecutive lines — a rule that exists because an early build left a single static plot on screen for fourteen lines.

Eight computed visual kinds are available — plot, annotated, split, bars, flow, card, quantize, segment — plus image for the rare line that genuinely cannot be computed, and continue to hold the previous frame. Static text cards are the exception, not the default: if more than a quarter of your lines are cards, the script is thin and the fix is a better illustration, not a longer pause.

Why not Manim?

Manim is excellent at what it does, and this is not competing with it.

Manim renders landscape lecture animation, needs a LaTeX distribution, and — in the LLM-driven workflow — depends on a model writing correct animation code. stem-shorts targets vertical short-form, keeps rendering deterministic behind a validated spec, and needs no LaTeX install. If you want a twenty-minute 3Blue1Brown-style lesson, use Manim. If you want a three-minute vertical explainer with plots computed from real data, this is narrower and more predictable.

What is HyperFrames?

HyperFrames is the HTML-to-video framework this builds on: compositions are HTML with data-* timing attributes, rendered to MP4 headlessly.

It is an external dependency, and the version is pinned to 0.7.58 — the version every video in this project was built and tested against. Newer releases exist and are probably fine, but “probably fine” is not a claim worth making about the thing that renders your output, so the bump waits for a real test.

Known limitations

• The skill instructions are written in Vietnamese. SKILL.md and everything in references/ are read by the model, not by you, and translating them has not been worth the maintenance cost of keeping two copies in step. The pipeline itself is language-agnostic; only the default voice is Vietnamese.
• The default voice is vi-VN-NamMinhNeural. Change EDGE_TTS_VOICE in .env to any of the 300+ Edge voices for your language.
• Eight computed visual kinds, no plugin system. Adding a ninth means writing a Python function and registering it — see references/visual-pipeline.md. There is no runtime extension point.
• edge-tts is an unofficial API. It is free and needs no key, but it has no SLA and can rate-limit. The generator retries; a hard failure is reported rather than silently swapped for another engine.
• Rendering is not fast. A three-minute video means dozens of computed frames plus a headless video render. Expect minutes, not seconds.
• The showcase is small. One published Short and the clips above, all unedited output rather than a reel assembled to flatter the tool. Judge it on the bundled example, which you can rebuild yourself.

Contributing

Bug reports, new visual kinds, and additional examples are all welcome — see CONTRIBUTING.md. Issues and pull requests in English or Vietnamese are equally fine.

License

MIT — see LICENSE.

Bundled Noto Serif is licensed separately under the SIL Open Font License 1.1; see skills/stem-shorts/assets/fonts/.