KS

kieransimkin/stemlab

Developer tools
0 stars Quality 42 Trend 8

Local music analysis for stems, timing, beats, harmony, lyrics, MIDI and loopable sections for Shorts. https://kieransimkin.co.uk/

Overview

Local music analysis for stems, timing, beats, harmony, lyrics, MIDI and loopable sections for Shorts. https://kieransimkin.co.uk/ The real-song reference timeline preserves its explicitly hash-pinned public detector JSON byte for byte, including newlines, on Windows and Linux. Generic exports still remove private paths. Container model weights and model-specific caches live under the owning project's ignored /opt/stemlab/models/ directory; they are excluded from the build context and source/package artifacts. the canonical project name is . StemLab is part of the project. The PyPI distribution is named danceflow-stemlab solely because Python package-registry names are globally unique. The Python import, CLI, GitHub repository and container image remain stemlab. Agents using StemLab should inspect its available capabilities, use them fully where they fit the task, and look for useful improvements wherever possible.

README

StemLab

By Kieran Simkin · DanceFlow ecosystem · Vector logo and usage guide.

Local music analysis for stems, timing, beats, harmony, lyrics, MIDI and loopable sections for Shorts. https://kieransimkin.co.uk/

The real-song reference timeline preserves its explicitly hash-pinned public detector JSON byte for byte, including newlines, on Windows and Linux. Generic exports still remove private paths. Container model weights and model-specific caches live under the owning project’s ignored /opt/stemlab/models/ directory; they are excluded from the build context and source/package artifacts.

StemLab by Kieran Simkin · My Songs portfolio · Arcadians EPK · Source

Packaging identity: the canonical project name is StemLab. StemLab is part of the Dance Flow project. The PyPI distribution is named danceflow-stemlab solely because Python package-registry names are globally unique. The Python import, CLI, GitHub repository and container image remain stemlab.

Agents and contributors

Agents using StemLab should inspect its available capabilities, use them fully where they fit the task, and look for useful improvements wherever possible. When a required feature is missing, or a reusable reliability, usability, performance or documentation gap is found, improve this tool rather than maintaining a separate workaround. If no worthwhile change is justified, explain why.

Submit every improvement as a pull request to stemlab, with relevant tests, documentation and a reproducible example. Follow CONTRIBUTING.md and AGENTS.md for the contribution and validation workflow.

Project website: https://kieransimkin.co.uk/danceflow/.

Synopsis

StemLab is a local, evidence-led music analysis toolkit that turns a master recording into inspectable stems, timing, structure, harmony, lyric and sonic evidence, then finds complete-bar loopable sections for Shorts and other short-form videos.

It combines independent separation and music-information-retrieval routes rather than treating any single model as the answer. Results remain traceable through source hashes, manifests, JSON/TSV/NPZ artefacts, portable Sonic Visualiser sessions and a shared React timeline for playback and comparison. StemLab can run as a CLI, Python library, web service, container or local Codex MCP plugin. It is the music-understanding layer of Kieran Simkin’s DanceFlow workflow and can supply downstream motion systems such as the WordPress DanceMoves plugin, while remaining useful on its own.

Model weights are not redistributed by this project. They are fetched from their upstream registries/releases on first use. This avoids silently republishing checkpoints whose licensing may differ from the source code license, and lets upstream integrity metadata be used where available.

DanceFlow and DanceMoves

DanceFlow is the wider BPM and motion-response workflow. StemLab is its music-understanding layer; the WordPress DanceMoves plugin is a related downstream motion-response component. StemLab has no WordPress dependency and communicates through normal analysis artifacts and service APIs, so it remains independently useful and reproducible.

See docs/danceflow.md for the component boundary.

StemLab’s browser workspace imports the reusable react-timeline-sequence control for audio playback, the shared playhead, seeking, zoom and generic sequence lanes. StemLab retains the upload and analysis-specific adapters.

Curated default model set (September 2026)

StemLab id Role Outputs Why it is included
bs_roformer_sw high-capacity vocals, drums, bass, guitar, piano, other (+ upstream instrumental) Current production-oriented BS-RoFormer inference package recommends this six-stem checkpoint.
mvsep_mega53 high-capacity / broad taxonomy 53 raw stems (discovered dynamically) Extremely broad one-checkpoint stem inventory for later assessment. Upstream warns it is memory-heavy and recommends at least 16 GB VRAM.
scnet_xl_ihf high-capacity vocals, drums, bass, other The MSST published checkpoint table reports a 10.08 dB average MUSDB test SDR and 9.92 dB Multisong average for this model.
htdemucs_ft high-capacity established baseline drums, bass, other, vocals Fine-tuned HTDemucs remains a useful independent architecture/baseline rather than another RoFormer variant.
openunmix_umxhq compact / low-latency candidate vocals, drums, bass, other Compact PyTorch/Open-Unmix baseline. StemLab runs niter=0 to favour latency. “Realtime” is hardware- and buffer-dependent: benchmark it on the intended target.

An optional htdemucs_6s model is registered for guitar/piano comparison but is not part of the default top-four group.

Relevant upstreams:

Analysis performed

For every successful separator, every WAV it emits is retained. Stem names are discovered from files rather than truncated to a hard-coded four-stem schema, which is important for MVSep Mega 53. For the master and every stem, StemLab writes both a PNG spectrogram and compressed .npz spectrogram data (time axis, frequency axis, dB matrix, FFT metadata).

Full-resolution spectrogram evidence stays in the .npz; the PNG renderer is bounded to 8,192 time frames so long tracks cannot expand into multi-gigabyte Matplotlib RGBA buffers. If an older run retained the NPZ but failed while drawing the PNG, repair it in place without rerunning models:

stemlab repair-spectrograms "path\to\analysis-results"

Beat-grid comparison also retains an explicit inferred-index hypothesis when saved detections skip whole beats. It reuses the constant-grid fitter with integer gaps, preserving every observed timestamp and the raw half/double-time tempo. A poor sequential fit is replaced only by a better first-anchored gap fit within the existing error guard; both fits remain in the comparison report. Missing detections are not newly detected beats or canonical approval.

Speech transcription

The preferred vocal stem is passed through the Silero VAD implementation bundled with faster-whisper. Speech regions are used to create a timeline-preserving spoken_word.wav (non-speech is zeroed rather than concatenated), then Whisper is run with word timestamps. Outputs include whisper.json, speech_regions.json, transcript.txt, transcript.srt, and words.tsv.

For singing, compare --whisper-vad-mode full when the speech mask removes musical vocal phrases. This retains every input sample in a separate normalized 16 kHz mono transcription_input.wav using faster-whisper’s own PyAV decoder; this full-audio path needs no Torch import. It does not concatenate regions or change source timestamps. The default remains speech. Optionally set --whisper-language en when the language is known. Saved evidence records the mode, whether a mask was applied and the requested language. The public Python API isolate_and_transcribe(..., vad_mode="full", language="en") also accepts an exact master mix or saved vocal stem and a local model-directory path.

These are measured word-reconciliation scores from the same pinned large-v3 model, not acoustic accuracy or listening approval. Canonical wording stays locked; the 70% threshold and observed-anchor requirements still apply. A line with no observed timing remains a hold. Chart context and hashes.

Canonical-word reconciliation accepts edge punctuation on one observed lexical word, so Go, can match canonical Go without changing either text or timing. It does not split one multiword ASR timestamp into invented per-word timings.

Whisper runs in repetition-safe mode by default (condition_on_previous_text=False) to prevent a repeated syllable in one window contaminating later windows. whisper.json records that setting and includes repeated-token diagnostics. Use --whisper-condition-on-previous-text only for an explicit comparison run; a flagged transcript remains evidence requiring section-wise recovery or listening QA, not a trustworthy lyric source.

Beat analysis runs:

  • BeatNet in offline/DBN mode through StemLab’s checksum-verified runtime bootstrap. StemLab uses the maintained madmom-prebuilt wheel and bypasses only BeatNet 1.1.3’s obsolete NumPy/Numba package metadata.
  • Beat This! using the final0 checkpoint and its minimal postprocessor.
  • Beat Transformer, using the original released model code/checkpoints. The model was trained on five demixed mel streams, so StemLab maps BS-RoFormer-SW to vocals, drums, bass, piano, and other + guitar, makes 128-bin mel-power spectrograms at the original 44.1 kHz / 4096 FFT / 1024-hop settings, and averages all eight released fold checkpoints by default. It uses the original madmom DBN decoder if madmom is importable, otherwise a documented SciPy peak-picking fallback.

Beat outputs are stored as JSON, TSV, and (for Beat Transformer) activation NPZ data.

StemLab also supports the official Vamp Plugin Pack, executed through Sonic Annotator. The curated Vamp pass focuses on musically useful outputs: Chordino chord transcription and harmonic-change likelihood; NNLS chroma and bass chroma; Queen Mary key and tonal-change detection; concert-pitch tuning; pYIN melody/F0 and monophonic note transcription on the preferred separated vocal stem; Silvet polyphonic note transcription; Segmentino song-structure segmentation; and the Queen Mary Vamp beat/bar tracker. Raw CSV, pinned transform files, JSON, NPZ and plots are retained under vamp/, with the most useful melody/harmony layers also embedded in the Sonic Visualiser session.

Comprehensive sonic, harmonic, rhythmic and semantic analysis

StemLab 1.0 consolidates a higher-level evidence-fusion pass without discarding any of the existing low-level outputs. The default deep pass now includes:

Action Evidence / model Main output
Sonic profile pyloudnorm BS.1770 + librosa DSP LUFS, dynamics, true-peak estimate, timbre and stereo
Groove / meter all successful beat grids + onset analysis tempo stability, meter, swing, offbeat energy and quantisation error
Tempo regimes beat-grid BIC models + fixed-grid residuals stable sections, gradual ramps, abrupt changes, same-BPM phase skips and per-section precision
Harmony Chordino + NNLS chroma + QM key/tuning chord progression, harmonic rhythm, key evidence and tonal changes
Functional structure All-In-One-Infer 3.1 BPM, beats/downbeats and intro/verse/chorus/bridge/outro-style sections
Rhyme / prosody CMU Pronouncing Dictionary + timing rhyme scheme, internal rhyme, syllables, repetitions and delivery rate
Lyric semantics SentenceTransformers theme similarity, continuity and unsupervised line clusters
Song map StemLab evidence fusion section-level sonic/rhythm/chord/lyric summaries on one timeline

Optional actions include Basic Pitch polyphonic MIDI/note transcription on isolated instrument stems and MuQ-MuLan zero-shot audio/text semantics. MuQ-MuLan’s released weights are CC-BY-NC 4.0, so that route is deliberately opt-in and its licence is embedded in every result. Embedding similarities are labelled as similarities, never probabilities.

Inspect the routes and their dependencies with:

stemlab analysis-actions

Useful switches include --no-structure, --no-text-semantics, --audio-semantics, --basic-pitch, and --all-in-one-embeddings.

The derived artifacts live under deep/ (sonic/, rhythm/, harmony/, structure/, lyrics/, semantic_text/, optional semantic_audio/ and basic_pitch/, plus song_map/song_map.json and summary.json).

Find DanceRudiments matching candidates from each stem

Install the separately maintained optional native package with python -m pip install -e ".[rudiments]". StemLab calls dancerudiments.catalogue() and dancerudiments.sample() directly—no cloned movement functions or bundled catalogue. It extracts per-stem RMS attack peaks, maps them onto detected beats, and ranks 16-beat windows against position/speed/acceleration salience of actual native motion samples. Scores are comparative similarities, not probabilities or guarantees of a good motion.

# Match the mix plus every saved stem without running separation again.
stemlab rudiments ./analysis-master --top 20
# Limit an existing analysis to two stems, or choose one of six official collections.
stemlab rudiments ./analysis-master --stem drums --stem bass --collection club
# Include matching as an opt-in part of a new full analysis.
stemlab analyze master.wav --output ./analysis-master --rudiments
# The bundled Arcadians MP3 has a documented BPM but no pre-separated stems.
stemlab rudiments 'examples/arcadians/Arcadians - 320kbps.mp3' \
  --bpm 145 --collection initial --top 28 --output ./arcadians-rudiments

Measured Arcadians benchmark

The mix-only offline benchmark found 1,821 energy-rise peaks and 15 candidate patterns at ≥0.68 similarity among the authentic 28-pattern DanceRudiments initial collection, led by groove_b_played (0.739), groove_b_grid (0.737), and lfo_morph (0.724). The benchmark read DanceRudiments’ officially compiled sample tables through an offline adapter; the native C+±extension test is separate and skips when that optional package is not installed. The BPM grid starts at zero with unverified beat phase. No stem separation was claimed.

These are actual Chromium screenshots of the measured local HTML report, not schematic illustrations or captures of the main React timeline.

The analysis writes deep/rudiments/report.json, index.html, and overview.png. See method, score caveats, provenance, examples and screenshot reproduction.

Native CI coverage: the main GitHub Actions CI workflow now has a dedicated Python 3.10/3.11 job that installs the official DanceRudiments binary wheel, requires the actual C++ module, and runs the Arcadians test with skipping disabled. Fewer than 12 real-audio matches fails CI; passing runs publish the JSON, HTML and measured overlay PNG as downloadable CI artifacts. See the native CI validation instructions.

BPM-synchronised recordings, harness and batch generation

Export any number of independently routed master/stem dimensions at 64 pips per beat, using the separately maintained DanceRudiments float64 recording format. Every exported DanceRudiments dimension is projected onto the same bipolar -1 to +1 movement scale before JSON, legacy-pack or harness output. The normalization option selects the amplitude reference (PCM unity, shared peak, or per-axis peak); it no longer changes the output coordinate range. Silent axes remain centered at 0, and signed gain is applied after bipolar projection.

A BPM estimate or confirmed value is required; raw audio uses --bpm, while saved analysis can supply current beat evidence. --bpm-confirmed records explicit confirmation, separately from estimated/unverified beat phase.

# Bass on X, treble on Y; generate JSON plus the actual DanceRudiments visualizer.
stemlab record-rudiment master.wav --bpm 145 \
  --axis x=master --low-pass x=180 \
  --axis y=master --high-pass y=3000 --output ./bass-treble
# Explore filtered/unfiltered master and every saved stem, including N-D combinations.
stemlab generate-batch ./analysis --bpm 145 \
  --dimensions 2 --dimensions 3 --dimensions 6 \
  --max-candidates 96 --seed 17 --include-audio --output ./movement-batch
# Fourier analysis: 32 frequency buckets become exactly 32 recording dimensions.
stemlab record-rudiment master.wav --bpm 145 --frequency-bands 32 \
  --fft-size 2048 --normalize global --output ./frequency-32
# Reuse a saved stem instead of analysing the master spectrum.
stemlab record-rudiment ./analysis --frequency-bands 32 \
  --frequency-source htdemucs_6s/drums --normalize global --output ./drum-frequency-32

--frequency-bands N maps ascending frequency bands to band_000 through band_N-1, with one periodic-Hann real FFT per pip and window-corrected RMS band levels. The default is equal-width bands from DC to the source Nyquist frequency; --frequency-min, --frequency-max and --frequency-scale linear|log select a different range/spacing. Log spacing requires a positive minimum. --fft-size controls the frequency/time resolution; unresolved empty bands are errors. Channel powers are averaged without cancelling antiphase stereo. The existing clock, bipolar projection, gain, normalization, JSON and native harness routes are reused. This produces band-level motion, not separated audio files or instrument stems. See Fourier options, calibration and boundaries.

Open the output index.html for searchable movement cards, the existing four-card comparison, shared audio transport, review choices/notes, and X/Y/Z selectors for any stored dimension. Native C++/WASM playback and visualizer assets stay in DanceRudiments, not a copied engine or lookalike page in StemLab. Recordings stop at their end, retain all dimensions, and do not silently become loops.

Choose per-axis peak/mean-absolute-amplitude windows and low/high/band-pass routes. Batch mode covers each source, explores seeded combinations, and reuses cached filter results/deduplicated columns instead of an unbounded Cartesian product. Source audio is copied only with --include-audio; no inference or model download runs. Outputs must be new directories. This requires the companion DanceRudiments recording/harness update; older published packages lack the API. Existing matching and XYZ formats remain compatible (--legacy explicitly exports a bounded XYZ loop).

See installation, all options, limits, timing, demo and validation.

Real Chromium capture of the bundled synthetic-audio batch, not Arcadians model inference. Restricted-environment QA used in-memory loading; normal file loading and audible listening are not claimed by that screenshot.

Export a master or stem as a signed waveform rudiment

The existing record-rudiment exporter also supports --mode waveform. It downsamples actual signed PCM onto DanceRudiments’ 64-pips-per-beat recording clock, using zero-phase anti-alias filtering and pip-centre sampling. Unlike rectified peak/average/Fourier envelopes, negative waveform values remain negative and silence stays at zero. Source-rate low/high/band-pass filtering is applied before anti-aliased downsampling.

# Master, with optional low-pass before downsampling.
stemlab record-rudiment master.wav --bpm 145 --mode waveform \
  --axis x=master --low-pass x=35 --output ./master-waveform
# A saved stem from an existing analysis, reusing its BPM evidence.
stemlab record-rudiment ./analysis --mode waveform \
  --axis x=htdemucs_6s/bass --high-pass x=12 --output ./bass-waveform

Use --axis-mode x=waveform to mix signed PCM and envelope axes in one recording. High audio frequencies do not survive a genuine downsample to a roughly 100–200 Hz movement clock; for instrument energy use --mode peak or average instead. Signed stereo channels are averaged (opposite phase can cancel); the native DanceRudiments recording JSON, WASM viewer, source hashes, normalization and existing transactional exports are reused. See signed waveform resampling.

Correct beats and export variable-tempo synchronisation

The timeline now includes a Beat correction editor, automatically seeded at all exact consensus beat sample frames. Drag beats individually against the master/stem waveform, nudge at single-sample precision, insert missing beats, edit the local BPM envelope, inspect the phase CV curve, shift global phase, and explicitly select the anchor for Beat 1. The original detector output is never modified. Corrections live separately at timing/beat-timeline.json with undo/redo and revision-safe saving.

Export Beat 1 + full beat timeline JSON downloads one versioned stemlab.beat-timeline.v1 object containing every corrected beat, locally varying BPM intervals, original master/consensus hashes, beat_1_start_seconds, beat_1_start_pips and the signed sync_phase_offset_pips (plus DanceMoves 16-tick conversions). The viewer also supports copying this whole JSON value, importing previously edited versions, and restoring original consensus anchors. Export preserves irregular timing; no constant tempo approximation is substituted.

DanceMoves compatibility: the existing WordPress beat-alignment.v1 importer is constant BPM only and rejects this schema. To drive variable rhythm in DanceMoves, add the corresponding beat-timeline.v1 import/scheduler support (including WordPress exact-audio hash checks) before pasting/using it. The old beat-offset handoff remains available for true constant-BPM songs.

Read editable beat timelines, schemas and consumer requirements. Rebuild src/stemlab/web/app.js and timeline.css from the unchanged published React timeline dependency after applying source changes: npm ci && npm run build:web; check in both outputs with the source.

First-beat alignment and tempo families

Compare the saved consensus and every individual detector/source at canonical and combined BPM, with exact signed 64-pip offsets that map the first detected beat to pip 0. Half- and double-time are normal interpretations: observed beats remain intact, alongside ×½ / ×1 / ×2 comparisons and an explicitly estimated constant-grid fit. The real timeline shows a blue dashed original grid and an amber solid offset grid, with independent BPM and detector selectors.

stemlab beat-offset path/to/results --audio path/to/exact-master.mp3 --output new-offset-report

The Truth is a Virus reference uses real saved BeatThis/BeatNet detections and consensus. Its fitted tempo is 109.0013 BPM; 95% of consensus timestamps are within 17.627 ms of that grid. The guarded grade remains loose because first-anchored end drift is 67.114 ms. Its raw median tempo and unconfirmed canonical BPM remain visible. Read the exact mapping, workflow and limitations.

Arcadians: complete analysis timeline and four movement demos

Every new release builds one public site containing five working examples, all covering the full original Arcadians song:

  • Analysis timeline: the real StemLab React viewer, original audio, six genuine HTDemucs stems, waveforms/spectrograms, master/percussion beat estimates and consensus, Whisper words, all 13 curated Vamp analyses, functional structure, sonic/rhythm/harmony/lyric analysis, text semantics, song map and loop discovery.

  • Movement comparison: the existing 48 DanceRudiments movements with 2D/3D/6D filtered/unfiltered master routes, 145 BPM, 64 pips per beat and float64 recordings. This example remains master-derived; it does not claim its movements use separated stems.

  • Stem-pair comparison: 60 two-dimensional recordings with different actual stems driving X and Y.

  • Amplitude-mode comparison: 30 fixed input/filter pairs, each with peak, average and four moving-average windows: 180 recordings on shared normalization references.

  • Fourier frequency bands: select 4, 8, 16, 32, 64 or 128 dimensions derived from the full original master. Each equal-width band supplies one dimension through a centered Hann 2048-point rFFT and window-corrected band RMS. Global normalization preserves relative band strengths within each resolution; it does not calibrate resolutions against each other. Choose any X/Y/Z axes or see every band in the shared Bars view.

The root landing page links to all five. These are deployment destinations; a patch or a green build alone does not establish that the new page is live.

The same CI publication also retains the separate The Truth is a Virus beat reference. Its frozen detections are disclosed; current CI generates its fits, offsets and real timeline rather than asserting a fresh learned-model run.

The timeline’s arcadians-core-v2 recipe runs all the selected core families on the complete checksum-verified MP3. It is not the five-separator full comparison profile or every optional MIDI/evidence/NC model. Coverage, dependency versions and source/export revisions are disclosed. Missing stems, beat results, Vamp plugins or required deep-analysis evidence fail the build; there is no synthetic fallback. Artist-confirmed 145 BPM, lyrics, manually edited lyric timing and artist sections stay distinct from machine estimates. A completed loop scan may legitimately return zero accepted loops and explain unresolved sections.

Build and publish all five examples

Start Actions → Arcadians release demo → Run workflow → main. The workflow first runs the full CPU analysis in StemLab’s existing Docker environment, then exports and browser-tests the timeline. It also builds/tests the original movement example and both new comparison pages, reusing the exact separated float WAVs, plus all six full-master Fourier resolutions, and combines all five into a single verified Pages artifact. The model analysis is substantially heavier than master-peak extraction; upstream model downloads and inference must succeed. A new tagged release calls the same combined workflow.

All five examples include audio playback, without autoplay. The timeline needs no live StemLab API, Socket.IO connection or CDN: the original React renderer loads precomputed local data, waveform envelopes and spectrogram strips. It supports play/pause, seeking, zoom/Fit, and saved-loop audition when loops exist. The original MP3 is unchanged; public stem audio uses full-duration PCM24 FLAC listening derivatives with their conversion disclosed. Original float WAVs and raw results are kept in a separate seven-day Actions analysis artifact.

Keep Pages Source = GitHub Actions and STEMLAB_PAGES_ENABLED=true for automatic publication after a successful manual demo or a tagged release. Preserve github-pages environment approvals and allow the relevant main/tag refs. Alternatively, Publish Arcadians demo to Pages → Run workflow → main publishes a selected successful combined demo (blank run ID chooses the latest), without rerunning models or making a package release. Old artifacts without all three additional galleries and the timeline are rejected so they cannot overwrite the complete five-example site. The standalone Arcadians complete analysis timeline job is build-only, not a separate Pages deployment.

The publisher verifies all viewers’ provenance, artifact checksums and successful HTTP browser reports, then probes the actual public pages, timeline assets/data and partial MP3 responses. Failed or in-memory-only QA cannot authorize publication. QA files, private folders, model caches and runtime logs stay outside the public upload. All five examples are included in StemLab-Arcadians-visualizer.zip, with file hashes and build provenance. Native recording downloads are losslessly compressed as .json.gz; decompress before loading with the native recording API. No samples are discarded or quantized. Every resolution has mandatory native/bar/browser/audio QA. An incomplete build never replaces the live site.

Export an existing analysis as a static timeline

# Install the server extra for shared export helpers; do not start a server.
stemlab export-timeline ./analysis-arcadians \
  --include-audio --output ./arcadians-timeline-site

The output must be a new directory outside the analysis. Host it over static HTTP(S), not file://. Audio/lyric inclusion is explicit. Generic exports show saved evidence without claiming the complete Arcadians recipe ran.

See complete recipe, workflow, static export, timing and validation and movement/audio details and Pages configuration.

The browser workspace is intended to make the analysis output inspectable, not just downloadable. The illustrations below explain the main analysis families using the same shared timeline vocabulary as the frontend: one clock, one playhead, zoomable lanes, and consistent section alignment across spectrogram, beat, lyric and deep-analysis layers.

Illustrations, not captured frontend screenshots. These SVGs are generated by scripts/render_readme_analysis_gallery.py; the script does not execute the React timeline component, decode audio, run separation or perform neural inference. Only the Arcadians section labels, lyric text/timing and reference BPM come from the included artist-authored metadata. Other plotted patterns and example chord labels are explicitly schematic, not measured results for Arcadians. In particular, these pictures are not evidence of vocal-free cuts, measured tempo stability, chord/key estimates or loudness values. Actual frontend captures remain outstanding.

1. Separation, spectrograms, beats and sections

This is the core inspection view. It explains how StemLab lines up:

  • the AI / reference section map (deep/structure/ and deep/song_map/),
  • the beat and bar grid from BeatNet, Beat This! and Beat Transformer (beats/*.json, beats/*.tsv),
  • the master spectrogram, and
  • each stem spectrogram written under spectrograms/.

In practice, this is the view you use first when checking whether a separator, beat model or section model has produced outputs that are musically believable.

2. Lyrics, speech regions and transcript QA

StemLab keeps lyric and speech analysis on the same clock as the audio:

  • reference lyric timing from a known-correct LRC or JSON source,
  • Whisper word timing from the preferred vocal stem,
  • speech-region / VAD evidence indicating detected speech activity; this alone does not establish the absence of singing,
  • the derived spoken_word.wav timeline-preserving speech isolate, and
  • downstream rhyme, prosody and repetition-safety diagnostics under speech/ and deep/lyrics/.

The important idea is that you can compare what the model heard against what the artist supplied, then immediately relate both to loop boundaries, choreography cues or lyric-sheet QA.

3. Harmony, melody and song-map fusion

StemLab’s harmonic and structural analysis is wider than a single chord track. The harmony/song-map family brings together:

  • chord and harmonic-change evidence from the curated Vamp pass,
  • key, tonal-change and tuning outputs,
  • melody / F0 style lanes derived from vocal analysis,
  • the section-level song map that fuses structure, rhythm, harmony and lyric evidence into one readable narrative.

These outputs live primarily under vamp/, deep/harmony/ and deep/song_map/, and they are the layer that makes the raw low-level DSP outputs easier to interpret in musical terms.

4. Sonic profile, rhythm regimes and semantics

The higher-level deep pass also tracks how a song feels over time:

  • sonic profile summaries (loudness, peaks, dynamics, timbre and stereo behaviour),
  • groove / rhythmic metrics such as swing tendency, offbeat energy and quantisation error,
  • tempo-regime detection for stable regions, ramps and abrupt changes,
  • semantic summaries from lyric/text analysis and, optionally, zero-shot audio semantics.

This is the analysis family that turns StemLab from a simple BPM-and-stems tool into a richer musical understanding layer for DanceFlow and downstream motion-response systems.

5. Where loops fit in

The loop-discovery patch does not replace the earlier analyses; it depends on them. Loop selection uses:

  • section labels to find candidate verse and chorus spans,
  • the beat grid to align cuts to complete bars,
  • full-vocal stem and available timing evidence to avoid detected vocalisations; VAD alone is insufficient,
  • and waveform seam checks to reduce discontinuities; accepted joins still need listening checks.

That means the loop lane is best understood as an applied synthesis of the structure, rhythm and vocal analyses already present in StemLab.

Loop discovery and playback

StemLab now attempts at least one complete-bar loop for every identified verse and chorus occurrence. It checks the beat grid, vocal-clear cut neighborhoods and every audio channel’s waveform seam. Unsafe or unsupported sections are reported rather than forced. Start/end sample frames, quality diagnostics and provenance are written to deep/loops/loops.json and loops.tsv by default.

# New analysis: loop metadata is included in the default deep pass.
stemlab analyze master.wav -o analysis-master --profile practical

# Also save exact native-rate WAV slices.
stemlab analyze master.wav -o analysis-master --profile practical --export-loops

# Reuse existing separation, structure and beat results; no models rerun.
stemlab loops analysis-master --export-loops

# Keep all candidates within a chosen short-video edit budget, in seconds.
stemlab loops analysis-master --max-seconds 30 --per-section 3

# Audition possible new cut pairs anywhere in the song, even where the passage
# does not already repeat. These near-misses are NOT accepted seamless loops.
stemlab loops analysis-master --mode exploratory --search-scope whole_song \
  --max-seconds 30 --per-section 20 --exploratory-algorithm spectral_context

In the existing analysis frontend, choose a highlighted loop region or the Loop selector, tick Enable loop, then press Play. The same playhead, seeking, zoom and spectrogram timeline remain in use. Previewing a loop does not require exported WAVs and writes no files. Zoom to loop focuses the shared timeline.

See the complete loop workflow for installation of both patches, sample indexing, thresholds, limitations and reproducible screenshot examples.

For line-complete alternatives when a musically satisfying restart is the goal, use stemlab phrase-loops analysis-master --feedback artist-ratings.json --export-audio. This opt-in search compares the musical context around both cuts, preserves canonical lyric occurrences, uses explicit artist feedback and selects diverse passages. Suggestions remain separate from strict accepted loops. See phrase-loop methodology, feedback format and synthetic example. For longer excerpts, add --min-seconds 20 --max-seconds 30; optional --max-bar-phase-error 0.125 also screens out mismatched measured bar positions. These filters suggest auditions and do not establish seamless or listening approval.

Recover a Short or video’s source timing

stemlab match-clip master.wav published-original.mp4 --output new-match.json locates a contiguous soundtrack excerpt without running models or downloading anything. It retains both original hashes, timing precision, competing source positions and independent window checks. Matched reports provide source offset, soundtrack duration and player start for synchronised consumers. Short fades and compression are tolerated; ambiguous repetition, edit jumps, different recordings and rate changes remain unresolved. Use --clip-start / --clip-end for an explicit known soundtrack interval. FFmpeg is required for video decoding. See clip matching for limitations and a synthetic example.

Audio-to-MIDI model comparison

MIDI extraction is an optional, independent model pass. It adds Transkun V2 and high-resolution note/pedal piano transcription, plus MR-MT3 and YourMT3 multi-instrument transcription, alongside Spotify Basic Pitch. Automatic source selection uses piano/keyboard stems for the piano specialists, pitched stems for Basic Pitch and the mix for MT3-family models. Missing stems are reported, never silently substituted. The legacy --basic-pitch command remains supported.

# Install the chosen runtime; the standard profiles do not install MIDI models.
python -m pip install -e ".[midi-transkun]"
stemlab midi-models

# Use an existing separated piano, without rerunning any earlier analysis.
stemlab midi analysis-master --model transkun

# Or transcribe a directly supplied piano recording to a new folder.
stemlab midi piano.wav -o midi-piano --model transkun

# Full mix comparison requires the separately installed midi-mt3 extra.
# Both requested checkpoints need explicit first-use download consent.
stemlab midi mix.wav -o midi-comparison --model mr_mt3 --model yourmt3 --allow-model-downloads

Each model/source gets its original .mid, tempo-aware notes.json and notes.csv, an optional real note piano-roll.png, source/checkpoint hashes, package versions and a worker log. Native MIDI controllers, pitch bends, tempo changes and drums are preserved. Note sample numbers are original-rate estimates, not a claim that the transcription is sample-accurate ground truth. The MIDI clock is not used to overwrite StemLab’s measured beat grid.

New analyses can opt in with repeated --midi-model switches and --midi-allow-downloads. Existing Codex installations gain a workspace-scoped stemlab_start_midi_scan job tool after updating the Python installation and restarting the session. Models remain optional and no Python dependencies are automatically installed. Incompatible research runtimes can use separate --backend-python MODEL=PATH interpreters.

See the MIDI guide for all models, installation choices, checkpoint and licensing caveats, saved-stem targeting, Codex usage, exact output semantics, real-inference smoke commands and test limitations. The React timeline remains the analysis inspector; this patch adds no MIDI synthesizer or new interactive piano-roll lane. Static note plots are not frontend screenshots.

Align timed musical cues to detected beats

Use saved analysis to snap an LRC cue file without rerunning models or editing audio:

stemlab snap-cues analysis-master "musical cues.lrc" -o cue-review
stemlab snap-cues analysis-master "musical cues.lrc" -o cue-review-100ms --tolerance-ms 100
stemlab generate-cues analysis-master -o generated-cues --title "Song title"

Only timestamps change: bracketed labels such as [DROP: CHORUS 1], metadata, encoding and line layout are preserved. A cue with no detected beat within ±150 ms is highlighted as REVIEW, with its nearest-beat gap, and kept unchanged by default. --force-snap moves distant cues too but retains their warnings. --downbeats restricts targets to reported bar starts; --beat-model selects a specific detector. No constant-tempo beats are invented to fill detection gaps.

Each new output folder contains the rewritten .beat-snapped.lrc, report.json, a colour-coded review.html, and review.txt. Exact mode preserves the stored beat timestamp; explicit millisecond/centisecond compatibility modes report rounding. Source-rate sample-frame estimates are included when the rate is known. Use --fail-on-review to write results and exit 2 when review is needed. Generated section-cue files use a title-specific, WordPress-safe filename so multiple releases can be uploaded and mapped without ambiguous duplicate names. For the retained 3 October 2026 future-release batch, scripts/run-upcoming-full-cues.ps1 resumes full analyses without overwriting an incomplete directory, and scripts/summarize-upcoming-cues.ps1 reads each closed manifest/report to produce one reproducible hash, cue-action and error-count row per release. The summary script refuses missing or ambiguous generated LRCs.

For a new analysis, add --snap-cues "musical cues.lrc" and optionally --cue-tolerance-ms 100. Cue alignment uses that run’s detected events and appears in analysis.json. See the cue alignment guide for exact precision, offsets, all switches, safety rules and interpretation of warnings.

Complementary evidence models

StemLab now has an optional evidence-model layer for models that add a different kind of musical evidence instead of duplicating the normal separation or MIDI backends. Nothing in this layer is enabled by a normal profile.

stemlab evidence-models

# Singing/speech/music activity + continuous pitch on saved vocals.
stemlab evidence analysis-master -o evidence-vocals \
  --model firered_aed --model swift_f0 \
  --model-path firered_aed=/models/FireRedVAD/AED

# Align approved lyrics without replacing them.
stemlab evidence analysis-master -o evidence-align \
  --model qwen_forced_aligner --canonical-text lyrics.txt --allow-model-downloads

# Independent harmony evidence.
stemlab evidence analysis-master -o evidence-chords --model lv_chordia

The built-in registry covers FireRed AED, HeartTranscriptor, Qwen3 ForcedAligner, SwiftF0, SongFormer, lv-chordia and ADTOF PyTorch. It also exposes explicit external-runtime bridges for GAME, SheetSage2, MOSS-Music and AudioSep so large or restricted research dependencies never get silently bundled into StemLab. Outputs stay independent: singing activity, forced alignment, pitch, structure, chords and drum hits are review evidence, not a hidden combined ground-truth score.

Saved analyses automatically route vocal models to a preferred full vocal stem, drum transcription to a drum stem, continuous pitch to pitched stems, and structure/harmony models to the master. Missing specialist stems are reported. Use --backend-python MODEL=/path/to/python for conflicting research environments.

See the evidence-model guide for model terms, routing, external bridges and interpretation rules. scripts/benchmark_evidence_corpus.py probes an authorised evaluation collection without copying audio into the repo.

Install

Python 3.11 is recommended for the broad optional-model ecosystem. The beats and all extras install Beat This! plus the compatible madmom-prebuilt decoder. BeatNet 1.1.3 itself still publishes obsolete NumPy/Numba dependency pins, so StemLab downloads its official pure-Python wheel separately, verifies the published SHA-256, and exposes only the BeatNet code and bundled model weights from its cache. This route is supported on Python 3.11-3.13; a missing BeatNet result must not be mistaken for a successful detector run.

From PyPI, install the released StemLab distribution with:

pip install danceflow-stemlab

The installed Python package and command remain stemlab. For development from a source checkout:

python -m venv .venv
. .venv/bin/activate              # Windows: .venv\Scripts\activate
python -m pip install -U pip
pip install -e ".[all]"

SCNet and Beat Transformer are research repositories rather than stable pip inference APIs. They are isolated/downloaded on demand. To prepare everything in advance:

stemlab bootstrap all

For BeatNet alone:

pip install "danceflow-stemlab[beats]"
stemlab bootstrap beatnet
stemlab doctor

stemlab doctor reports BeatNet as ready only when the normal package or the verified StemLab cache can actually be found. analyze --no-bootstrap never downloads BeatNet and records the unavailable backend instead.

For only the Vamp analysis stack:

stemlab bootstrap vamp

bootstrap vamp downloads the pinned Sonic Annotator runtime and launches the official Vamp Plugin Pack installer. Complete the installer once, then rerun the command if needed to verify that the requested plugin outputs are visible.

CLI

Full requested analysis:

stemlab analyze master.wav --output ./analysis-master --profile full --device auto

Useful alternatives:

# Skip the 53-stem model for a materially lighter run
stemlab analyze master.wav -o ./analysis-master --profile practical

# Run only selected separators
stemlab analyze master.wav -o ./analysis-master \
  --model bs_roformer_sw --model scnet_xl_ihf --model openunmix_umxhq

# Fail immediately rather than recording a backend failure and continuing
stemlab analyze master.wav -o ./analysis-master --strict

# Skip Vamp if the plugin pack is intentionally not installed
stemlab analyze master.wav -o ./analysis-master --no-vamp

stemlab models
stemlab doctor

By default model weights and external research code are cached outside the output folder; all generated analysis artifacts and datasets are written inside the output folder. The copied master is also placed in input/ so the Sonic Visualiser session is portable as one directory.

Output layout

analysis-master/
  input/master.wav
  stems/
    bs_roformer_sw/*.wav
    mvsep_mega53/*.wav
    scnet_xl_ihf/*.wav
    htdemucs_ft/*.wav
    openunmix_umxhq/*.wav
  spectrograms/
    master.png
    master.npz
    /.png
    /.npz
    speech/spoken_word.png
    speech/spoken_word.npz
  speech/
    spoken_word.wav
    speech_regions.json
    whisper.json
    transcript.txt
    transcript.srt
    words.tsv
  beats/
    beatnet.json / beatnet.tsv
    beat_this.json / beat_this.tsv
    beat_transformer.json / beat_transformer.tsv
    beat_transformer_activations.npz
  vamp/
    report.json
    transforms/*.n3
    raw/*.csv
    data/*.json
    data/*.npz
    plots/*.png
  deep/
    sonic/
    rhythm/
    harmony/
    structure/
    lyrics/
    semantic_text/
    song_map/
    rudiments/             # optional native DanceRudiments candidate report
      report.json
      index.html
      overview.png
    summary.json
  sonic_visualiser/
    session.sv
    session.xml
    open_sonic_visualiser.bat
    open_sonic_visualiser.sh
  analysis.json
  manifest.json

session.sv is genuine Sonic Visualiser bzip2-compressed session XML. It contains a master pane with beat/downbeat/Whisper time-instant layers and one synchronized waveform + Sonic Visualiser spectrogram pane for every separated stem. The uncompressed session.xml is retained for inspection/debugging. The Windows .bat looks on PATH and in the usual Program Files locations.

Web service

StemLab includes an upload/timeline web UI plus HTTP and Socket.IO APIs for content-addressed analysis jobs. See docs/web.md for launchers, endpoints, events and the Arcadians reference workflow.

Documentation

Codex plugin and MCP server

Inspect through the real React timeline first. The Codex plugin now exposes stemlab_open_timeline to serve saved results in StemLab’s bundled react-timeline-sequence frontend. Use an available local browser to compare waveform, spectrogram, beat and section lanes, select/enable loops, zoom and audition the repeat. Exact JSON reports support numerical claims; saved PNGs are a fallback when browser access is unavailable. Screenshots must come from the rendered component, never a redraw. stemlab_close_timeline closes the private, read-only viewer without changing results. See the inspection workflow.

Use StemLab from Codex to analyse timing, inspect real evidence and find/export loopable music sections for Shorts videos. This is a local stdio MCP server plus an installable skill bundle, not a remote audio-upload service or a video editor. It reuses the existing pipeline and preserves prior result folders.

# From this patched checkout, in an activated environment:
python -m pip install -e ".[codex]"
stemlab-codex configure --workspace "/absolute/path/to/music-workspace"
stemlab-codex doctor
codex plugin marketplace add .

Install StemLab from the local plugin directory on a supported Codex host; for CLI-only use, register the MCP command directly. See setup and tools for Windows paths, model extras, consent, polling and exact sample conventions.

Release builds validate the MCP integration and attach a versioned plugin ZIP, checksums and MCP Registry metadata to the GitHub release. A release-only codex-plugins branch provides a marketplace channel after PyPI publication. This does not automatically submit to OpenAI’s public Plugins Directory. Distribution and optional MCP Registry publishing explains the separate channels and approval requirements.

Codex skill

The repository includes a reusable Codex skill at skills/use-and-improve-stemlab. It makes StemLab the default analysis engine, selects a proportionate profile, preserves evidence boundaries, and routes reusable capability gaps back into StemLab with tests instead of creating an undocumented parallel stack.

Python API

from pathlib import Path
from stemlab.models import FULL_PROFILE
from stemlab.pipeline import run_pipeline
from stemlab.types import PipelineConfig

result = run_pipeline(PipelineConfig(
    input_wav=Path("master.wav"),
    output_dir=Path("analysis-master"),
    models=FULL_PROFILE,
    device="auto",
))

External runtime/cache details

The BS-RoFormer adapter uses bs-roformer-infer, which manages its own checkpoint registry and SHA-256 verification. SCNet is pinned to the v1.0.15 SCNet XL IHF release asset URLs; StemLab records computed hashes after download. Beat Transformer is cloned from its original upstream and consumes the released fold checkpoints from that repository. Set STEMLAB_CACHE=/some/path to relocate StemLab’s external cache.

The full run is intentionally expensive. Mega-53 is the dominant VRAM/storage pass and dozens of stems mean dozens of additional spectrogram files and Sonic Visualiser panes. Use --profile practical while iterating and --profile full for exhaustive analysis.

Releases

Version tags publish StemLab to GitHub Releases, PyPI (danceflow-stemlab), GHCR and Docker Hub after validation. See docs/publishing.md for the release contract and Trusted Publishing setup.

Reproducibility and licensing

analysis.json records model/backend metadata and errors; manifest.json hashes every output artifact. Model checkpoints, research code and datasets retain their upstream terms. In particular, do not assume that an MIT-licensed inference wrapper automatically grants redistribution rights for every checkpoint it can download.

Arcadians timeline CI decoder and artifact checks

The complete Arcadians demo checks lint before expensive analysis. Its Docker runtime includes pinned TorchCodec and performs a synthetic MP3 decoder smoke test; the timeline builder also checks the original Arcadians MP3 before model inference. Missing decoders fail early without changing the source audio or relaxing the complete-analysis/publication gates. After Docker exits, the workflow restores runner ownership of its dedicated output folder so private-mode JSON files remain readable for browser validation and diagnostic artifact upload, including on failed analysis runs. See Docker checks and the timeline workflow.

For a new combined timeline and movement deployment, run Arcadians release demo on main. It calls Arcadians complete analysis timeline internally; starting that standalone workflow too duplicates analysis and does not itself publish the combined Pages site.

The Docker build also checks audio-first lyric-semantic imports in a fresh, offline process: it exercises both resampy Kaiser filters before resolving the actual Transformers and SentenceTransformer classes. This catches lazy-loading failures before full-song inference. The container runtime temporarily pins resampy==0.4.2 and setuptools==81.0.0 because Basic Pitch 0.4.0 excludes the newer resampy that no longer needs pkg_resources. These are container-only compatibility pins, not a downgrade of normal package-build tooling. This smoke test does not download weights or claim semantic inference; the full analysis and browser/publication gates remain required.

Separated-stem movement batches and controlled smoothing

stemlab generate-stem-batch separates a raw song once (default htdemucs_6s) or reuses a saved analysis, then generates 2D DanceRudiments with different stems on X and Y. It covers raw ordered pairs and seeded low-/high-/band-pass variants; 120 candidates is the bounded default. Supply a BPM estimate or confirmed value for raw audio. --analysis-output keeps the prepared stems for reuse.

stemlab generate-stem-batch song.wav --bpm 145 --bpm-confirmed \
  --analysis-output ./song-stems --include-audio --output ./stem-pairs
stemlab generate-average-comparison ./song-stems --bpm 145 --max-pairs 32 \
  --window-beats .125 --window-beats .25 --window-beats .5 --window-beats 1 \
  --include-audio --output ./average-comparisons

The second command keeps each pair’s source, filters, orientation and amplitude scale fixed while comparing peak, per-pip average, and moving-average settings. Moving average is a separate centered smoothing window on the rectified amplitude envelope, configurable in beats; it does not average signed audio or change the 64-pips-per-beat / float64 recording format. It is also available in record-rudiment and generate-batch via --mode moving-average and --moving-average-beats; individual axes use --axis-window-beats NAME=BEATS.

The Arcadians workflow now assembles five examples: the existing arcadians/ movements and timeline/arcadians/ analysis, plus arcadians-stems/ (60 separated-stem movements) and arcadians-averages/ (30 pairs × six settings). It hands off the timeline analysis’s real float stems, without rerunning separation, and bundles the original song for synchronized playback. Both new galleries have mandatory native/browser/audio checks and public-URL verification before success is reported. The Fourier page adds six full-master resolutions without rerunning separation. All pages remain in one Pages deployment. See settings, boundaries, budgets and examples.

Beat detection after separation: master and percussion hypotheses

With beat analysis enabled, the pipeline completes separation first, then runs BeatNet and BeatThis on the master and every labelled drum/percussion stem from every successful selected separator. Each result retains its own source hash, separator, role and timeline lane. Master JSON filenames remain compatible; percussion files have distinct source-qualified IDs. Failed sources are recorded separately, and no guessed percussion stem is substituted.

All-In-One structure remains master-based, and Beat Transformer runs once on its actual five demixed streams—not repeatedly on filenames it does not consume. Consensus allows one vote per detector family, not one per stem, preserving all source hypotheses without letting repeated drum runs dominate the vote. The arcadians-core-v2 publication gate requires the new, correctly attributed master/drum beat results. See routing, provenance and failure handling.

Timeline overlays and movement display modes

Master/stem audio appears as one channel per source, with Waveform / Spectrogram / Both in the left heading. Both overlays the waveform on the time-aligned spectrum. Each channel has a height slider and reset: wheel pans time, Ctrl/Command + wheel zooms time, Alt + wheel resizes the hovered channel, and Shift + wheel scrolls between channels. These are display-only changes.

The shared DanceRudiments Movement view selector switches between the existing X–Y trajectory and Bar graph, one labelled signed bar per stored dimension. It retains the original native sampler, audio clock and review state. BeatNet/BeatThis still analyse both the master and every drums/percussion stem after separation; the master pass is never replaced.

See viewer controls and the released-viewer workflow. Release both companion packages first; StemLab resolves the latest stable official npm/PyPI releases and rebuilds from those distributions. Development pins and local vendor archives are not production dependencies. Existing exported pages must be regenerated.

Coordinated releases and latest-release dependencies

Keep DanceRudiments, react-timeline-sequence, and stemlab as sibling checkouts. The release order is React Timeline Sequence → DanceRudiments → StemLab: publish and verify the companions before updating the consuming app. StemLab must use their latest stable official releases, not an unreleased commit, sibling source package or vendored npm archive. Each build records exact released versions/commits/integrity after latest-release resolution so all its jobs use the same bytes; CI fails clearly when that snapshot is stale.

From the parent folder, preview and then explicitly execute:

.\stemlab\scripts\Release-DanceFlow.ps1 -ProjectsRoot .
.\stemlab\scripts\Release-DanceFlow.ps1 -ProjectsRoot . -Execute

For the first migration use the same commands with .\release-kit\Release-DanceFlow.ps1 after extracting the complete kit. It applies the three repository patches, updates versions, performs scoped commits and normal pushes, waits for exact-commit CI, then creates new release tags. StemLab’s CLI, both plugin manifests and MCP registry metadata are versioned together. The existing GitHub Actions workflows publish packages; no protection rules, credentials, old tags or assets are changed. The initial expected next-minor versions are only a preview proposal, never a claim that they are published.

Use Node.js 24, Python 3.11+, PowerShell 5.1+, Git and authenticated GitHub CLI. Required main/registry/Pages jobs must pass; missing approvals and partial publication are blockers, not success. The default script releases all three projects, is opt-in via -Execute, and supports a saved exact-source -Resume session. A changed checkout or pre-commit build failure requires review rather than an automatic stash/reset or guessed recovery.

StemLab’s helper updates npm manifests/lock, optional native dependency minimum, release/build metadata and the genuinely rebuilt JS/CSS together. Its daily release-update workflow proposes subsequent updates by PR without auto-merging. A tagged StemLab release generates all five Arcadians pages and validates actual Pages deployment, live HTML and seekable audio. Do not start a duplicate manual Arcadians analysis during that tagged build. Read the full release, resume, verification and troubleshooting guide before running it. Keep this README and AGENTS.md synchronized with future changes.

Standalone Linux CLI release gate

The Linux one-file stemlab-linux-x86_64 executable is a required release artifact, not a best-effort extra. It is packaged with PyInstaller, so normal Python/native modules are bundled rather than translating the complete ML stack into C. Its dedicated Standalone CLI workflow uses Python 3.11 so the maintained madmom-prebuilt decoder and StemLab’s checksum-verified BeatNet 1.1.3 runtime can coexist. The build bootstraps the official BeatNet wheel, verifies its SHA-256, embeds the BeatNet code and model weights plus required package data/metadata, then smoke-tests the actual executable with an empty external StemLab cache. --help, models, and doctor must work, and doctor must report embedded BeatNet as ready.

Main-branch release coordination requires binary.yml to pass before tagging. Tagged releases call the same reusable workflow and cannot publish a fallback STEMLAB-BINARY-NOTE.txt; absence or failure of the executable fails the release.

Editable song timing: fixed/variable BPM, lyrics, words and cue sections

The Stemlab browser workspace has Song timing alongside Beat correction. It uses the shared react-timeline-sequence 0.4+ timing lanes, not a second playhead or bespoke renderer. Its three editable tracks are per-line lyric blocks, words belonging to the selected line, and cue section boundaries. The same original master audio, transport and zoom apply to all three.

The timing clock is either an explicit fixed BPM + Beat 1 offset or the current SHA-bound corrected variable BPM timeline. Variable mapping integrates the exact existing consensus/corrected sample frames, and never inserts imaginary beats into missing spans. Switching clock mode does not move cue, line, or word timestamps. An optional 150-ms snap tolerance (editable) applies only to edits that the user makes; it is disabled by default. Timing tooltips show whether and by how many signed milliseconds a marker misses a real or fixed-grid beat. A changed beat map must be saved before saving timing edits bound to that variable map.

Editable word starts/ends are constrained within the selected lyric line. Dragging a complete line carries its words; changing a line boundary never silently clips words. The editor includes add-at-playhead, delete, section rename, line/word text, import, artist review status and save/reload controls. Cue renames are restricted to [TYPE: STANDARD SECTION]: the supported types are SECTION, ENTRY, DROP, BREAK. Names use the recognised INTRO, VERSE, PRE-CHORUS, CHORUS, BUILD, DROP, FINAL DROP, BREAKDOWN, OUTRO, etc., with optional numbering. Examples:

[00:00.125][SECTION: INTRO]
[00:57.460][SECTION: VERSE 1]
[01:25.000][ENTRY: VOCAL INTRO]
[01:44.010][DROP: CHORUS 1]
[02:34.440][BREAK: SPOKEN BREAKDOWN]

The edit document is stored separately as timing/song-timing-edits.json, schema stemlab.timing-edits.v1: source SHA-256, sample rate/frame count, fixed_clock, clock mode, revision-pinned beat map, lines/words/cues with integer native source frames, and artist review status. Original audio, canonical.json, ASR evidence, source LRC and beat detector results remain unchanged. The web service supports:

  • GET/PUT /api/{sha256}/song-timing (optimistic revision guard),
  • POST /api/{sha256}/song-timing/import (LRC lyric lines, LRC/TSV/JSON words, canonical cue-section LRC; importing lines with existing words is refused),
  • POST /api/{sha256}/song-timing/preview (strict validation and export without saving).

Export DanceMoves timing bundle provides a JSON source identity and fixed or full variable BPM map alongside line-LRC, per-word TSV and JSON, and cue-LRC content. For DanceMoves’ separate full beat-map field, use the original Beat correction editor’s stemlab.beat-timeline.v1 export (or the bundle’s nested clock.beat_timeline), not the enclosing song-timing bundle JSON. Word timings are kept in original source seconds; LRC/TSV representations round to compatible decimal positions while the JSON retains exact native frames. Uploaded word and cue files are independently validated by DanceMoves; pasting a full variable timeline into the old fixed beat-alignment field is not supported. Users must assign each file to the matching verified song.

For a fresh checkout, release react-timeline-sequence v0.4+ first. Stemlab only consumes the latest published stable shared component, never an untagged source copy. After publication run python scripts/configure_viewer_build.py (which updates official viewer receipts and compiled assets), then run:

python -m pytest tests/test_timing_edits.py tests/test_beat_map.py
node --test tests/web/timing-workflow.test.mjs tests/web/beat-map.test.mjs
python scripts/check_viewer_build.py --latest
npm ci && npm run build:web
python scripts/check_viewer_build.py

Commit the released npm/lock/version receipt, generated src/stemlab/web/app.js and timeline.css, source, docs and tests together; never commit patch payloads, model weights or local development dependency overrides.

Windows pytest temporary-directory permissions

StemLab configures a writable, repository-local .pytest-tmp/ directory for pytest’s tmp_path and tmp_path_factory fixtures. This avoids Windows PermissionError: [WinError 5] Access is denied failures caused by an inaccessible, stale %TEMP%\pytest-of- directory. The cache is already excluded by .gitignore; it is never packaged or committed. Pytest creates separate numbered directories for concurrent runs and retains only its normal recent sessions. The fix applies to pytest and python -m pytest, including runs launched from subdirectories, without requiring administrator privileges or deleting the system’s existing temporary files.

Run tests as usual:

python -m pytest tests/test_timing_edits.py tests/test_beat_map.py
python -m pytest

To override the default for a particular run, pass --basetemp=PATH or set PYTEST_DEBUG_TEMPROOT before invoking pytest. Explicit overrides are not changed. Avoid choosing a directory shared with unrelated processes for --basetemp, because pytest intentionally clears that directory at the start of a run. The repository-local numbered root does not have that problem.

View this README on GitHub

Recommended Tools

Try a different keyword or remove a filter.

Install

npx skillfish add kieransimkin/stemlab