kanaworksai-hue/audio-studio-mcp

539 stars 40 forks Quality 83 Trend 67MIT

Local AI BGM and sound effects for Claude Code and any MCP client. Stable Audio 3 on Apple Silicon (MLX) and Windows NVIDIA GPUs (PyTorch). No cloud, no API key.

Overview

gives Claude Code, Claude Desktop, Cursor and any other MCP client a sound studio that runs on your own Mac or Windows PC. Ask for background music or a sound effect in plain language; the model generates it locally with Stable Audio 3. Then it can loop it, normalize the loudness and export it for your game engine or video editor. There is no cloud service and no API key, and generating doesn't cost anything. A real session, shortened. Claude picked a loop length of exactly 11 bars so the beat stays in time when it repeats. - every file gets a .json sidecar with its prompt, seed and settings. - the server teaches the client the official Stable Audio 3 prompt format, so a request like "make it sound like a creaky door" becomes a prompt the model understands. - on a Mac, results for the same seed are bit-for-bit identical to the official sa3 CLI. On Windows it calls the official Stable Audio 3 PyTorch API.

README

Audio Studio MCP gives Claude Code, Claude Desktop, Cursor and any other MCP client a sound studio that runs on your own Mac or Windows PC. Ask for background music or a sound effect in plain language; the model generates it locally with Stable Audio 3. Then it can loop it, normalize the loudness and export it for your game engine or video editor. There is no cloud service and no API key, and generating doesn’t cost anything.

A real session, shortened. Claude picked a loop length of exactly 11 bars so the beat stays in time when it repeats.

✨ Highlights

Reproducible: every file gets a .json sidecar with its prompt, seed and settings.
Prompting built in: the server teaches the client the official Stable Audio 3 prompt format, so a request like “make it sound like a creaky door” becomes a prompt the model understands.
Same output as upstream: on a Mac, results for the same seed are bit-for-bit identical to the official sa3 CLI. On Windows it calls the official Stable Audio 3 PyTorch API.

🎧 Hear it

All of these were generated by this server on an M1 Mac and were not edited by hand. Open the sample page →

	Sample	Prompt
	Lo-fi cafemusic · 30 s · 85 BPM	Lo-fi hip-hop chill track with mellow electric piano, soft vinyl crackle, low-pass filtered drums…
	Epic orchestralmusic · 30 s	Epic cinematic orchestral trailer music with thundering taiko drums, soaring French horns…
	Chiptune adventuremusic · 20 s · 150 BPM	Upbeat 8-bit chiptune adventure theme with a bright square wave lead…
	Glass shattersfx · 3 s	source: glass bottle · action: shattering on impact · material: thick green glass · space: concrete floor
	Rain & thundersfx · 12 s	source: heavy rain and distant thunder · space: outdoor forest at night, wide stereo
	Synthwave, extendededit · 20 s → 40 s	`edit_audio` continue: a 20-second track extended to 40 seconds

MP3s are in docs/samples/, with prompts and seeds in samples.json.

🧭 How it works

flowchart LR
    U["You“make a cozy cafe BGM”"] --> C["Claude Code · DesktopCursor · any MCP client"]
    C -- "MCP" --> S["audio-studio-mcpgenerate · edit · process"]
    S --> M["Stable Audio 3MLX on Apple SiliconPyTorch on NVIDIA GPUs"]
    M --> F["WAV / OGG / FLAC + .jsonready for game & video"]

🚀 Quick start

macOS (Apple Silicon)

Requirements:

Apple Silicon Mac (M1 or later), 16 GB RAM recommended
About 6 GB of disk for the model
git and ffmpeg (brew install ffmpeg)

git clone https://github.com/kanaworksai-hue/audio-studio-mcp.git
cd audio-studio-mcp
./install.sh

The installer sets up uv if needed, clones the official Stable Audio 3 runtime into vendor/ at a tested commit, creates .venv/, and downloads the medium model (1.4B, ~5.9 GB) from Hugging Face. It finishes by printing the exact command to register the server:

claude mcp add --scope user audio-studio -- /path/to/audio-studio-mcp/.venv/bin/audio-studio-mcp

Windows (NVIDIA GPU)

Requirements:

64-bit Windows 10 or 11 with an NVIDIA GPU (8 GB of VRAM or more) and a current driver. Tested on an RTX 3060 12 GB
About 14 GB of disk for the model and PyTorch
git (winget install Git.Git) and ffmpeg (winget install Gyan.FFmpeg)
A free Hugging Face account. The PyTorch weights are gated: open stabilityai/stable-audio-3-medium and accept the license once

git clone https://github.com/kanaworksai-hue/audio-studio-mcp.git
cd audio-studio-mcp
powershell -ExecutionPolicy Bypass -File install.ps1

The installer does the same steps as on macOS, but installs PyTorch with CUDA and downloads the official PyTorch checkpoint of the medium model (~10.4 GB). If you aren’t logged in to Hugging Face yet, it asks for an access token with read permission (create one here). When it runs without a console (for example, started by an AI agent), it never asks: it prints the steps to log in and stops, and you re-run it after logging in. It finishes by printing the command to register the server:

claude mcp add --scope user audio-studio -- "C:\path\to\audio-studio-mcp\.venv\Scripts\audio-studio-mcp.exe"

Flash Attention is not needed. Without it, the runtime falls back to PyTorch’s built-in attention.

[!TIP] Let your agent install it. Paste this into Claude Code: “Install https://github.com/kanaworksai-hue/audio-studio-mcp by following its README, including the notes for AI agents, then register it as an MCP server.”

Notes for AI agents (Windows only; macOS needs no login): the weights are gated and need the user’s own Hugging Face login.

Before downloading weights, check the login with hf auth whoami (after the first installer run: .venv\Scripts\hf.exe auth whoami). install.ps1 checks it too and stops with ACTION NEEDED when nobody is logged in.

If the user is not logged in, or the download fails with 401/403 (license not accepted, or accepted with another account), stop and pass the steps the installer printed on to the user: create a free account, accept the license on each model page, then run hf auth login in their own terminal. Wait until they say they are done, then re-run the installer.

Never ask the user to paste their token into the chat, and don’t use -SkipWeights to get past this: generation would fail later with the same error.

Then just ask:

“Make a 60-second lo-fi BGM loop at 85 BPM for a cafe scene, give me 3 candidates.”
“I need a heavy wooden door slamming in a stone hallway. Export it as OGG for Godot.”
“Extend this track to 90 seconds and normalize it to −16 LUFS.”

🧰 Tools

Tool	What it does
`generate_audio`	Text to BGM, instrument loop/stem or sound effect. 1–4 candidates per call, `bpm`, structured SFX fields
`edit_audio`	`variation` (re-style the whole clip), `inpaint` (regenerate a time range), `continue` (extend). 1–4 candidates
`process_audio`	Trim → trim silence → mono → resample / loudness → seamless loop → fades → peak normalize or ceiling → WAV / OGG / FLAC. Never modifies the input
`unload_models`	Free the model memory right away

🌐 Web UI

Prefer clicking to chatting? The same server comes with a local web page. Generate BGM and sound effects, compare the candidates, edit them (make a variation, replace a part, extend), export them for your game or video, and browse everything you’ve made in the library.

.venv/bin/audio-studio-mcp --web 8765

Your browser opens http://127.0.0.1:8765/. Add --no-browser to start without opening it, or --preload to load the model at startup. Press Ctrl+C to stop it. On Windows: .venv\Scripts\audio-studio-mcp.exe --web 8765.

The page and your MCP clients can share one process and one loaded model: the same port also serves MCP at /mcp. Register it once and use both at the same time. Requests from the page and from agents wait in one queue.

claude mcp add --scope user --transport http audio-studio http://127.0.0.1:8765/mcp

Files are written to the output folder (outputs/, or AUDIO_STUDIO_OUTPUT_DIR). The library shows that folder, and the editor works on the files in it.
The page is in English, Japanese and Chinese and follows your browser’s language. You can change it in the settings.

[!WARNING] The web UI is for your own computer only. It listens on 127.0.0.1 only and has no login. Don’t make it reachable from other machines (port forwarding, a reverse proxy or a tunnel): anyone who can open it can use your GPU and write files to your output folder.

⚙️ Configuration

⚡ Performance

Medium model on an M1 Mac with 16 GB:

Task	Time
Two 3 s sound effects, cold start	~13 s
20 s of music, warm	~10 s (2× real time)
Continue to 35 s, incl. loading the encoder	~27 s
15 s of music with `sm-music`, warm	~1 s

Medium model on Windows with an RTX 3060 12 GB:

Task	Time
Two 3 s sound effects, cold start (incl. ~20 s model load)	~25 s
20 s of music, warm	~1.3 s
Continue to 30 s, warm	~2.2 s
380 s of music (the maximum), warm	~15 s
15 s of music with `sm-music`, warm	~0.7 s

On a Mac, loaded models use about 5–7 GB of memory. On the RTX 3060, the medium model peaked at about 5.3 GB of VRAM. Models are freed after the idle timeout or by unload_models.

📝 Prompting tips

❓ FAQ

🗺️ Roadmap

[x] Apple Silicon backend (Stable Audio 3, MLX)
[x] Windows backend for NVIDIA GPUs (Stable Audio 3, PyTorch)
[x] Local web UI (--web)
[ ] LoRA support (the upstream runtime already has it)

Issues and pull requests are welcome.

⭐ Like it? Star it

If Audio Studio made a sound you ended up using, please click ⭐ Star at the top of the repository page. It’s free, takes one second, and helps more than it looks:

More people find it. Stars make the project easier to discover on GitHub, so more game developers and video makers get to use it.
It keeps updates coming. Every star tells me someone finds this useful. That’s what keeps me working on the next update, like LoRA support.

Other ways to help: open an issue for a bug or an idea, or share a sound you made on X and tag @KanaWorks_AI.

🛠️ Development

uv run pytest                              # offline tests, no model needed
.venv/bin/python scripts/smoke_test.py     # end-to-end with the real model (~30 s on an M1)

On Windows: .venv\Scripts\python scripts\smoke_test.py (~45 s on an RTX 3060, including model loading).

src/audio_studio_mcp/
  server.py            MCP tools (backend-independent)
  prompts.py           prompt building and the instructions sent to clients
  post.py              post-processing and export
  backends/base.py     backend interface
  backends/sa3_mlx.py  Stable Audio 3 on Apple Silicon (MLX)
  backends/sa3_torch.py  Stable Audio 3 on NVIDIA GPUs (PyTorch)
  web.py               web UI server (--web) and its JSON API
  static/              web UI (HTML/JS/CSS, no build step)

A new platform is added as another backend in backends/.

📄 License and credits

This project: MIT.
Model weights: licensed under the Stability AI Community License. They are downloaded from Hugging Face (macOS: stable-audio-3-optimized; Windows: stable-audio-3-medium and the small models), not distributed here.
Brand: the KanaWorks_AI name, logo and the Kana character are © 東京AI映像合同会社 (Tokyo AI Visuals LLC) and are not covered by the MIT license. See docs/brand.
Fonts: the web UI ships IBM Plex Sans, © IBM Corp., under the SIL Open Font License 1.1 (OFL.txt). The Kana images in the web UI (static/brand) are covered by the Brand note above.
Built on: Stable Audio 3, MLX, PyTorch and the Model Context Protocol.

This is an independent project. It is not affiliated with or endorsed by Stability AI, Apple, NVIDIA or Anthropic.

View the full README on GitHub

Install

This server does not publish a one-line install command.

Open the repository installation guide New to MCP? Start with the mcpskill.org guide →

Configuration

{
  "mcpServers": {
    "audio-studio": {
      "command": "/path/to/audio-studio-mcp/.venv/bin/audio-studio-mcp"
    }
  }
}