Voodoo Dynamic Quantization

Learned per-tensor mixed-precision quantization for any LLM — llama.cpp-exact by construction.

Voodoo replaces hand-tuned quantization recipes with gradient descent: every weight tensor in the model votes, through its own gates, on how many bits it deserves, under a global size budget and full-model distillation loss. The result is a per-tensor quant assignment exported as a normal GGUF that runs in stock llama.cpp. Voodoo’s methodology reduces perplexity by up to 89.5%.

   teacher (frozen, BF16/Q8)          student = same model, weights replaced by
   ─────────────────────────          Σ pₖ · Wₖ   (softmax over K quant candidates)
        │ logits                              │ logits
        └──────────────┬──────────────────────┘
                       ▼
     L = CE + λ·KL(teacher ‖ student) + size-budget dead-zone penalty
                       ▼
     argmax gate per tensor → quant assignment → GGUF (llama.cpp-exact)

• Architecture-agnostic. If transformers can load it, Voodoo can quantize it — attention, SSM/linear-attention hybrids, gated-shortconv hybrids (LFM2/LFM2.5), tied embeddings, MoE-free any-shape Linears (zero-padded internally). Arch specifics (TP sharding policy, GGUF tensor mapping) live in small, detached adapters under voodoo_quant/arch/.
• Hardware-agnostic. CUDA or ROCm, single GPU, layer-wise sharding, or full tensor parallelism. Vendor specifics live in one tidy module.
• Exact. Every quantized byte is produced by llama.cpp’s own C++ quantizer via libggml-base — no train/deploy quantization gap, ever.

How it works: WHITEPAPER.md · What to do: the skills below · Agent conventions: AGENTS.md

Is your organization looking for help?

I’m available for hire. Reach out at https://voodooquant.com.

---

Quickstart

make bootstrap        # venv + package + llama.cpp libggml (nice defaults)
make doctor           # verify GPU + torch + libggml resolve
voodoo data --model Qwen/Qwen3.5-0.8B-Base --output_dir data/qwen35-0.8b
configs/qwen35_08b_voodoo45.sh    # single-GPU train → Voodoo45

make bootstrap clones llama.cpp into third_party/ and builds libggml-base (CPU-only build — no GPU toolchain needed for quantization). Already have a llama.cpp checkout? Set VOODOO_GGML_LIB=/path/to/libggml-base.so and skip that step.

---

Using this repo with an agent

The workflow is encoded as skills in skills/ — install, data, train, export, eval. If you use an agentic coding tool (Claude Code, Kimi, Cursor, etc.), point it at this repo and it will drive the pipeline through the skills; base instructions live in AGENTS.md.