westwaters/pollard-weights

边境型号。小型硬件。绝不妥协。

下载前就知道你的硬件能运行什么——然后构建一个适合它的模型，并发布到任何运行时。

Pollard权重是为目标设备的内存设计的模型，而非为 位宽图。 注意，路由器和嵌入保持高精度;专家级 FFN负责压缩;热层在给建造者供料时保留更多位 一个测量的路由配置文件——整个构建的大小取决于任何内存/显存 你把它指向，但没有一个可用的储备。

任何目标，任何设备——没有固定的大小范围。告诉它目标的 预算（--ram/--vram），而且它适合那个*——手机或树莓派（数百MB）， 一台16GB的Mac Mini，一台24GB的5090，一台256GB服务器，一台512GB的晶圆节点。基于一台构建 机器，部署到另一个平台——你构建的大磁盘盒通常不是所在 模型运行，所以--ram是目标的预算，不是构建机器的 （[--ram auto 仅在相同时使用）。下面16GB的示例仅为 普通的拆除，不是天花板。

一个测量分配，五条输出通道 — GGUF（llama.cpp/Ollama/LM Studio），

Pollard Weights are models built for a target device’s memory, not for a bit-width chart. Attention, routers and embeddings keep high precision; expert FFNs carry the compression; hot layers keep more bits when you feed the builder a measured routing profile — and the whole build is sized to whatever RAM/VRAM you point it at, minus a working reserve.

Any target, any device — there is no fixed size range. Tell it the target’s budget (--ram/--vram) and it fits that — a phone or a Pi (hundreds of MB), a 16GB Mac Mini, a 24GB 5090, a 256GB server, a 512GB wafer node. Build on one machine, deploy to another — the big-disk box you build on is usually not where the model runs, so --ram is the target’s budget, not the build machine’s (--ram auto only when they’re the same). The 16GB examples below are just a common demo, not a ceiling.

One measured allocation, seven output lanes — GGUF (llama.cpp/Ollama/LM Studio; trellis flagship on ik_llama.cpp), MLX (Apple), GPTQ (vLLM/SGLang), MX (NVFP4 / MXFP4 / FP8 / MXFP6 and measured mixes, Blackwell / Vera Rubin), EXL3 (exllamav3), EXLR8 (one trellis encode, every rung a manifest), and DRAFT (a speculative-decoding drafter matched to the quantized target) — pick the runtime, keep the allocation. Every lane ships the whole model (vision / audio towers included), and every lane lands in one repo per model.

How it works

flowchart LR
    HF["FP16 / BF16model"] -->|"optional: smooth · abliterate(pollard-hf-smooth / -abliterate)"| ALLOC
    ALLOC["Pollard measured allocation(sensitivity → KL knapsack)"] --> GGUF["GGUF · llama.cpp"]
    ALLOC --> GPTQ["GPTQ · vLLM / SGLang"]
    ALLOC --> MLX["MLX · Apple Silicon"]
    ALLOC --> EXL3["EXL3 · exllamav3"]
    ALLOC --> MX["MX · NVFP4 / MXFP4 / FP8 / MXFP6Blackwell / Rubin"]
    ALLOC --> EXLR8["EXLR8 · GR8 / reference decode"]
    ALLOC --> DRAFT["DRAFT · speculative drafterllama.cpp / vLLM / SGLang / MLX"]
    GGUF --> V["pollard-verify(real-reconstruction gate)"]
    GPTQ --> V
    MLX --> V
    EXL3 --> V
    MX --> V