这是一份构建完全本地AI驱动的编程环境的完整指南，能在您的硬件上运行——此外还可选为较重模型和企业级RAG配备云GPU服务器。

这是什么

一个跨越两台机器的生产准备 AI 编码配置：

本地（RTX 5090,32GB）— 更新于2026年4月

• Qwen3.6-35B-A3B（主层）— MoE（3B活跃，8个路由+1个共享专家，共256个）。混合型门控DeltaNet+门控注意力架构（30个线性层+10个全注意力层）。原生262K上下文，可扩展至1M。~177 t/s生成，采用TurboQuant turbo3。原生工具调用+开发者角色支持（Codex/OpenCode）。
• Qwen3.5-35B-A3B（遗留）— MoE（3B活跃），188 t/s生成。
• Gemma 4 31B — 稠密（31B激活），生成速度61 t/s。单次任务的最佳质量/推理。

云端（RTX PRO 6000,96GB）

• Qwen3.5-122B-A10B — MoE（10B激活），Unsloth UD-Q4_K_XL（72GB）+ TurboQuant turbo3 KV压缩。比35B版本更优的基准测试、工具使用和推理。Unsloth最新的GGUF包含固定的聊天模板用于工具调用/编码。
• AnythingLLM — 包含工作区、RBAC和代理技能的RAG知识库

Local AI Coding Pipeline — RTX 5090 + Cloud GPU + Qwen3.6 + TurboQuant

A complete guide to building a fully local AI-powered coding environment that runs on your hardware — plus an optional cloud GPU server for heavier models and enterprise RAG.

What This Is

A production-ready AI coding setup spanning two machines:

Local (RTX 5090, 32GB) — Updated April 2026

• Qwen3.6-35B-A3B (primary) — MoE (3B active, 8 routed + 1 shared experts, 256 total). Hybrid Gated DeltaNet + Gated Attention architecture (30 linear + 10 full attention layers). Native 262K context, extensible to 1M. ~177 t/s generation with TurboQuant turbo3. Native tool calling + developer role support (Codex/OpenCode).
• Qwen3.5-35B-A3B (legacy) — MoE (3B active), 188 t/s generation.
• Gemma 4 31B — Dense (31B active), 61 t/s generation. Best quality/reasoning for single-shot tasks.

Cloud (RTX PRO 6000, 96GB)

• Qwen3.5-122B-A10B — MoE (10B active), Unsloth UD-Q4_K_XL (72GB) + TurboQuant turbo3 KV compression. Better benchmarks, tool use, and reasoning than the 35B variant. Unsloth’s latest GGUFs include fixed chat templates for tool-calling/coding.
• AnythingLLM — RAG knowledge base with workspaces, RBAC, and agent skills
• Mistral OCR MCP — PDF text extraction via Mistral API as an MCP tool for AnythingLLM agents
• Extract API — PDF → OCR → LLM extraction → clean structured-JSON pipeline

Both powered by llama-server with Google’s TurboQuant KV cache compression. The local RTX 5090 uses TheTom’s fork (has LLM_ARCH_QWEN3NEXT for Gated DeltaNet support — required for Qwen3.6); the cloud server uses Madreag’s fork (+13-69% faster decode at 32K on SM120 for non-hybrid models).

Tools:

• OpenCode — Agentic coding with file editing, bash, grep, and MCP tools
• Claude Code — Anthropic’s CLI (works with local models via Ollama API)
• OpenClaw — Telegram bot bridge to your local model
• Open WebUI — Chat interface in the browser
• AnythingLLM — Enterprise RAG platform (cloud server)

---

Quick Start

# One-command setup (WSL Kali, WSL Ubuntu, or native Linux)
bash scripts/setup.sh

This installs CUDA, builds llama.cpp (mainline + TurboQuant), installs OpenCode, and generates config files. See scripts/setup.sh for details.

Manual quick start:

# Start the server (pick your model)
./scripts/start-server.sh qwen  YOUR_MODEL_PATH/Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf
./scripts/start-server.sh gemma YOUR_MODEL_PATH/gemma-4-31b-it-UD-Q4_K_XL.gguf
./scripts/start-server.sh turbo YOUR_MODEL_PATH/Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf

# Start coding
cd your-project && opencode

---

Hardware & Models

| Component | Details |