bbuf/ai-infra-auto-driven-skills

This repository is built for AI infrastructure engineers who want agents to do real work, not recite generic prompts.

It gives an agent the operational memory needed to benchmark SGLang, vLLM, TensorRT-LLM, and TokenSpeed fairly; turn a new SGLang model architecture into an auditable Day-0 support and release plan; explain serving capacity from startup logs; split prefill and decode profiler evidence; inspect traces at layer and kernel level; estimate operator FLOPs and MFU; review SGLang patches against real maintainer discussion patterns; run Humanize-governed SGLang and vLLM SOTA loops; triage SGLang production incidents from a replay; and keep model-family optimization history close to the code that actually changed.

For standalone kernel campaigns and kernel evidence tools, see the sibling project KDA-Pilot.

For model-by-model image, video, audio, and 3D optimization runs, see the Markdown-only SGLang Diffusion optimization flow library. Every model and component flow is self-contained and intentionally repeats its environment, download, benchmark, profiling, optimization, and acceptance instructions.

If this saves you one stale model-support assumption, one misleading profiler trace, or one late-night benchmark loop, a star helps more AI-infra engineers find it.

Core Skills

SGLang SOTA Performance Loop

sglang-sota-humanize-loop always patches SGLang, while the competitor set is caller-controlled. By default the comparison framework set can include vLLM, TensorRT-LLM, and TokenSpeed; a prompt can also narrow it, for example:

Use sglang-sota-humanize-loop for .
comparison_frameworks: [vllm]
Do not consider TensorRT-LLM or TokenSpeed; record them as user-excluded.

Model PR History Catalog

The model optimization layer is now one knowledge base: model-pr-optimization-history. It contains 66 PR-driven history dossiers and a small query helper. These are not per-model runbook skills; they preserve diff-backed model evolution records for SGLang, vLLM, TensorRT-LLM, and TokenSpeed so SOTA loops can read prior source and PR evidence before patching.

Covered families include:

DeepSeek V3/R1/V3.1/V3.2/V4, Qwen3, Qwen3-Coder, Qwen3-Next,
Qwen3.5/Qwen3.6, Qwen VLM/Omni/ASR, GLM 4.5/4.6/4.7/5,
Kimi, MiniMax, Llama 4, Mistral Small 4, Mixtral, Nemotron,
Gemma, Ernie 4.5, Intern-S1, InternVL, Hunyuan, MOSS-VL,
GPT-OSS, Step 3.5, Mimo, and model-specific MoE/quantization paths.

Each model-family history is designed to answer practical questions:

• Which PRs changed this model path?
• Was the PR merged, closed, or still open?
• Which files and symbols moved?
• What optimization or correctness risk should be checked before touching it?
• Which upstream idea should be compared before writing a new kernel or fusion?

Query examples:

cd model-pr-optimization-history
python3 scripts/query.py --list
python3 scripts/query.py --framework sglang --model qwen3-core --paths-only
python3 scripts/query.py --framework vllm "qwen3 fused qk norm"
python3 scripts/query.py --framework tokenspeed --model qwen35 qk rmsnorm

Open PR freshness is tracked separately from merged history cards:

python3 tools/check_open_pr_watch.py --format markdown \
  --output model-pr-optimization-history/open-pr-watch.md

This report uses the GitHub pulls API with an anonymous REST fallback when gh api is rate-limited. If every repo fetch fails, the tool exits non-zero instead of writing a misleading empty report.

Evidence Standards

The repo is opinionated about evidence because performance work gets noisy fast.

• Benchmark rows should include model, framework, GPU count, workload, request rate or concurrency, SLA status, launch command, benchmark command, and raw artifacts.
• Profiler reports should keep prefill and decode separate, then emit the same three tables: kernel table, overlap-opportunity table, and fuse-opportunity table.
• SOTA claims should be scoped to the exact model, hardware, framework commits, precision, workload, and SLA used in the run.
• SGLang human review should use the full PR episode corpus: inline review threads for line-local findings, PR conversations for design/test/repro negotiation, and review submissions for blocking maintainer summaries.
• Humanize SOTA loops should keep only the fixed fair benchmark outside the patch loop; gap decisions, profiler triage, required layer-pipeline deep dives, kernel evidence, target-framework code changes, and revalidation all stay inside one model-level RLCR loop.
• Kernel-local fixes inside that loop should use ncu-report-skill when Nsight Compute counter evidence is needed, store NCU digests, and still pass the same real-model benchmark/profile gate.
• Incident triage should start from replayable evidence instead of changing code from symptoms alone.
• Model optimization histories should point back to PRs, files, diffs, and risk surfaces rather than vague summary text; they live as one PR-driven knowledge base, not per-model skills.
• Root-level update_prompt.md captures the full refresh and validation workflow for updating this repo again without relying on memory from a previous run.

Install

This repository is not Codex-only. The skills are plain SKILL.md directories and can be installed into Claude Code, Codex, Kimi, or another compatible agent runtime.

Claude Code (one-shot plugin install)

The repository ships a .claude-plugin/ manifest so the whole skill set can be installed as a single Claude Code plugin via the built-in marketplace flow:

/plugin marketplace add BBuf/AI-Infra-Auto-Driven-SKILLS
/plugin install ai-infra-auto-driven-skills@ai-infra-auto-driven-skills
/reload-plugins

After reload, the 11 skills appear namespaced as ai-infra-auto-driven-skills: (for example ai-infra-auto-driven-skills:sglang-sota-humanize-loop). Update later with /plugin marketplace update ai-infra-auto-driven-skills.

Claude Code (per-skill symlink, legacy)

Prefer this when you only want a subset of the skills, or when developing against a local checkout. Symlink is recommended for local development because updates to this checkout are picked up immediately:

git clone https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS.git
cd AI-Infra-Auto-Driven-SKILLS

mkdir -p ~/.claude/skills
ln -s "$PWD/skills/llm-serving-auto-benchmark" ~/.claude/skills/llm-serving-auto-benchmark
ln -s "$PWD/skills/llm-serving-capacity-planner" ~/.claude/skills/llm-serving-capacity-planner
ln -s "$PWD/skills/llm-torch-profiler-analysis" ~/.claude/skills/llm-torch-profiler-analysis
ln -s "$PWD/skills/llm-pipeline-analysis" ~/.claude/skills/llm-pipeline-analysis
ln -s "$PWD/skills/model-compute-simulation" ~/.claude/skills/model-compute-simulation
ln -s "$PWD/skills/model-optimization/sglang-model-day0-support" ~/.claude/skills/sglang-model-day0-support
ln -s "$PWD/skills/sglang-humanize-review" ~/.claude/skills/sglang-humanize-review
ln -s "$PWD/skills/sglang-sota-humanize-loop" ~/.claude/skills/sglang-sota-humanize-loop
ln -s "$PWD/skills/vllm-sota-humanize-loop" ~/.claude/skills/vllm-sota-humanize-loop
ln -s "$PWD/skills/sglang-prod-incident-triage" ~/.claude/skills/sglang-prod-incident-triage
ln -s "$PWD/skills/model-architecture-diagram" ~/.claude/skills/model-architecture-diagram
ln -s "$PWD/model-pr-optimization-history" ~/.claude/skills/model-pr-history-knowledge

Restart Claude Code after installing. The skills can then be invoked by name, for example [$llm-serving-auto-benchmark], [$llm-serving-capacity-planner], [$llm-torch-profiler-analysis], [$llm-pipeline-analysis], [$model-compute-simulation], [$sglang-model-day0-support], [$sglang-humanize-review], [$sglang-sota-humanize-loop], or [$vllm-sota-humanize-loop].

If you prefer copies instead of symlinks, replace ln -s with cp -R. Copy model-pr-optimization-history only when you want the agent to query the PR-driven model knowledge base locally. It replaces the old per-model runbook skill layout with one shared knowledge root.

Generic Agent Skill Directory

For Codex, Kimi, or another compatible runtime, copy or symlink the same directories into that runtime’s skill directory:

cp -R skills/llm-serving-auto-benchmark /llm-serving-auto-benchmark
cp -R skills/llm-serving-capacity-planner /llm-serving-capacity-planner
cp -R skills/llm-torch-profiler-analysis /llm-torch-profiler-analysis
cp -R skills/llm-pipeline-analysis /llm-pipeline-analysis
cp -R skills/model-compute-simulation /model-compute-simulation
cp -R skills/model-optimization/sglang-model-day0-support /sglang-model-day0-support
cp -R skills/sglang-humanize-review /sglang-humanize-review
cp -R skills/sglang-sota-humanize-loop /sglang-sota-humanize-loop
cp -R skills/vllm-sota-humanize-loop /vllm-sota-humanize-loop
cp -R skills/sglang-prod-incident-triage /sglang-prod-incident-triage
cp -R skills/model-architecture-diagram /model-architecture-diagram
cp -R model-pr-optimization-history /model-pr-history-knowledge

How I Drive The Agents

These skills are exercised with coding agents in full-autonomy mode. For reproducibility, here is exactly how I launch them.

Claude Code — use the current Opus alias with Auto permission mode:

claude --model opus --permission-mode auto

opus tracks Claude Code’s current Opus model instead of pinning a model number. Auto mode can approve routine work while retaining the permission system. bypassPermissions (or --dangerously-skip-permissions) should be reserved for an isolated sandbox, ideally without internet access.

Codex — full-access, no approval prompts:

codex --sandbox danger-full-access --ask-for-approval never

The Codex command is intentionally unsandboxed and should likewise be used only inside an isolated environment with a scoped checkout and its own benchmark and correctness gates.

Repository Map

skills/
├── llm-serving-auto-benchmark/      # serving benchmark search and comparison
├── llm-serving-capacity-planner/     # startup memory and request capacity analysis
├── llm-torch-profiler-analysis/     # profiler capture and trace triage
├── llm-pipeline-analysis/           # forward/layer/kernel trace analysis
├── model-compute-simulation/        # operator FLOPs, tensor shapes, and MFU
├── sglang-humanize-review/          # human SGLang PR review corpus and workflow
├── sglang-sota-humanize-loop/       # Humanize-governed SGLang SOTA loop
├── vllm-sota-humanize-loop/         # Humanize-governed vLLM SOTA loop
├── sglang-prod-incident-triage/     # replay-first serving incident workflow
├── model-architecture-diagram/      # public architecture diagram resolver
└── model-optimization/
    └── sglang-model-day0-support/   # model Day-0 PR and release gates

model-pr-optimization-history/
├── SKILL.md                         # knowledge-base usage instructions
├── scripts/query.py                 # local model/keyword query helper
├── sglang/                          # 31 PR-driven SGLang model histories
├── vllm/                            # 31 PR-driven vLLM model histories
├── tensorrt_llm/                    # TensorRT-LLM competitor histories
└── tokenspeed/                      # TokenSpeed competitor histories

prompts/
├── sglang-sota-b200-prompts.md       # B200 SGLang SOTA task prompts
├── sglang-sota-b200-codex-goal-prompts.md
├── sglang-sota-h200-prompts.md       # H200 SGLang SOTA task prompts
└── sglang-sota-h200-codex-goal-prompts.md

Related Projects

• Humanize provides the RLCR workflow that powers the Humanize-governed SGLang and vLLM SOTA loops.
• KDA-Pilot is the sibling home for standalone kernel loops, kernel knowledge, and NCU report workflows.

Star History