Skill Conductor

A skill that creates, evaluates, and improves other skills. Meta-level.

Architecture-first skill lifecycle: design → build → test → evaluate → package.

Most skill tools jump straight to “write SKILL.md.” Conductor makes you choose the architecture first — because rewriting a wrong pattern costs more than writing it right.

Install

# skills.sh — installs into ~/.claude/skills
npx skills add smixs/skill-conductor

# Claude Code plugin
/plugin marketplace add smixs/skill-conductor
/plugin install skill-conductor@smixs

Synthesized from

1. Anthropic Skill Creator — eval infrastructure, grader/comparator/analyzer agents, benchmark pipeline
2. The Complete Guide to Building Skills for Claude — architecture patterns, success metrics
3. Superpowers / writing-skills by Jesse Vincent — TDD approach, the “description trap” discovery, match-the-form-to-the-failure, the micro-test protocol, pressure scenarios and rationalization tables
4. Skills Best Practices by Minko Gechev — three-stage LLM validation, eval methodology
5. hamelsmu/evals-skills by Hamel Husain — critique-before-verdict judge outputs, borderline few-shot examples, judge calibration discipline
6. grafana/skills — skill-authoring — the pushy description pattern in production, judge score-variance discipline (“three consecutive local passes before shipping”)
7. softaworks/agent-toolkit — skill-judge — the Expert/Activation/Redundant knowledge-delta taxonomy, directive loading triggers, the freedom-consequence test
8. neolabhq/context-engineering-kit — threshold-blind judges (never tell the judge the bar)
9. trailofbits/skills — skill-improver — the stop-hook pattern for unattended improvement loops (referenced, not implemented)

Methodology foundations

• Standard Operating Procedures: A Writing Guide — Richard Stup, Penn State Extension. Format selection, hierarchical vs. flowchart procedures.
• Procedure Writing: Principles and Practices — Wieringa, Moore & Barnes (Battelle Press, 1998). Imperative steps, removing modal weasel-words.
• Toyota TWI (Training Within Industry) — the “Job Instruction” method: step → key point → why; the 5 Whys root-cause practice (Job Methods manual, The Roots of Lean).
• McDonald’s Operations Manual — the canonical 600+ page SOP system; checklists at the point of use. The manual itself is proprietary; it is documented in John F. Love’s McDonald’s: Behind the Arches.
• Ask, Don’t Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement — the BinEval method behind Conductor’s evaluation.
• Guardrails Beat Guidance (5000+ Claude Code runs on SWE-bench) — rule polarity: helpful rules are negative constraints, harmful ones positive directives → Principle #10.
• TICK: Generated Checklists Improve LLM Evaluation and Generation — checklist as spec + eval + feedback; refinement plateaus and degrades past 3–4 iterations → the 3-iteration cap.
• CheckEval and Prosa — binary decomposition makes judges reproducible across model families → cross-family judge calibration.
• Self-Preference Bias in Rubric-Based Evaluation — judges favor their own family even on binary rubrics → the out-of-family acceptance rule.
• LLMs Cannot Self-Correct Reasoning Yet and BIG-Bench Mistake — self-correction needs an external gate; models fix errors well only when an external checker locates them → the gated self-update loop.
• SkillJuror — progressive disclosure with explicit loading triggers beats both flat files and passive reference lists → directive loading rules.
• SkillReducer — >60% of public skill-body text changes no agent behavior → the actionability test in Principle #9.
• IFScale and Prompt Design at Scale — compliance collapses near 80 simultaneous rules; format matters less than rule count → the rule budget and MOC structure.

6 Modes

Architecture patterns

Choose before writing a single line:

Eval infrastructure

                    ┌─────────┐
                    │  SKILL  │
                    └────┬────┘
                         │
              ┌──────────┼──────────┐
              │          │          │
         ┌────▼────┐ ┌──▼───┐ ┌───▼────┐
         │ Grader  │ │ A/B  │ │Analyzer│
         │         │ │Blind │ │        │
         │assertions│ │compare│ │root    │
         │+ claims │ │      │ │cause   │
         └─────────┘ └──────┘ └────────┘
              │          │          │
              └──────────┼──────────┘
                         │
                   ┌─────▼─────┐
                   │ Benchmark │
                   │ mean±std  │
                   └───────────┘

Quality is scored with BinEval: binary yes/no questions per dimension, each answered with evidence; the skill passes when every critical question answers yes — not when a scalar clears a threshold.

Installation layout

skills/
└── skill-conductor/
    ├── SKILL.md
    ├── agents/
    │   ├── grader.md
    │   ├── comparator.md
    │   ├── analyzer.md
    │   └── bineval.md
    ├── eval-viewer/
    │   ├── generate_review.py
    │   └── viewer.html
    ├── references/
    │   ├── patterns.md
    │   ├── schemas.md
    │   ├── sop-practices.md
    │   ├── bineval-method.md
    │   ├── quality-questions.md
    │   ├── pressure-testing.md
    │   └── runtime-setup.md
    ├── assets/
    │   └── eval_review.html
    └── scripts/
        ├── init_skill.py
        ├── eval_skill.py
        ├── run_eval.py
        ├── run_loop.py
        ├── improve_description.py
        ├── aggregate_benchmark.py
        ├── generate_report.py
        ├── package_skill.py
        ├── quick_validate.py
        ├── test_smoke.py
        └── utils.py

Claude Code: the install commands above drop it into .claude/skills/. Auto-activates when the agent detects a skill-building task.

Key discovery

Never put process steps in the skill description. If your description says “exports assets, generates specs, creates tasks” — the model follows the description and skips the body. Tested experimentally.

# ✅ Good
description: Analyze design files for developer handoff. Use when user uploads .fig files.

# ❌ Bad - model follows this and ignores SKILL.md body
description: Exports Figma assets, generates specs, creates Linear tasks, posts to Slack.

License

MIT — see LICENSE.