Dazhuang Skill Creator

中文文档 | English | Changelog

Official original = Claude Code’s official skill-creator
My Skill Creator Copy = my second iteration
Dazhuang Skill Creator = the final version in this repo

Dazhuang Skill Creator starts from Claude Code’s official skill-creator, then rebuilds it with a stronger view of prompt architecture, skill architecture, and how CLI tool execution actually behaves in real usage.

This is not just a wording tweak. I reworked the workflow, structure, bundled resources, and maintenance model so the generated skill is easier to evolve, easier to debug, and easier to collaborate on over time.

Update v1.5.0 (2026-04-11): this major memory release introduces memory modes (off / adaptive / lessons / auto), lesson-to-hard-rule promotion, stricter memory invariants in quick_validate.py, and regression coverage for no-rehire behavior. Update v1.5.2 (2026-04-11): memory judgment is now a required step during scaffold creation; if auto is used without --intent and it currently falls to off, initialization pauses and asks for explicit intent or explicit mode. Update v1.5.3 (2026-04-11): onboarding friction is reduced. init_skill.py now prints copy-ready fixes when --path is missing, and quick_validate.py now prints copy-ready Step 4 event lines when memory commands are missing. Update v1.5.4 (2026-04-11): fixed three real flow pain points: no more default pointers to missing references/examples.md; quick_validate.py now supports --strict (and packaging uses strict checks by default); memory-judgment wording is now dynamic so explicit modes do not look like re-classification. Update v1.5.5 (2026-04-11): fixed fenced-code heading parsing in quick_validate.py; selecting --resources assets now always scaffolds assets/output-format.md; report/review/table-like tasks now get earlier assets-path guidance during init. Update v1.5.6 (2026-04-11): clarified auto-mode resource completion (lessons/adaptive always auto-add references/ + scripts/), and fixed throttled update checks so stale cached versions are no longer shown as latest. Update v1.5.7 (2026-04-11): the evaluation flow now stops to align the judging standard first. You write an “evaluation proposal” first, then a “formal evaluation plan”; without that plan, benchmark and review now stop by default. Update v1.5.8 (2026-04-11): multi-skill benchmark summaries now show every config, not just the first two; benchmark aggregation now hard-checks eval-to-dimension coverage against the evaluation plan; review also shows required report sections and dimension mapping. Update v1.5.9 (2026-04-11): fixed the big evaluation-entry bug. The first response to any skill-evaluation request must now stop at the alignment proposal; no direct scoring, no direct A/B run, and no direct benchmark before the user confirms the judging scheme.

For evaluation, I used Codex in headless mode - no GUI, no need to open the CLI page, just terminal execution - and ran at least 3 independent conversation tests per benchmark item. The full benchmark standards and archived reports are included in 测评报告/.

If this project is useful to you, please consider giving it a star. For contact or collaboration:

• WeChat: yinyinGyL
• Email: [email protected]

What Was Benchmarked

1. Five task-type capability comparisons

• A | Content creation - whether prompts, templates, and platform style can be organized into reusable skills
• B | Structured output - whether the skill can follow a strict JSON schema and keep the output stable
• C | Tool research - whether it reads source files, cites evidence, and avoids hand-wavy summaries
• D | Script execution - whether the generated scripts actually run and fail safely when needed
• E | Hybrid orchestration - whether prompt, reference, asset, and script layers work together coherently

2. Five capability archetype comparisons

• Minimal compressed output
• Strict structured output
• Safety judgment
• Template-based abstraction
• Dirty-input normalization

Evaluation Method

• Benchmarks were run with Codex headless mode in the terminal
• Each case was tested with at least 3 independent conversations
• The 3-version capability-archetype benchmark compared:
  • Claude Code official skill-creator
  • My Skill Creator Copy as the second iteration
  • Dazhuang Skill Creator as the final version
• The archived reports include:
  • 45 creation runs + 15 baselines for the 3-version benchmark
  • 30 creation runs + 15 baselines for the head-to-head task-type benchmark
• Source directory integrity checks remained clean during benchmarking (manifest diff = 0)

Evaluation Dimensions

The benchmark scoring rolls up into five top-level dimensions:

• Process efficiency
• Precision
• Product quality
• Actual-use effect
• Stability

Results

Overall conclusion

• Dazhuang Skill Creator ranks first in both benchmark sets archived in this repo
• In the 3-version capability-archetype benchmark, the final version wins with a total score of 99.43
• In the head-to-head task-type benchmark, the final version beats the official original with a total score of 99.44 vs 96.20
• The result is a clear overall win, but not a “crushing” win according to the benchmark’s own verdict rule

3-version capability-archetype benchmark

Key takeaways:

• Final version leads the runner-up by 11.59 points
• Final version achieves 100.0 downstream semantic accuracy in this benchmark set
• Final version is also the smallest of the three by average skill size (4,290 bytes vs 7,063 and 6,613)

5 task-type benchmark: final version vs official original

Additional head-to-head results:

• Total score: 99.44 vs 96.20
• Actual-use effect: 100.00 vs 98.08
• Process efficiency: 97.74 vs 89.37
• Downstream semantic accuracy: 96.76 vs 96.52
• Runtime validation: both versions scored 100.0

Why This Version Is Easier To Maintain

Compared with the original version, this repo puts more emphasis on maintainable structure:

• Keep the main SKILL.md centered on durable rules and workflow
• Keep single-file skills inside a fixed section whitelist: 角色, 规则, 工作流程, 例子, 输出格式, 索引
• Treat 例子 as model-facing internal references, not user prompt examples; treat 输出格式 as model-facing templates
• Push long explanations into references/
• Put reusable templates into assets/
• Put deterministic or repetitive work into scripts/
• Keep frequently adjusted defaults in config.yaml

That makes follow-up iteration much easier. The original version can become hard to modify once generated, while this version is designed to remain editable and team-friendly over time.

Default Strategy For Existing Skills

This repo no longer treats “optimize an existing skill” as “just tweak the description”, and it does not treat refactoring as a separate methodology either.

• Creation and refactoring follow the same blueprint; the difference is whether you start from scratch or reorganize material from an old skill
• First diagnose whether the real issue is triggering, structure, or both
• If the old skill is bloated, scattered, or easy to derail across long contexts, default to structural refactoring so it realigns with the Dazhuang architecture
• Only run trigger eval / description optimization after the skill body itself is structurally sound
• The goal is alignment with the same blueprint, not mechanically forcing every skill into the same template; simple skills can still stay single-file

Project Structure

• SKILL.md - the final Dazhuang Skill Creator skill definition
• VERSION - the local creator version marker used by the runtime update check
• agents/ - benchmark and comparison agent prompts
• references/ - architecture notes, evaluation workflow, packaging guidance, internal examples, and schemas
• assets/ - model-facing templates, reusable assets, and report templates
• scripts/ - initialization, validation, update checking, evaluation, optimization, reporting, and packaging tools
• config.yaml - editable defaults for init, update checking, evaluation, optimization, and packaging
• 测评报告/ - archived benchmark reports and screenshots

Quick Start

Install into a Claude Code / Codex / Open Claude skill directory

The recommended install path is git clone, because runtime update checks and auto-update both rely on a real git working tree.

cd 
git clone https://github.com/DazhuangJammy/DazhuangSkill-Creator.git

If you ask another AI to install this repo for you, be explicit:

• use git clone https://github.com/DazhuangJammy/DazhuangSkill-Creator.git
• do not just download a zip
• do not just copy the folder contents
• do not remove .git

Standard prompt for Claude / Codex / other installation-oriented AIs:

Please install this skill into my skill directory, and you must use git clone.

Repository:
https://github.com/DazhuangJammy/DazhuangSkill-Creator.git

Requirements:
1. Use git clone, not a zip download
2. Do not just copy the folder contents
3. Keep the .git directory intact
4. Confirm the installed directory is a normal git working tree
5. If the target directory already exists, tell me first before deciding whether to pull or reinstall

Create a new skill scaffold

On Windows, replace python3 with py -3 (preferred) or python.

One easy-to-miss point:

• --path is not optional in practice. If you do not pass --path, you must set init_skill.output_path in config.yaml first.
• For first-run success, explicitly pass --path ./out, then move it into config later if you want.

python3 scripts/init_skill.py my-skill --path ./out --memory-mode auto --intent "low-risk deterministic task"

If a single-file skill needs extra inline modules, declare them explicitly:

python3 scripts/init_skill.py my-judge-skill --path ./out --sections role,output-format --memory-mode auto --intent "boundary-heavy review task"

If you want references/examples.md to be created on day one, add --examples. Without --examples, that file is not created and the scaffold will not force a pointer to it. If you enable --resources assets, the scaffold now always creates assets/output-format.md (independent of --examples).

Memory modes:

• off: no memory layer
• lessons: enable memory pipeline from day one (memory-state + memory-events + memory-lessons)
• adaptive: start without memory, then auto-enable after repeated runtime friction
• auto (default): classify before scaffold creation and choose off / adaptive / lessons
• Resource auto-completion: if the resolved mode is lessons or adaptive, init will auto-add references/ and scripts/ even when they are not explicitly listed in --resources.

Memory-layer judgment is a required step:

• Always decide whether memory is needed, even if the final mode is off.
• Recommended default: --memory-mode auto --intent "".
• If you manually choose off, include a one-line reason (for example: low risk, low variability, deterministic flow).

These memory_* settings in config.yaml are scaffold defaults for generated skills only; they do not turn on memory for this creator repo itself.

If you want to force memory from day one, enable lessons mode:

python3 scripts/init_skill.py my-review-skill --path ./out --memory-mode lessons

If you want pre-creation auto-classification, provide intent text:

python3 scripts/init_skill.py my-analysis-skill --path ./out --memory-mode auto --intent "high-variance analysis with iterative refinement"

If you want runtime auto-enable, use adaptive mode:

python3 scripts/init_skill.py my-analysis-skill --path ./out --memory-mode adaptive

Both lessons and adaptive add scripts/memory_mode_guard.py, references/memory-state.json, and references/memory-events.jsonl.

• lessons: starts with memory enabled; repeated failure signatures are promoted into lessons.
• adaptive: starts disabled; reaches thresholds, then auto-enables lessons.
• In both modes, stable lessons are promoted into a MEMORY_HARD_RULES block inside generated SKILL.md.
• Safety rail: if the final setting ends up as memory_mode=off (via CLI or config) while auto-classification suggests memory should be enabled, initialization now blocks by default. Use --force-memory-off only when you truly want to override it.
• New rail: if you use memory_mode=auto without --intent and auto currently falls to off, initialization now pauses and asks you to either provide intent or choose a mode explicitly.

Validate a skill

python3 scripts/quick_validate.py ./out/my-skill

quick_validate.py now also enforces memory-skill invariants (when memory files are detected): MEMORY_HARD_RULES markers, Step 1/Step 4 guard commands, and required memory runtime files. For pre-release checks, use strict mode:

python3 scripts/quick_validate.py ./out/my-skill --strict

Strict mode additionally blocks TODO / TBD / placeholder markers and requires non-empty frontmatter description. package_skill.py now runs strict validation by default before packaging.

If it reports missing Step 4 retry/failure event commands, add these lines inside Step 4 of the target skill:

 "/scripts/memory_mode_guard.py" --skill-dir "" --event retry
 "/scripts/memory_mode_guard.py" --skill-dir "" --event failure

Manually check creator updates

python3 scripts/check_update.py --force

Refactor an existing skill

• The refactor still follows the same blueprint used for new skills; it just starts by extracting what is worth keeping from the old skill
• First classify the intervention level as light optimization, structural refactor, or full overhaul
• If the problem is structural bloat, path drift, or losing the main line in long contexts, refactor SKILL.md and rebalance references/, assets/, and scripts/ first
• Only move on to the trigger workflow after the structure is stable

Evaluate whether a skill actually got better

The default path is now split into two layers. Do not start by writing eval prompts immediately:

1. align the judging standard first
2. then run the execution-side evaluation

Hard gate:

• if the user just said “evaluate this skill”, “test with vs without”, or “compare these skills”, the first response must stop at the alignment proposal
• do not jump straight to scoring, A/B results, benchmark, or winner selection before the user confirms the plan

Planning entry points:

• references/eval-planning.md
• assets/evaluation-alignment-script.md
• assets/evaluation-proposal-template.md
• assets/evaluation-plan-template.md

Execution entry points:

• references/eval-loop.md
• references/schemas.md

Recommended order:

1. write the “evaluation proposal” with assets/evaluation-proposal-template.md
2. if you need a ready-made talk track, start from assets/evaluation-alignment-script.md
3. align primary direction, secondary direction, weights, dimensions, and out-of-scope items with the human
4. write the “formal evaluation plan” with assets/evaluation-plan-template.md
5. save it as evals/eval-plan.json
6. tag each eval’s eval_metadata.json with dimension_ids / dimension_labels
7. run benchmark aggregation
8. generate both review.html and report.html
9. only then ask the user to read results or make a call

Benchmark and the final HTML artifacts now check for that plan by default:

python3 scripts/aggregate_benchmark.py ./workspace/iteration-1 --skill-path ./out/my-skill --eval-plan ./out/my-skill/evals/eval-plan.json
python3 scripts/generate_eval_artifacts.py ./workspace/iteration-1 --benchmark ./workspace/iteration-1/benchmark.json --eval-plan ./out/my-skill/evals/eval-plan.json

Formal evaluation is only considered complete when both files exist:

• review.html: the base evidence workbench
• report.html: the human-friendly full report with plan, prompts, answers, scoring, and conclusion

Even if someone directly runs generate_review.py or generate_report.py, the script now auto-generates the companion HTML by default; if that second file cannot be created, the first file is rolled back and the command fails.

Only use this for old archived results:

• --allow-missing-eval-plan

Once a plan exists, benchmark aggregation also hard-checks:

• every eval is tagged with plan-aligned dimensions
• no eval points to dimensions that are not in the plan
• the plan’s key dimensions are actually covered by the eval set

If you compare skill_a / skill_b / skill_c, benchmark markdown and review now render all configs instead of collapsing to only two columns.

Evaluate triggering behavior

This is trigger eval only, not the main path for output-quality or delivery-effect evaluation:

python3 scripts/run_eval.py --eval-set ./path/to/eval-set.json --skill-path ./out/my-skill

Run the optimization loop

Use this only after the skill body is already structurally sound and you are optimizing trigger behavior:

python3 scripts/run_loop.py --eval-set ./path/to/eval-set.json --skill-path ./out/my-skill

Package a skill

python3 scripts/package_skill.py ./out/my-skill ./dist

Runtime Update Check

This repo now ships with a lightweight self-update path that hangs off Step 1 of the creator skill:

• Recommended source: https://github.com/DazhuangJammy/DazhuangSkill-Creator.git
• Recommended install mode: git clone into the target skill directory
• Every real invocation of the skill starts by running scripts/check_update.py
• By default it checks the network at most once every 24 hours
• In throttled mode, JSON exposes the cached remote value as cached_remote_version; latest_version is no longer downgraded to an older cached value than local
• When a new version is found, it reminds once, then stays quiet until an even newer version appears
• The default config already enables update_check.auto_update: true; as long as the current install is a clean git clone, the script will try git pull --ff-only
• If the skill was installed by manually copying the folder, or the working tree has local edits, the script falls back to reminder-only mode
• Even after a successful auto-update, the refreshed files fully apply on the next invocation of the skill; the current run continues with the already loaded version

Minimal config:

update_check:
  enabled: true
  interval_hours: 24
  auto_update: true

If you prefer to disable auto-update explicitly:

update_check:
  enabled: true
  auto_update: false
  interval_hours: 24

Benchmark Reports

You can inspect the archived benchmark outputs here:

• 测评报告/5 个能力原型对比/
• 测评报告/5 个类型性能对比/
• 测评报告/iShot_2026-04-04_12.17.26.png

License

Apache 2.0. See LICENSE and LICENSE.txt.