Self-Improving Project Factory

The self-improving evolution of Project Factory — the multi-agent SDD delivery framework — extended with a reflection & self-improvement layer: the factory now measures its own process with the same exit-coded rigor it applies to the product, turns human corrections into artifacts no gate can pass over, and propagates earned lessons into the next project’s gates. It is a public superset — every project-factory capability plus the reflection layer; nothing here is client-specific (all fixtures are synthetic, and campaign lessons cite their field reports abstractly). The layer is grounded in three field studies in this repo: the reflection mechanism design (the winning “Process-Health Ledger” design, 8 mechanisms, 12-step plan), the pixel-perfect forensics (the motivating failure: a run that printed “Pass” over an acceptance method it never executed), and the 13-project course field study.

What is new in the self-improving layer

The eight mechanisms (each shipped with an executed red→green proof under tests/; run npm test):

1. Process-health ledger (ledger.mjs) — every check run appends {check, exitCode, scope_n, phase, warningsByClass, …} to trace/ledger.jsonl; fail-open telemetry whose consumers are hard gates.
2. Acceptance-contract auditor (check-acceptance-methods.mjs) — joins every declared verification tag to a real mechanism (G3, existence) and a fresh threshold-passing artifact (G4+, artifact); the pixel-case killer, hard from day one.
3. Three-valued gate semantics — PASS / NOT-EARNED / FAIL + explicit SKIP-pending; absence of evidence never renders as success, and no overall “Pass” may sit on a NOT-EARNED constituent.
4. Correction intake (correct.mjs) — a user’s “this is not pixel perfect!” becomes retro/corrections/*.correction.json; undispositioned corrections are red OPEN-CORRECTION lines on every gate.
5. Process ratchet with tighten-only direction guard (check-process-ratchet.mjs + retrofitted coverage/eval ratchets) — per-class phase-aware warning budgets; any loosening --update is auto-rejected and requires a waiver artifact + owner-signed RULES-CHANGELOG.md row.
6. Reflection pass — deterministic digest (ledger-report.mjs → docs/qa/process-health.md) always; LLM process-auditor (plain subagent, maker≠checker) at phase boundaries, on demand, and at abandonment/handover → docs/qa/process-defects.json.
7. Bounded self-improvement queue — accepted defects become openspec/changes/improve-PD-x/ proposals with the exact diff, an EXECUTED red→green proof, and one-revert rollback; human-approved, and the next retro verifies the metric actually moved (else auto-flagged for revert). Promotion ladder lives in RULES-CHANGELOG.md.
8. Integrity lock + lessons library (check-factory-integrity.mjs, lessons/) — gate-bearing scripts are hash-locked at post-adaptation G0 (drift without an approved Refs: PD-x commit = hard red); proven lessons upsert into every new project’s AGENTS.md at init/onboard.

Also new: check-visual-fidelity.mjs (the pixel-parity gate the motivating failure lacked), the operator guide in docs/self-improvement.md, and the non-negotiable invariant: honesty checks are gate fixes with no off-switch — FACTORY_TELEMETRY=off disables telemetry emission only.

Parity Playbook

For pixel-parity / visual-fidelity work (reproduce a live site at ≥ near-100%), the layer adds a multi-layer parity acceptance pattern upstreamed from the parity campaign (see docs/self-improvement.md for the full chapter, and the templates/docs/requirements.template.md NFR example). Three earned lessons drive it (installed into AGENTS.md at init):

1. block-conquest-doctrine — iterate block by block to a per-block definition-of-done (unpaired 0 / geometry 0 / paint 0 / asset 0 / pixel ≥ floor); the full-page pixel average is telemetry, never the gate. Converge each block with the overlay / onion-skin feedback pattern.
2. sampling-blindness — verified samples are never continuum coverage. Every sampled check declares its sampling dimension and a stricter-instrument escalation path (5-width matrix → fine-step continuum pixel sweep; geometry channel → pixel channel).
3. capture-determinism — neutralize every capture non-determinism source (free-run carousel timers, sub-pixel clip origin, captureBeyondViewport fixed-chrome bleed, unsettled lazy widgets, uncleared persistence) before a below-floor sample is trusted as a product defect.

The acceptance escrow ties in: the multi-layer parity roll-up is the fresh, threshold-passing artifact that hard check-acceptance-methods --mode=artifact joins to each declared pixel-diff / vision-verify / e2e tag — parity convergence is proven by an exit code, not by narrative.

---

Everything below is the upstream framework this self-improving layer builds on.

Project Factory — Multi-Agent SDD Delivery Framework

A reusable, multi-agent framework for building business applications with spec-driven development (OpenSpec), capability slicing, relentless validation, and a QA proof pack that ends with automated UAT video recordings.

Packaged as a Claude Code plugin and works with other modern Agents like Codex, Cursor, Github Copilot: add it to any new or existing repo and an orchestrator skill drives the agents through the whole lifecycle (see How to use it).

It encodes the complete, proven lifecycle tested on sample projects.

What it produces for a new project

• Scaffolding, agent rules (AGENTS.md), and skills installed.
• Numbered requirements (FR-x/NFR-x/TC-x/BC-x) and a product brief.
• OpenSpec baseline specs per capability, strictly validated.
• A dependency-ordered capability plan with one owner per requirement.
• Implemented capabilities, each through a gated loop: spec → design → tasks → tests(red) → implement(green) → review gates → archive.
• Four test layers: unit, DB integration, Playwright E2E, production build.
• Multi-dimensional adversarial review gates (correctness, security, spec-compliance, best practices) on every slice and once globally.
• A QA proof pack: traceability matrix, manual test plan, risk register, acceptance report, automated verification runner.
• Graded evals: an eval-suite that scores qualitative behavior (error clarity, usability) against rubrics with a fresh judge (maker≠checker), guarded by an eval ratchet — the quality bar the recordings then illustrate.
• Trajectory evals: check-trajectory proves the path each slice took (review evidence ran clean, Slice: trailers, module scope) and a trajectory-eval workflow judges process-soundness — output evals grade the result, trajectory evals grade the route.
• Demo + UAT proof recordings: per-scenario video + screenshot + markdown explainer, generated by a Playwright recorder script.
• Handoff docs (docs/current-state.md), technical docs, ADRs, estimation, and a customer-facing delivery report with an effort log.
• A built-in UAT bug-triage loop for when the customer’s QA report arrives: bug→requirement mapping, verdicts, root-cause fixes, regression tests, and bug-fix proof recordings.
• Optional standing automations: cost-tiered, off-by-default scheduled watchers (trace/coverage/eval drift, CI-failure triage, dependency audit) that propose findings as reports/issues and never auto-fix — runnable locally or in the cloud, with a one-flag master switch.

How to use it

It’s a Claude Code plugin — install once, use in any repo (new or existing).

Install

# add this repo as a marketplace, then install the plugin
claude plugin marketplace add 
claude plugin install project-factory
# …or, for local development, just point Claude Code at the repo:
claude --plugin-dir 

…or as a standalone skill

The skills/project-factory/ folder is self-contained — its SKILL.md bundles the playbook, gates, and init/onboard procedures under references/ (no ${CLAUDE_PLUGIN_ROOT} dependency), so you can drop it in as a plain Agent Skill instead of installing the plugin:

# user-level (every repo) — or .claude/skills/ inside one repo for project-level
cp -r skills/project-factory ~/.claude/skills/project-factory

It auto-loads next session and triggers on the same phrases. The orchestrator guidance is fully present standalone; the multi-agent loop payload (the 11 agents, 5 workflows, check-* scripts) is still installed into your target repo by init, which copies it from this plugin/repo — so keep the plugin installed or this repo on hand for the build itself. (The plugin path is simplest because it carries both at once.)

Run it

Open Claude Code in the target repo. Let the project-factory skill trigger (“set up project factory”, “build this to spec”), or drive it explicitly:

• New / greenfield: /project-factory:init installs the loop (Gate G0); then hand the orchestrator your requirements — it runs Phases 1–8.
• Existing codebase: /project-factory:onboard detects the stack, installs the loop non-destructively, reverse-engineers requirements + baseline specs from your code, then governs all new work through the gates.

You opt in to multi-agent orchestration; then answer the clarification batch and the approval checkpoints (scope, capability plan, and — for onboard — the reverse-engineered baseline). Everything else runs autonomously, gate by gate.

Prefer the manual flow? You can still read MASTER-PROMPT.md and execute it directly — the plugin simply automates the install + dispatch.

Works in your tool

Project Factory is agent-agnostic — the same gated lifecycle runs under Claude Code, Cursor, GitHub Copilot, and Codex, with AGENTS.md + the project-factory skill as the portable core:

• Claude Code — the .claude-plugin/ plugin (above).
• Cursor — install the .cursor-plugin/ plugin, or commit .cursor/rules/project-factory.mdc; same /project-factory:init / :onboard.
• GitHub Copilot — .github/copilot-instructions.md + the /project-factory-init / -onboard prompt files.
• Codex / others — AGENTS.md (read natively) + .codex/prompts/.

The deterministic loop (specs, gates, check-*, evals, trajectory) is identical everywhere; only the orchestration substrate differs — parallel subagent fan-out on Claude Code, sequential fresh-context passes elsewhere (maker ≠ checker preserved). init --tools=… lays the adapters into a target repo. Full detail: docs/portability.md.

Contents

The two enforcement pillars

1. Everything trackable — the chain FR → spec → slice → test (@trace) → recording manifest → commit trailer is validated by check-traceability (deterministic, exit-coded); the traceability matrix is generated, never hand-written, and CI fails when it goes stale.
2. Deterministic checks before agent judgment — git hooks, agent edit hooks, and CI close the inner loop on every edit and commit; LLM review (review-gate) runs only on top of green commands. See LOOP.md.

Default stack (overridable)

Defaults to a proven, production-validated stack — Next.js (App Router) · Postgres (Neon) + Drizzle · Better Auth · Resend · Vitest · Playwright · OpenSpec · Vercel. Phase 0 includes an explicit stack-decision step: any piece may be swapped if the requirements demand it, recorded as an ADR. The conventions (module layout, validation cadence, proof recordings) survive a stack swap; the reference scripts assume Node + Playwright.

Design principles (paid for in bugs)

1. Errors must surface. No thrown validation error may become a generic 500; no external call (email!) may fail silently while the UI claims success.
2. Root cause over symptom. Bug reports are clustered by cause; one fix per class, applied everywhere the class occurs.
3. UI state mirrors server state. Uncontrolled forms are keyed by the server data they display.
4. Never archive before smoke. A capability is done only after a real-DB smoke flow and the full validation battery pass.
5. Shared databases drift. Seed helpers re-pin baseline state on every run; tests use local calendar dates, not UTC slices.
6. Proof, not claims. Every delivered behavior gets a recording; every fix gets a regression test; every report states what failed as plainly as what passed.

License

MIT © 2026 Vyacheslav Koldovskyy. Use it freely — including in commercial and closed-source projects — provided the copyright notice is kept.