Skill RSI

Watch the Intro Video here

Want Codex to install it for you? Paste this into a fresh Codex thread:

Set up Skill RSI from https://github.com/justinwetch/Skill-RSI. Clone the repo, read AGENTS.md, install dependencies, build the UI, configure and install the Codex plugin, run the plugin smoke checks, and tell me when to start a fresh Codex thread. Do not start any model-backed Skill RSI run yet.

What if a skill could improve itself?

Recursive self-improvement is the phrase AI researchers reach for when they describe systems that get better without us in the loop. Usually it’s a thought experiment. This one runs on your laptop.

Give Skill RSI a goal and it writes an Agent Skill to chase it. Hand it a skill you already have and it makes that one better instead. Then the loop starts: study the skill, form a theory about what would help, build a variant, test it, keep the winner. Again and again, on its own, until you stop it.

Before it writes a single line, it studies the domain. It researches the field into sourced claims, practitioner vocabulary, and intertextual context, then compiles them into an ontology: a working model of who the skill serves, what excellent output looks like, the failure modes worth catching, the expert distinctions novices flatten, and the authorities whose strong opinions sharpen what “good” means here. That ontology becomes the shared map every later loop builds on and the guardrail that keeps the work from drifting.

Once there’s a champion to beat, Skill RSI deconstructs it into more than a dozen testable surfaces, every one carrying a hypothesis, a regression risk, and a way to measure whether the change actually worked. Each loop the current champion is the control and the next variant is the treatment, both run against the same prompts under the same criteria. One variable moves at a time, so when a challenger wins, you know exactly what won.

Skill RSI improves skills the way a research lab works: every loop is an experiment whose results are kept and built on, including a record of dead ends, so it never wastes time re-testing what doesn’t work. Each pass starts better informed than the last, converging on what does.

You set the goal. The research, the experiments, the evaluation, and the record-keeping run themselves.

This is the sequel to SkillEval. SkillEval could tell you which of two skills was better. The question it left open was what to build next, and that part was still on you. Skill RSI closes the loop.

For the origin story, see Open Sourcing SkillEval.

From Codex, Skill RSI opens the local app and prepares the project without starting model-backed work in chat.

The sidebar keeps the result and next-loop controls visible beside the conversation.

During a run, the UI tracks the current stage and the evidence being produced.

Evidence-Backed Decisions

Every promotion decision links back to prompt-level evidence: the prompt, judge rationale, criterion scores, and both candidate outputs.

Skill RSI also keeps the candidate packages inspectable, so every champion change stays traceable.

UI Walkthrough

The summary view shows the head-to-head result at a glance.

The detailed breakdown explains where each candidate won across the criteria.

The prompt list keeps every judgment inspectable.

The champion skill can be opened directly from the UI.

The history view records the improvement trajectory over time.

The next loop plan carries forward what the latest evidence says to try, preserve, or avoid.

How It Works

For the canonical explanation of the loop and agent roles, see docs/HOW_IT_WORKS.md.

Research Depth

When model-native research is available, Skill RSI now aims for controlled depth rather than a tiny citation packet. A sourced run defaults to roughly 12-15 strong sources, 6-10 evidence claims, 4-8 authority entries, and 20-50 practitioner lexicon entries.

The practitioner lexicon is meant to capture top-percentile vocabulary: field terms, methods, artifacts, metrics, failure modes, schools, debates, boundary terms, and near-synonym distinctions. Each entry should say what an expert means, how a novice might misuse it, why it matters for the skill, how evals could test it, and what evidence supports it.

The intertextual map records relationships rather than bibliography alone: canonical texts, standards and institutions, schools of thought, recurring debates, concept lineages, adjacent-domain borrowings, and common misreadings. Thin lexicon or intertextual coverage creates inspectable quality warnings, but those warnings do not force a revise loop by themselves.

First scratch run:

load project
  -> build research packet and ontology
  -> seed initial parameterization
  -> plan cold-start duel
  -> generate Candidate A and Candidate B
  -> adversarial preflight review
  -> evaluate Candidate A vs Candidate B
  -> crown first champion when evidence is usable
  -> write history and next-loop plan

Later runs, or baseline-upload projects:

load champion and history
  -> deconstruct current champion into improvement parameters
  -> use ontology as domain guardrail, not as a rewrite brief
  -> plan one controlled challenger
  -> generate challenger as a localized ablation of the champion
  -> adversarial preflight review
  -> evaluate challenger vs champion
  -> promote or keep current champion
  -> write history and next-loop plan

Baseline uploads skip the initial ontology step and start from deconstructing the uploaded champion. Existing ontologies are reused or conservatively refreshed when the champion changes.

Diagram Views

The core loop keeps every run framed as a controlled experiment.

The ontology turns research into a domain guardrail before the first skill is written.

The experiment plan decides what changes, what stays fixed, and what evidence should count.

History keeps each loop from forgetting what already worked or failed.

Quick Start

git clone https://github.com/justinwetch/Skill-RSI.git
cd Skill-RSI
npm install
cp .env.example .env

Add your OpenAI key to .env for CLI/server use:

OPENAI_API_KEY=sk-...

Stub mode runs without API calls:

node src/cli.js init my-skill \
  --goal "Help agents write clear technical documentation." \
  --output text \
  --model gpt-6-sol \
  --target-iterations 3
node src/cli.js run my-skill --stub --loops 3

Start from an existing skill package:

node src/cli.js init my-skill \
  --goal "Improve this existing skill." \
  --baseline ./path/to/skill-or.zip

Real agentic run:

node src/cli.js run my-skill --agentic --real-eval --loops 1

Inspect results:

node src/cli.js status my-skill
node src/cli.js summary my-skill
node src/cli.js progress my-skill
node src/cli.js timeline my-skill
node src/cli.js report my-skill
node src/cli.js skill my-skill --source champion

Local UI

Build and start the local server, then open http://127.0.0.1:8765:

npm run build:ui
npm run server

To use a non-default port:

SKILL_RSI_SERVER_PORT=8766 npm run server

PowerShell:

$env:SKILL_RSI_SERVER_PORT=8766; npm run server

For hot-reload UI development, run these in separate terminals:

npm run server
npm run dev:ui

The UI supports:

• creating projects from scratch or from an uploaded skill folder, single SKILL.md, or zip
• output artifact selection: Text, Code, or Code + visuals
• OpenAI model selection: gpt-6-sol by default, gpt-6-astra for the most demanding work, or gpt-6-luna for lower-cost iteration. Existing projects can still select the GPT-5.6, GPT-5.5, and GPT-5.4 Mini choices.
• browser-local OpenAI API key entry, with .env as server fallback
• live loop progress, next-loop plan, detailed eval data, champion/challenger skill viewing, and visual screenshots when available
• automation status for manual runs, cron/LaunchAgent setup commands, and queued Codex hook context

Model choice can be changed between runs; each run records the models it used. API keys are not stored in project config; the UI stores pasted keys locally in the browser.

The current curated OpenAI choices use the GPT-6 API model IDs. The CLI also accepts a model ID directly for advanced use. Model availability can vary by API account; no model-backed run starts merely by changing the selection.

Codex Plugin

Skill RSI also ships a repo-tracked Codex Plugin at plugins/skill-rsi. The plugin teaches Codex to open the local Skill RSI web app in the Codex sidebar, use the MCP control plane for structured operations, and fall back to CLI commands when needed.

The plugin currently expects a local Skill RSI checkout. Use the copy-paste Codex setup prompt at the top of this README, or see AGENTS.md for the agent-facing setup guide.

Manual setup:

git clone https://github.com/justinwetch/Skill-RSI.git
cd Skill-RSI
npm install
npm run build:ui
cp .env.example .env
# Add OPENAI_API_KEY to .env
npm run plugin:configure
codex plugin marketplace add .
codex plugin add skill-rsi@skill-rsi

If your Codex version cannot auto-install plugins from the CLI, install from the Codex UI instead: open Plugins in the sidebar, click Built by OpenAI, select Skill RSI, open the Skill RSI plugin page, then click Add to Codex.

After installing or updating the plugin, start a new Codex thread so Codex loads the latest plugin skill and MCP tools. You can start from the Skill RSI plugin page or one of its starter prompts, or mention @Skill RSI from chat.

Validate and smoke-check the plugin from the repository root:

npm run plugin:configure -- --check
npm run plugin:validate
npm run plugin:smoke

Skill RSI model-backed runs use the OpenAI API key in .env or the browser-local UI key field. A ChatGPT Plus/Pro subscription does not fund local Skill RSI API calls.

The plugin is a Codex-native operator surface. By default, it opens the local web app at http://127.0.0.1:8765/ because that remains the most reliable guided console in Codex desktop. MCP tools can prepare fresh setup drafts, create or import projects when explicitly requested, return focused launch URLs, inspect existing projects only when explicitly requested, run bounded loops only when explicitly requested, export champions, and explicitly consume queued Codex context. The CLI remains the best surface for reproducible automation.

MCP-UI support is host-dependent and is not the default Codex desktop path. skill_rsi_open returns the local app launch URL only; skill_rsi_cockpit is the optional embedded console for hosts with robust MCP Apps/UI rendering. Codex hooks only queue context; they do not run RSI loops or spend model budget by themselves.

For install/update details and troubleshooting, see plugins/skill-rsi/README.md. For the plugin release checklist, see docs/CODEX_PLUGIN_RELEASE.md.

Operating Modes

Manual:

node src/cli.js run ux-design --agentic --real-eval --loops 1

Scheduled or continuous:

node scripts/skill-rsi-cron-runner.mjs ux-design --agentic --real-eval --max-runs 20 --max-new-runs 1 --agent-model gpt-6-sol

Queue hook context:

node src/cli.js hook-record ux-design --event hook.json

Codex hooks only queue context; the next manual or scheduled run consumes it. For cron, PowerShell, or macOS LaunchAgent setup, see docs/SCHEDULING.md. For Codex hook setup, see docs/CODEX_HOOKS.md.

What A Run Produces

Cold start:

runs//
├── deconstruction/
│   ├── research-packet.json
│   ├── ontology.json
│   ├── parameterization.json
│   └── experiment-plan.json
├── candidates/
│   ├── candidate-a/
│   └── candidate-b/
├── eval/
│   ├── config.json
│   └── candidate-duel.json
└── analysis/
    ├── recommendation.json
    └── report.md

Champion challenge:

runs//
├── deconstruction/
│   ├── parameterization.json
│   └── experiment-plan.json
├── challenger/
├── eval/
│   ├── config.json
│   └── challenge.json
└── analysis/
    ├── recommendation.json
    └── report.md

Visual runs also persist rendered HTML, screenshots, render diagnostics, and screenshot references under the run’s eval artifacts.

After a promotion, champion/skill/ is updated and history/current-summary.md records what changed, what was learned, what not to repeat, and what the next loop should try.

CLI Reference

# Project management
node src/cli.js doctor
node src/cli.js init  --goal "..." --output text|code|code_visual --model gpt-6-sol --target-iterations 3
node src/cli.js init  --goal "..." --baseline ./path/to/skill-or.zip
node src/cli.js projects
node src/cli.js status 
node src/cli.js delete 
node src/cli.js diagnose [project]
node src/cli.js support-prompt [project]

# Running loops
node src/cli.js run  --stub --loops 3
node src/cli.js run  --agentic --real-eval --loops 1
node src/cli.js run  --agentic --real-eval --loops 1 --model gpt-6-sol
node src/cli.js step 
node src/cli.js continuous  --agentic --real-eval --max-runs 20 --max-new-runs 1
node src/cli.js continuous  --agentic --real-eval --max-runs 20 --max-new-runs 1 --consume-hooks
node src/cli.js hook-record  --event hook.json

# Inspection
node src/cli.js summary 
node src/cli.js progress 
node src/cli.js compare 
node src/cli.js timeline 
node src/cli.js report 
node src/cli.js run-detail 
node src/cli.js skill  --source champion
node src/cli.js skill  --source challenger --run latest --json
node src/cli.js export-skill  --source champion --out ./exported-skill
node src/cli.js inspect-skill 

# Standalone evaluation
node src/cli.js evaluate  \
  --a ./skill-a --b ./skill-b \
  --prompts prompts.json --criteria criteria.json \
  --output text \
  --gen-model gpt-6-sol --judge-model gpt-6-sol \
  --out result.json

node src/cli.js evaluate  \
  --a ./skill-a --b ./skill-b \
  --prompts prompts.json --criteria criteria.json \
  --output code_visual \
  --visual-artifacts-dir ./visual-artifacts \
  --gen-model gpt-6-sol --judge-model gpt-6-sol \
  --out result.json

# Annotation, not required promotion
node src/cli.js decide  --decision annotate --note "Reviewed."

init --output code_visual checks local screenshot runner availability before creating the project. Run node src/cli.js doctor for OpenAI key status, supported model names, and visual runner diagnostics. node src/cli.js diagnose [project] optionally creates a support zip under .skill-rsi-diagnostics/; it may contain run data, but it should not contain API keys. node src/cli.js support-prompt [project] prints copy-paste instructions for Codex, including asking an agent to audit what you are sending. Per-run model flags override project config and should be used intentionally because they make the run history less uniform.

Project Workspace

All state lives under .skill-rsi/projects// as plain files.

.skill-rsi/
└── projects/
    └── ux-design/
        ├── config.json
        ├── project.yaml
        ├── state.json
        ├── champion/skill/
        ├── ontology/current.json
        ├── parameterization/current.json
        ├── prompt-bank/
        ├── runs/
        └── history/
            ├── current-summary.md
            ├── index.json
            └── detailed/

config.json is the machine-readable source for triggers, budgets, promotion policy, eval policy, model choices, research policy, and portability. project.yaml is a compact human-readable companion.

Current UI-created model config:

{
  "models": {
    "agent": "gpt-6-sol",
    "generation": "gpt-6-sol",
    "judge": "gpt-6-sol"
  }
}

API Keys

CLI and local server runs load .env automatically:

OPENAI_API_KEY=sk-...

The UI can also use a pasted key saved locally in the browser. Project files store model names, not API keys.

Visual Evaluation

Code + visuals projects require generated skills to produce complete standalone browser-renderable UI code. Skill RSI renders each output with Playwright/system Chromium, captures desktop/tablet/mobile screenshots, records render diagnostics, and sends the text plus images to the judge.

The product exposes Code + visuals, where the artifact is still browser-renderable code. Separate rendered-media generation is not part of the product surface.

Built On

Skill RSI’s evaluation engine is a headless adaptation of SkillEval. Skill packages follow the Agent Skills standard. For architecture and roadmap notes, see IMPLEMENTATION_PLAN.md.

Built by Justin Wetch