muminur/autoresearch-skill-andrej-karpathy

---

✨ What is this?

Autoresearch is a Claude Code skill that turns a free-form goal like

/autoresearch reduce API p95 latency to 200ms

into an autonomous, self-correcting optimization loop that:

1. 🧠 Parses the goal into seven machine-readable slots
2. 📦 Ingests real data (refusing synthetic corpora)
3. 🛠️ Builds a single-file benchmark harness
4. 📐 Captures a baseline + regression test count
5. 🔁 Iterates — one atomic change at a time
6. ✅ Keeps wins, 🗑️ auto-discards regressions, logs everything
7. 🏁 Stops when the target metric is hit

No hand-holding. No “should I continue?” Just mechanical iteration until the goal is reached.

---

🌟 Why use it?

---

🚀 Quick Start

1. Install (global skill)

# Clone into your Claude Code skills directory
git clone https://github.com/Muminur/autoresearch-skill-Andrej-Karpathy.git \
  ~/.claude/skills/autoresearch

On Windows:

git clone https://github.com/Muminur/autoresearch-skill-Andrej-Karpathy.git `
  "$env:USERPROFILE\.claude\skills\autoresearch"

2. Invoke

Open Claude Code in any project and type:

/autoresearch reduce avg_latency_ms below 500

Claude will print the parsed slot dump, walk through the five harness phases, then begin the autonomous loop.

3. Watch it work

Results are appended to autoresearch/results.tsv in your project:

commit    metric     avg_latency_ms  status      description
953b71d   1.000000   2008.9          baseline    77-case corpus, 10 signals executed
953b71d   1.000000   646.4           keep        parallelize _place_exits with asyncio.gather
953b71d   1.000000   592.1           keep        add 30s TTL cache on account()
953b71d   1.000000   488.8           keep        prewarm account+connection pool, serialize signals

---

🔌 Use with other AI coding assistants

The skill is plain Markdown — drop it into any agent that supports custom prompts, rules, or instructions. Clone the repo once to a staging location, then symlink or copy into each tool’s config path:

git clone https://github.com/Muminur/autoresearch-skill-Andrej-Karpathy.git ~/autoresearch-skill

On Windows substitute %USERPROFILE%\autoresearch-skill for ~/autoresearch-skill in the commands below.

🧠 OpenAI Codex CLI

Codex CLI loads custom prompts from ~/.codex/prompts/.

mkdir -p ~/.codex/prompts
cp    ~/autoresearch-skill/SKILL.md  ~/.codex/prompts/autoresearch.md
cp -r ~/autoresearch-skill/references ~/.codex/prompts/

Invoke in Codex chat: /autoresearch reduce API p95 to 200ms.

💡 Tip — add @file ~/.codex/prompts/autoresearch.md to ~/.codex/AGENTS.md to keep the rubric pre-loaded in every session.

🤖 GitHub Copilot (VS Code & JetBrains)

Copilot Chat supports prompt files (*.prompt.md) and instructions files (*.instructions.md).

Project-scoped (checked into the repo):

mkdir -p .github/prompts
cp    ~/autoresearch-skill/SKILL.md  .github/prompts/autoresearch.prompt.md
cp -r ~/autoresearch-skill/references .github/prompts/

User-scoped (available in every project):

Enable prompt files in VS Code settings.json:

{ "chat.promptFiles": true }

Invoke: /autoresearch in Copilot Chat.

🪐 OpenCode

OpenCode keeps custom commands at ~/.config/opencode/command/.

mkdir -p ~/.config/opencode/command
cp    ~/autoresearch-skill/SKILL.md  ~/.config/opencode/command/autoresearch.md
cp -r ~/autoresearch-skill/references ~/.config/opencode/command/

Invoke: /autoresearch in OpenCode.

⚡ Cursor

Cursor loads rules from .cursor/rules/*.mdc.

mkdir -p .cursor/rules
cp    ~/autoresearch-skill/SKILL.md  .cursor/rules/autoresearch.mdc
cp -r ~/autoresearch-skill/references .cursor/rules/

Prepend this frontmatter to .cursor/rules/autoresearch.mdc:

---
description: Autonomous goal-directed iteration
globs:
alwaysApply: false
---

Invoke: @autoresearch reduce API p95 to 200ms in Cursor Chat (the @rule syntax attaches the rule to the prompt).

For a global install across every project, use Cursor’s Settings → Rules → User Rules pane and paste the skill content.

🌊 Windsurf

Windsurf loads rules/workflows from .windsurf/rules/.

mkdir -p .windsurf/rules
cp    ~/autoresearch-skill/SKILL.md  .windsurf/rules/autoresearch.md
cp -r ~/autoresearch-skill/references .windsurf/rules/

Prepend this frontmatter:

---
trigger: manual
description: Autonomous goal-directed iteration
---

Invoke: @autoresearch reduce bundle size below 200KB in Windsurf Chat.

For global scope, drop the files in ~/.windsurf/rules/ instead of the project-local path.

📝 Vanilla VS Code / Continue / any Markdown-aware chat

Any extension that respects .github/copilot-instructions.md (e.g. GitHub Copilot Chat, Cline, Roo Code):

mkdir -p .github
cat ~/autoresearch-skill/SKILL.md >> .github/copilot-instructions.md

For Continue:

mkdir -p ~/.continue/commands
cp ~/autoresearch-skill/SKILL.md ~/.continue/commands/autoresearch.md

Then prompt the assistant: “Apply the autoresearch skill to reduce p95 to 200ms” — the full playbook is already in context.

🧾 Compatibility matrix

⚠️ Note — /autoresearch works autonomously. For best results use a model with strong agentic and long-context capabilities (Claude Opus/Sonnet 4+, GPT-4.1+, Gemini 2.5 Pro).

---

🧭 How it works

flowchart TD
    A["/autoresearch [goal]"] --> B[Parse goal → 7 slots]
    B --> C{"corpusrequired?"}
    C -- yes --> D[📦 Phase AIngest real data]
    C -- no --> E[🛠️ Phase BBuild harness]
    D --> E
    E --> F[📐 Phase CCapture baseline]
    F --> G[🛡️ Phase DRun regression suite]
    G --> H[🔎 Phase ERead hot path]
    H --> I((LOOP))
    I --> J[Modify ONE file]
    J --> K[git commit]
    K --> L[Run harness]
    L --> M[Run regression suite]
    M --> N{metricimproved?}
    N -- yes --> O[✅ keep + log]
    N -- no --> P[🗑️ discard + reset]
    M -- regressed --> P
    O --> Q{goalhit?}
    P --> I
    Q -- no --> I
    Q -- yes --> R[🏁 Done]

    style A fill:#8A2BE2,stroke:#333,color:#fff
    style D fill:#FF6F00,stroke:#333,color:#fff
    style E fill:#FF6F00,stroke:#333,color:#fff
    style F fill:#FF6F00,stroke:#333,color:#fff
    style G fill:#FF6F00,stroke:#333,color:#fff
    style H fill:#FF6F00,stroke:#333,color:#fff
    style O fill:#2e7d32,stroke:#333,color:#fff
    style P fill:#c62828,stroke:#333,color:#fff
    style R fill:#1565c0,stroke:#333,color:#fff

---

🎛️ Subcommands

Chain with Claude Code’s /loop for bounded runs:

/loop 25 /autoresearch reduce bundle size below 200KB

---

🧩 Goal-parsing rubric

When you type /autoresearch , Claude extracts seven slots from your free-form text:

Three worked examples:

/autoresearch reduce API p95 latency to 200ms
→ metric=p95_latency_ms, direction=minimise, target=200, verify_cmd=python benchmark.py

/autoresearch reduce test flakiness to 0%
→ metric=flaky_test_rate, direction=minimise, target=0, corpus=CI run history

/autoresearch increase signal-parser reliability to 99%
→ metric=reliability, direction=maximise, target=0.99, regression_cmd=pytest -q

---

🏗️ The Harness Protocol

The default path runs through five mandatory phases before the loop begins.

sequenceDiagram
    autonumber
    participant User
    participant Claude
    participant Repo
    participant Tests

    User->>Claude: /autoresearch [goal]
    Claude->>Claude: Parse goal → 7 slots
    Claude->>User: Print parsed-slot dump

    Note over Claude,Repo: 📦 Phase A — Corpus Ingestion
    Claude->>Repo: Scrape/locate real data
    Claude->>Repo: Write autoresearch/data/*.jsonl
    Claude-->>User: corpus N cases from source

    Note over Claude,Repo: 🛠️ Phase B — Harness Construction
    Claude->>Repo: Write benchmark.py (single file)
    Claude->>Claude: verify stdout starts with metric:

    Note over Claude,Repo: 📐 Phase C — Baseline Capture
    Claude->>Repo: Run benchmark.py
    Claude->>Repo: Append iteration #0 to results.tsv

    Note over Claude,Tests: 🛡️ Phase D — Regression Gate
    Claude->>Tests: Run pytest -q (or equivalent)
    Claude->>Repo: Record N_pre in .regression-baseline

    Note over Claude,Repo: 🔎 Phase E — Hot-Path Reading
    Claude->>Repo: Trace entry → handler → I/O
    Claude-->>User: 3-5 candidate ideas

    loop Until goal hit or interrupted
        Claude->>Repo: Modify ONE file
        Claude->>Repo: git commit
        Claude->>Repo: Run benchmark.py
        Claude->>Tests: Run regression suite
        alt regression passed && metric improved
            Claude->>Repo: status=keep
        else regression failed OR metric worse
            Claude->>Repo: git reset --hard HEAD~1
            Claude->>Repo: status=discard
        end
    end

Full protocol: references/benchmark-harness.md

---

🛡️ The Eleven Critical Rules

---

📊 Real-world case study

Executed in this project (WhatsApp Signal Trader on Binance testnet):

Result: 2008 ms → 488 ms (76% reduction) with zero regressions across 373 pre-existing tests. Goal of < 500ms achieved in 3 iterations.

---

📁 File structure

autoresearch/
├── SKILL.md                             # Entry point read by Claude Code
├── README.md                            # This file
├── LICENSE                              # MIT
└── references/
    ├── core-principles.md               # 7 generalisable Karpathy principles
    ├── autonomous-loop-protocol.md      # Phase-by-phase loop rules
    ├── benchmark-harness.md             # Corpus + harness + regression gate
    ├── results-logging.md               # TSV schema for iteration logs
    ├── plan-workflow.md                 # /autoresearch:plan wizard
    └── security-workflow.md             # /autoresearch:security audit

---

🧬 Domain adaptability

---

🙏 Credit & Inspiration

• Andrej Karpathy — for the original autoresearch pattern: single file, single metric, iterate.
• Strix — adversarial AI security testing with PoC validation (inspiration for /autoresearch:security).
• OWASP Top 10 — the industry-standard vulnerability taxonomy.
• STRIDE — Microsoft’s threat-modeling framework.

The core insight from Karpathy that drives every design decision here:

Autonomy scales when you constrain scope, clarify success, mechanize verification, and let agents optimize tactics while humans optimize strategy.

---

📝 License

MIT — see LICENSE.

---