Docker Workbench 和 Agent Skill 用于运行针对代理技能的确定性评估。

使用这个仓库有两种方式：

• 在你的代理中安装skill-optimizer技能/插件，使其能够编写和调试评估套件。
• 运行本地CLI以在Docker中针对OpenRouter模型执行案例和套件。

安装

安装方式因代理而异。规范技能是skills/skill-optimizer/SKILL.md;每个插件清单都指向同一文件。

克劳德·科德

将本仓库注册为 Claude Code 插件市场：

然后安装插件：

OpenAI Codex CLI

将本仓库注册为Codex插件市场：

然后打开插件搜索界面：

选择skill-optimizer并安装。

OpenAI Codex 应用

在Codex应用中，从侧边栏打开插件，搜索skill-optimizer，然后从编码部分安装。

如果没有列出，请先从 Codex CLI 安装：

光标

用开放技能CLI安装该技能：

Cursor 也可以通过设置 -> 规则 -> 项目规则 -> 添加规则 -> 远程规则（Github）从 GitHub 导入该技能。Cursor 插件的元数据位于 .cursor-plugin/plugin.json。

开放代码

告诉OpenCode：

skill-optimizer

Docker workbench and Agent Skill for running deterministic evals against agent skills.

Use this repo in two ways:

• Install the skill-optimizer skill/plugin into your agent so it can author and debug eval suites.
• Run the local CLI to execute cases and suites in Docker against OpenRouter models.

Installation

Installation differs by agent. The canonical skill is skills/skill-optimizer/SKILL.md; every plugin manifest points at that same file.

Claude Code

Register this repository as a Claude Code plugin marketplace:

/plugin marketplace add fastxyz/skill-optimizer

Then install the plugin:

/plugin install skill-optimizer@skill-optimizer

OpenAI Codex CLI

Register this repository as a Codex plugin marketplace:

codex plugin marketplace add fastxyz/skill-optimizer

Then open the plugin search interface:

/plugins

Select skill-optimizer and install it.

OpenAI Codex App

In the Codex app, open Plugins from the sidebar, search for skill-optimizer, and install it from the Coding section.

If it is not listed, install it from Codex CLI first:

codex plugin marketplace add fastxyz/skill-optimizer

Cursor

Install the skill with the open skills CLI:

npx skills add fastxyz/skill-optimizer --skill skill-optimizer -a cursor -y

Cursor can also import the skill from GitHub via Settings -> Rules -> Project Rules -> Add Rule -> Remote Rule (Github). The Cursor plugin metadata lives at .cursor-plugin/plugin.json.

OpenCode

Tell OpenCode:

Fetch and follow instructions from https://raw.githubusercontent.com/fastxyz/skill-optimizer/refs/heads/main/.opencode/INSTALL.md

Or add the plugin to opencode.json at user or project scope:

{
  "plugin": ["skill-optimizer@git+https://github.com/fastxyz/skill-optimizer.git"]
}

Restart OpenCode. See docs/README.opencode.md for details.

Gemini CLI

Install the Gemini extension from GitHub:

gemini extensions install https://github.com/fastxyz/skill-optimizer

To update:

gemini extensions update skill-optimizer

Skill-Only Install

If you only want the skill files without plugin metadata, use the open skills CLI:

npx skills add fastxyz/skill-optimizer --skill skill-optimizer -a claude-code -a opencode -a codex -a cursor -y

Local CLI Setup

Requirements:

• Node.js 20+
• Docker
• OPENROUTER_API_KEY for real model runs

Install and build:

npm install
npm run build

Only openrouter/... model refs are supported.

Quick Start

Run the suite against the models listed in suite.yml:

npx tsx src/cli.ts run-suite examples/workbench/pdf/suite.yml --trials 1

Run one case directly:

npx tsx src/cli.ts run-case ./case.yml --model openrouter/google/gemini-2.5-flash

CLI help:

npx tsx src/cli.ts --help
npx tsx src/cli.ts run-case --help
npx tsx src/cli.ts run-suite --help