magnus919/hermes-skillopt

本页由 AI 翻译，技术细节请以原文为准。查看原文

32 stars 质量 62 趋势 31MIT

Hermes 代理的受控技能优化——源自 Microsoft Research 的 SkillOpt： 52/52 设置，通过文本空间优化和验证门进行了改进。

概览

灵感来源于 Microsoft Research 的 SkillOpt 论文（arXiv 2605.23904），该论文证明文本空间优化能提升 52/52 场景下的智能体技能——7 个模型、6 个基准测试和 3 个智能体工具。大多数技能发展依赖于阅读技能来判断其质量。配套的SkillLens论文（arXiv 2605.23899）显示，LLM评判者在通过阅读技能区分有效与无效技能方面比偶然差46.4%。 SkillOpt 利用你的 Hermes Agent 内置看板系统运行六阶段优化流程：训练和验证任务集必须是不同的。只有当编辑能够明显提升或维持未被发现任务的性能时，才会被接受。当前的验证是多目标的：通过/不通过是硬的主门槛，然后输出质量、完成速度和令牌效率合并成加权分数。 1. — 将仓库克隆到你的技能目录中： 2.

概述（AI 翻译）

使用严谨、方法论驱动的流水线优化任何代理技能文档。 灵感来自Microsoft Research的SkillOpt论文（arXiv 2605.23904），该论文证明文本空间优化能提升52/52场景下的代理技能——7个模型、6个基准和3个代理工具。

核心问题

大多数技能发展依赖于阅读技能来判断其质量。**这行不通。**配套的SkillLens论文（arXiv 2605.23899）显示，LLM评判者在通过阅读技能区分有效与无效技能方面比偶然差46.4%。

SkillOpt的回答是：不要评估文本——要评估执行。

工作原理

SkillOpt 利用你的 Hermes Agent 内置看板系统运行六阶段优化流程：

每个阶段：

相位	发生了什么	神器
展开	在训练任务中执行技能	N 轨迹记录
反思识别各次推广中的系统性失败模式	反思文档
提议	生成1-4次有界编辑以修复失败	编辑提案
验证	每次编辑与未完成任务进行测试	用指标接受/拒绝

本页由 AI 翻译，技术细节请以原文为准。

README

以下为原文 README 节选，完整内容请查看原文。

SkillOpt — Controlled Skill Optimization for Hermes Agent

Optimize any agent skill document using a rigorous, methodology-driven pipeline. Inspired by Microsoft Research’s SkillOpt paper (arXiv 2605.23904), which proved that text-space optimization improves agent skills across 52/52 settings — 7 models, 6 benchmarks, and 3 agent harnesses.

The Core Problem

Most skill development relies on reading the skill to judge its quality. This doesn’t work. The companion SkillLens paper (arXiv 2605.23899) shows that LLM judges are 46.4% worse than chance at distinguishing effective from ineffective skills by reading them.

SkillOpt’s answer: don’t evaluate the text — evaluate the execution.

How It Works

SkillOpt uses your Hermes Agent’s built-in kanban system to run a six-phase optimization pipeline:

flowchart LR
    B[Backlog] --> R1[Rollout]
    R1 --> R2[Reflect]
    R2 --> P[Propose]
    P --> V[Validate]
    V -->|accepted| M[Merge]
    V -->|rejected| RB[Reject Buffer]
    RB -.->|every 4 epochs| SM[Slow/Meta]
    SM -.-> R1
    M -->|epochs &lt; 4| R1
    M -->|epochs ≥ 4| SM
    M --> D[Done ✓]

Each phase:

Phase	What Happens	Artifact
Rollout	Execute the skill against training tasks	N trajectory records
Reflect	Identify systematic failure patterns across rollouts	Reflection document
Propose	Generate 1-4 bounded edits to fix failures	Edit proposals
Validate	Test each edit against held-out tasks	Accept/reject with metrics
Merge	Deploy accepted edits, snapshot, increment epoch	Updated skill
Slow/Meta	Learn from the rejected-edit buffer every 4 epochs	Meta-reflection

The Validation-Gate Principle

Training and validation task sets MUST be distinct. Edits are only accepted if they demonstrably improve or maintain performance on unseen tasks. Current validation is multi-objective: pass/fail is the hard primary gate, then output quality, completion speed, and token efficiency are combined into a weighted score.

Quick Start

One-time install — clone the repo into your skills directory:

git clone https://github.com/magnus919/hermes-SkillOpt \
  ~/.hermes/skills/skillopt

Start a conversation with your Hermes Agent and tell it what you want:
```
I want to optimize my vault-note skill using SkillOpt
```
The agent loads the SkillOpt methodology, works with you to define a test suite, and orchestrates the full optimization pipeline.

Design Philosophy

Methodology over implementation — The phase design and validation-gate principle are the real deliverable. The scripts are wrappers around the methodology, not the other way around.
No new infrastructure — Uses Hermes Agent’s existing kanban system. No additional daemons, databases, or APIs.

在 GitHub 查看完整 README

安装

npx skillfish add magnus919/hermes-skillopt

新手？先看 mcpskill.org 教程 →