rigorous, 방법론 구동 파이프라인을 사용하여 에이전트 기술 문서 최적화. Microsoft Research의 SkillOpt Paper (arXiv 2605.23904)에 영감을 얻은 텍스트 공간 최적화는 52/52 설정에서 에이전트 기술을 향상시킵니다. - 7 모델, 6 벤치 마크 및 3 에이전트 하네스.

핵심 문제

대부분의 기술 개발은 품질을 판단하기 위해 기술을 읽고 있습니다. **이 작동하지 않습니다. ** 동반자 SkillLens Paper (arXiv 2605.23899)는 LLM 판단이 46.4%보다 더 악화되어 효과적인 능력을 읽을 수 있습니다.

SkillOpt의 답변 : ** 텍스트를 평가하지 마십시오. **

그것이 작동하는 방법

SkillOpt는 Hermes Agent의 내장 Kanban 시스템을 사용하여 6단계 최적화 파이프라인을 실행합니다.

각 단계:

SkillOpt — Controlled Skill Optimization for Hermes Agent

Optimize any agent skill document using a rigorous, methodology-driven pipeline. Inspired by Microsoft Research’s SkillOpt paper (arXiv 2605.23904), which proved that text-space optimization improves agent skills across 52/52 settings — 7 models, 6 benchmarks, and 3 agent harnesses.

The Core Problem

Most skill development relies on reading the skill to judge its quality. This doesn’t work. The companion SkillLens paper (arXiv 2605.23899) shows that LLM judges are 46.4% worse than chance at distinguishing effective from ineffective skills by reading them.

SkillOpt’s answer: don’t evaluate the text — evaluate the execution.

How It Works

SkillOpt uses your Hermes Agent’s built-in kanban system to run a six-phase optimization pipeline:

flowchart LR
    B[Backlog] --> R1[Rollout]
    R1 --> R2[Reflect]
    R2 --> P[Propose]
    P --> V[Validate]
    V -->|accepted| M[Merge]
    V -->|rejected| RB[Reject Buffer]
    RB -.->|every 4 epochs| SM[Slow/Meta]
    SM -.-> R1
    M -->|epochs < 4| R1
    M -->|epochs ≥ 4| SM
    M --> D[Done ✓]

Each phase:

The Validation-Gate Principle

Training and validation task sets MUST be distinct. Edits are only accepted if they demonstrably improve or maintain performance on unseen tasks. Current validation is multi-objective: pass/fail is the hard primary gate, then output quality, completion speed, and token efficiency are combined into a weighted score.

Quick Start

1. One-time install — clone the repo into your skills directory:

   git clone https://github.com/magnus919/hermes-SkillOpt \
     ~/.hermes/skills/skillopt
   

2. Start a conversation with your Hermes Agent and tell it what you want:

   I want to optimize my vault-note skill using SkillOpt
   

   The agent loads the SkillOpt methodology, works with you to define a test suite, and orchestrates the full optimization pipeline.

Design Philosophy

• Methodology over implementation — The phase design and validation-gate principle are the real deliverable. The scripts are wrappers around the methodology, not the other way around.
• No new infrastructure — Uses Hermes Agent’s existing kanban system. No additional daemons, databases, or APIs.