Skill Tester

Language / 语言: English primary · 中文概览如下。

中文概览

可复用 AI Agent Skill 的质量与触发评测门，检查模糊、错误或过弱的技能是否会在复用中放大。

Quality and trigger-evaluation gate for reusable AI agent skills

Purpose: NATIVE AI · Maturity: INTERNAL UTILITY · Evidence: INTERNAL EVALUATION

Part of TopPrism Native AI.

Skill Tester exists because a growing skill library only compounds organizational capability if weak, ambiguous or incorrectly triggered skills are detected before reuse.

Why this exists

In an agent-native organization, a reusable skill is effectively a unit of organizational knowledge.

Bad skills compound too:

• incomplete instructions get repeated across projects;
• overly broad triggers activate in the wrong context;
• hidden dependencies make workflows brittle;
• a skill that looks good structurally may still fail the task it claims to solve.

Skill Tester provides a repeatable gate between Skill Candidate and Trusted Organizational Skill.

Project / Knowledge Source
        ↓
Skill Candidate
        ↓
Skill Tester
        ↓
Structure + Quality + Trigger Evaluation
        ↓
PASS / IMPROVE / REJECT
        ↓
Organizational Skill Library

What it evaluates

The current implementation checks skills through a staged evaluation pipeline covering:

1. Analysis — understand the skill contract and expected behavior.
2. Generation — construct representative test cases / probes.
3. Execution — run the skill in evaluated contexts.
4. Evaluation — score quality and identify failure modes.

The current internal 4D rubric scores:

The 4D rubric is a TopPrism internal evaluation framework, not an external industry benchmark.

Current tier thresholds (per SKILL.md)

The current numeric tiers drive an operational decision:

These thresholds are part of the project contract (see SKILL.md) and can evolve with the evaluation protocol; they should be versioned together with the rubric.

Trigger quality matters

A reusable skill has two quality dimensions:

Can it solve the task?
        +
Does it activate only when it should?