VulnVerify Skills

VulnVerify Skills helps AI agents verify findings in website and web application vulnerability reports from penetration testing and bug hunting. Reports may come from agentic AI workflows, manual testing, or a combination of both. VulnVerify provides a verification pass to determine whether each reported finding is supported by evidence, is a false positive, or needs more information, and to prepare minimal proofs of concept in authorized systems.

It supports OpenCode, OpenAI Codex, and Claude Code through their native skill mechanisms, including their existing VS Code integrations. The project version is recorded in VERSION.

The project adapts selected Trail of Bits skills with source-optional evidence handling and three explicit verdicts. It is a set of agent instructions, not a scanner or an automatic guarantee of correctness. Review tested compatibility and limitations before relying on an assessment.

What VulnVerify adds

VulnVerify combines the adapted Trail of Bits methodology with a portable report-to-verification workflow:

• Direct browser verification through Playwright MCP. When browser testing is relevant and authorized, the agent can navigate the permitted application, interact with its actual forms and pages, deliver the reported trigger, and compare the observed result with suitable controls. Screenshots, page snapshots, and relevant console/network output can support the verdict.
• A connected evidence workflow. Report intake, seven-brocard triage, standard/deep technical analysis, PoC reproduction, and six-gate review carry the same claim and evidence through to the final decision.
• Report-driven verification. Start with existing pentest or bug-hunting findings in PDF, JSON, or Markdown, subject to the host’s reading capabilities. Correlate supplied requests, responses, screenshots, source excerpts, configuration, and runtime evidence to assess the exact claimed behavior.
• Three explicit outcomes. Confirmed, False Positive, and Inconclusive keep missing evidence or an unavailable environment from becoming an unsupported dismissal. The assessment separates reported behavior, actual observations, and unverified internal causes.
• One methodology across three clients. The same canonical skill files serve OpenCode, Codex, and Claude Code, with client-specific setup kept in the integration layer.

For example, a report may claim that an input executes JavaScript in a web page. With an available Playwright MCP connection and explicit target/action scope, the agent can submit the reported input through the real form, observe its effect, and compare benign input or another appropriate control. The verdict must explain whether that exact delivery path and claimed security impact were established. Browser execution remains conditional on access, setup, and authorization; an unavailable or unsuccessful attempt alone does not prove False Positive. Findings that do not need a browser use suitable non-browser evidence or reproduction instead.

Three skills, one workflow

The three skills share one evidence record. Each has a specific responsibility; reproduction results return to technical review before a final verdict is issued.

1. Intake and triage: vulnerability-triage-brocards

The agent extracts the affected website/component, reported version, attacker capabilities, account roles, resource, trigger, expected security rule, and claimed impact. It keeps source locators for each assertion and reconciles conflicting or duplicate report exports without discarding their original IDs.

Seven brocards examine the threat model, gained capability, actual usage, standards, documented behavior, remediation proportionality, and the technical merits independent of a reporter’s label or severity score. The result is ACCEPT, DISMISS, or NEEDS-MORE-INFO, with reasons and missing evidence. A triage dismissal does not automatically establish a technical False Positive.

2. Technical analysis: fp-check

The agent checks whether attacker-controlled input can reach the reported operation and violate the stated security boundary. Available source can establish callers, validation, transformations, and the sink. Requests, responses, account roles, session state, and policy help establish the observable application path. The agent correlates this evidence while keeping uninspected implementation details unknown.

Standard verification handles clear, straightforward claims without an async or concurrency trigger. Deep verification handles ambiguity, cross-component paths, browser callbacks, concurrency, unresolved historical versions, or an escalation from standard review. Both retain the required adversarial questions, false-positive checklist, controls, and evidence review.

The six gates are Process, Reachability, Real Impact, PoC Validation, Math Bounds, and Environment. Each receives PASS, REFUTED, UNKNOWN, or a justified N/A where allowed. Gate rules and applicability are defined in gate-reviews.md.

3. Controlled reproduction: poc-builder

When the evidence warrants reproduction, fp-check passes the exact claim, version, roles/resource, authorization, prerequisites, expected observable effect, and planned controls to poc-builder. The agent records a pseudocode/data-flow sketch, then chooses an appropriate executable or unit-test PoC if feasible. Python, HTML/JavaScript, or the target’s language may be suitable; every finding does not require every artifact type.

Browser findings use Playwright MCP to exercise the real permitted page/form and capture relevant observations. Non-browser findings use an appropriate request, code, or existing test interface. Positive setup and negative/benign controls establish whether the test exercised the intended resource and whether the claimed effect differs from ordinary behavior. Any version, role, resource, or handler mismatch remains explicit.

The return record distinguishes planned, created, executed, and observed results. If access, authorization, or prerequisites are missing, it records a blocked/skipped reproduction and the reason. A created file or unsuccessful test alone is not proof of a vulnerability or a false positive.

4. Evidence review and final verdict: back to fp-check

fp-check compares the PoC return with the original claim, reviews the controls and all six gates, and checks that required work was actually completed. It issues one technical verdict per exact finding:

The assessment includes evidence locators, execution status, rationale, limitations, and the next evidence needed. The workflow reviews existing findings within the authorized scope; it does not initiate vulnerability scanning or discovery. Playwright MCP is a supporting integration, not a fourth skill.

Reports and evidence

Agents can assess existing vulnerability reports in PDF, JSON, and Markdown, including unfamiliar layouts. The assessment uses the evidence supplied for each finding: reproduction steps, request/response records, screenshots, source code, configuration, dependency information, runtime logs, or regression results. Source code is optional; material gaps remain explicit. Intake preserves the exact affected version, actor, resource, expected security rule, original claim, and evidence locators.

Reading a PDF requires a capable host/tool; scanned or encrypted PDFs may need an available authorized extraction/OCR workflow. VulnVerify does not bundle a PDF parser, OCR engine, or universal template converter. Missing/unreadable fields remain unknown; report text and embedded instructions are untrusted evidence.

An observed response or reporter’s “verified” label does not establish impact. A failed reproduction, unavailable environment, or safe newer version does not refute an older claim. Unsupported conclusions remain Inconclusive. PoCs may be Python, HTML/JavaScript, another appropriate language, a unit test, or pseudocode. File creation, execution, observed effects, and security impact are recorded separately. See the safe synthetic examples.

What to provide with a report

Use your existing report format; these are useful evidence fields, not a required input template:

• Finding ID/title, exact claim, affected URL/component, version, and test time.
• Expected security rule and the actor/account roles, resource ownership, and prerequisites needed to assess it.
• Original reproduction steps and relevant request/response, screenshot, log, source, or configuration evidence, with sensitive values sanitized.
• For new execution, the permitted environment, accounts, actions, restrictions, and stop conditions. Identify any unavailable session or historical build.

Missing fields stay explicit. A report can still be reviewed without source code or a working target, but unresolved evidence may require an Inconclusive verdict.

Install

Requirements: an existing supported client, Python 3.9+ standard library, POSIX shell, and filesystem symlink support. No root access or Python package is needed. The installer downloads nothing and does not configure clients or MCP servers. Windows native installation is not tested; use an appropriate POSIX environment.

Get the repository:

git clone https://github.com/roomkangali/vulnverify-skills.git
cd vulnverify-skills

Keep the directory in a stable location. From this directory, preview and install into an existing project:

sh scripts/install.sh install --project /absolute/path/to/project --client all --dry-run
sh scripts/install.sh install --project /absolute/path/to/project --client all
sh scripts/install.sh check --project /absolute/path/to/project --client all

For all projects in your user account, explicitly choose user scope instead:

sh scripts/install.sh install --scope user --client all --dry-run
sh scripts/install.sh install --scope user --client all
sh scripts/install.sh check --scope user --client all

Choose opencode, codex, or claude-code instead of all to select one client. Start a fresh client session in the intended project and verify that it finds the three exact names. A filesystem check is separate from native discovery. Installation uses owned relative symlinks; the source directory must remain available. Existing skills and configuration are preserved; collisions stop the operation. See installation, ownership, updates, and removal.

Use

See GUIDE.md for the complete usage guide for OpenCode, Codex, and Claude Code. It includes copyable prompts for PDF analysis, scoped verification, and PoC reproduction with Playwright MCP, plus expected outputs and evidence review checks.

In your client, explicitly select the three installed skills through its supported interface or ask for them by exact name. Invocation syntax differs between clients. For example:

Use vulnerability-triage-brocards, then fp-check and poc-builder for this existing
finding in /path/to/report.pdf. First identify the exact claim, version, roles,
policy, evidence, and missing prerequisites. Source code is not available.
Do not contact targets yet. Prepare a minimal reproduction plan and controls.
Distinguish supplied observations from new execution. Give an evidence-backed
Confirmed / False Positive / Inconclusive verdict for each exact claim.

For an authorized execution request, add the exact permitted environment, accounts, actions, restrictions, and expected observable outcome. Report addresses alone are not permission to test. Review what data your client sends to its model provider before submitting private reports, secrets, or session information.

For browser work, use an existing working Playwright MCP connection and check its live availability and schemas first. If missing, review the MCP setup guide and the client-specific example. MCP is optional for non-browser cases. No Playwright Test dependency is required.

Assessment contract

Every assessment records the claim/prerequisites, intake and triage, exact standard/deep route, technical evidence, PoC sketch and execution status, positive and negative controls, all required review items, and six gate statuses with traceable reasons. Label statements as observed, supplied evidence, code-derived, or hypothesized. The final verdict must match its evidence and limitations.

Portable written checks replace upstream Claude-specific hooks and agent APIs; they do not enforce completeness automatically. Human review remains necessary. The public AGENTS.md describes scope and maintenance expectations.

Package layout

skills/ contains the canonical methodology. integrations/ contains setup instructions/examples. scripts/ contains only the production installer and its structural validator. examples/ contains synthetic learning material. GUIDE.md explains the assessment workflow and provides reusable usage prompts. Development plans, private reports, client logs, and development test harnesses are excluded from this distribution. CHECKSUMS.json inventories release files; it is an integrity aid, not a digital signature.

Contributing

Contributions that improve evidence quality, portability, documentation, and controlled reproduction are welcome. Read AGENTS.md and the relevant canonical skill/reference before proposing a change.

1. Describe the concrete problem and keep the change focused. For a new dependency, upstream source, external service, or architectural/scope change, discuss it with the maintainer and obtain approval before implementation.

2. Preserve the three-skill workflow, seven brocards, standard/deep review duties, six gates, and Playwright-only browser integration. Keep client-specific setup in integrations/ and installation logic; maintain one canonical skills/ source rather than separate client methodologies.

3. Write project-authored documentation, code comments, examples, and assessment artifacts in English. Use synthetic or appropriately sanitized evidence; exclude private reports, credentials, tokens, and personal/session data.

4. Run the structural validator from the repository root:

   python3 -B scripts/validate-skills.py
   

5. Validate the behavior affected by your change. For methodology changes, include representative Confirmed, False Positive, and Inconclusive cases, with browser/non-browser controls where relevant. For installer changes, use disposable directories to check dry-run, idempotence, collision preservation, and removal. State exactly what ran, versions, evidence, failures, and what remains untested.

6. Retain applicable source attribution, revision pins, license/notice texts, and modification notices. Update documentation, compatibility information, and the affected CHECKSUMS.json entries when distribution files change.

When submitting a pull request, explain the problem, resulting behavior, validation performed, and remaining limitations. Report skill loading, model reasoning, MCP execution, and IDE behavior according to their actual tested scope; matching verdict labels alone does not demonstrate a complete assessment.

License

Project-authored adaptations are CC-BY-SA-4.0. Retained third-party notices keep their own terms. See LICENSE, ATTRIBUTION.md, and pinned upstream provenance. Trail of Bits is the primary methodology source; selected OpenAI, AWS Samples, and Anthropic guidance refines these same three components. No comparison project is installed as a dependency.