reflect-mcp

An MCP server for LLM tasks that verify themselves: Generator → second-model Critic → human-in-the-loop gate. Never a silent wrong answer.

---

1. TL;DR

reflect-mcp is a small multi-agent pipeline that checks its own work before surfacing it, exposed as an MCP server so it plugs into Claude Desktop, Cursor, or any MCP client. A Generator extracts a typed object from messy text; a second-model Critic scores every field and the whole extraction; a confidence-thresholded gate either returns the result or, if it isn’t confident enough, returns a needs_review payload naming the exact fields a human should confirm.

                 ┌───────────┐      ┌──────────────┐      ┌────────┐      ┌──────────────┐
  messy text ──▶ │ Generator │ ───▶ │ heuristics + │ ───▶ │  Gate  │ ───▶ │  ok  /       │
                 │ (model A) │      │ Critic (B)   │      │ (HITL) │      │  needs_review│
                 └───────────┘      └──────────────┘      └────────┘      └──────────────┘
                   typed JSON        per-field + overall     threshold      flagged fields
                                       confidence            decision       + rationale

2. Why — the trust problem

LLMs are confidently wrong. The failure mode that actually hurts in production isn’t a model that says “I don’t know” — it’s a model that returns a clean, plausible, wrong value with no signal that anything is off. reflect-mcp makes verification a first-class step: a different model critiques the output, deterministic structural checks run alongside it, and anything below a confidence threshold is escalated instead of emitted.

The pattern is well established: naive extraction pipelines look great in a demo and then quietly degrade on real inputs, and adding a multi-model verification + human-review gate is a proven way to claw accuracy back. The point reflect-mcp makes concrete: the verification discipline is the product, not the extraction call.

In research terms — an evaluation harness with a verifiable reward