Independent portfolio project. Not affiliated with, endorsed by, or connected to Unity. All customer records are synthetic.

CSM Copilot

Move customer onboarding forward without turning uncertainty into a promise.

A customer-success portfolio by Yuchi Wang: a working Model Context Protocol (MCP) server and interactive case explorer that turn account evidence into bilingual follow-ups and technical handoffs.

Start the three-minute review · Project brief (PDF) · Interview walkthrough · Run locally

Workflow screenshot from source revision 0ea4b88. The website replays Python workflow outputs and saved model attempts; it does not run a live AI chat or send messages.

The business problem

A publisher wants to launch on Friday. An iOS test passed, but a non-test Android device shows a blank Offerwall. The CSM needs to decide what to ask the customer, what to document, and when to involve technical support.

The prototype follows that work across five fictional accounts and six app/platform integrations, covering Offerwall onboarding, blank-offerwall reports and reward callbacks. It preserves the difference between a recorded failure, missing evidence and a possible explanation.

Start with one case

1. Inspect Android. Currency approval and Test Only are unknown; the device check is recorded as not met. Expand the source view to read the dated evidence.
2. Read the next customer question. The draft asks for missing confirmations and a fresh device result. Switch to Chinese, then inspect the handoff’s missing fields.
3. Compare iOS. Its separate checklist is complete, so the next action is a launch-plan review. Completion is not launch approval.
4. Add fresh evidence. Passing new checks changes the Android follow-up to an issue-status question; it does not automatically close the earlier complaint.

Every draft remains unsent. Drafting does not update contact history. Review dates and source dates stay distinct: moving the review clock forward does not refresh customer records.

Where AI fits

CSM request → AI host → scoped tools → results returned to the AI host → human review

An AI host can choose tools and rewrite drafts. The Python server handles dates, platform scope, checklist states and source references deterministically. Its default drafts are templates, so the workflow can run without a model or API key.

The repository preserves sixteen local Qwen attempts across four prescribed tasks. In one, four tool calls succeeded but the explanation described iOS as having “full compliance.” Later host-assisted corrections brought structured fields into agreement with tool results; free prose still required review. The first queue-first attempt, dependency-instruction attempt and policy-enabled attempt all stopped at batch parsing before any tool executed. These are exploratory attempts with changing prompts and settings, not an independent benchmark.

The deterministic workflow is the working baseline: it already selects accounts, checks readiness and assembles drafts. The model experiments explore tool orchestration and explanation. The platform task supplies both integration IDs, so its successful calls do not demonstrate autonomous account discovery. No comparison has established that adding the model saves time or improves customer outcomes. Inspect the division of work and the evidence for it.

The host policy and constrained generation now restrict each turn to pending work and IDs returned by earlier tools. A new assisted morning run completed all seven requested calls: queue, context, two readiness checks, two English drafts and one handoff. The host then timed out waiting for the final explanation, so no model-written answer is claimed. The replay exposes the supplied state, decoding schemas, actual tool returns and terminal error. This provides an inspectable failure-to-test-to-fix-and-retest example while keeping tool completion separate from answer quality.

The host now saves requests before dispatch and records specific timeout causes. A subsequent run with a 600-second response budget completed the seven calls and returned an 80-word explanation after 272.986 seconds of final generation. It correctly names the missing handoff fields but describes unknown currency approval and a recorded failed device check together as missing verification. Two excerpt-to-field comparisons make the supported and problematic wording inspectable. This is a completed assisted execution, with reasoning and citation gaps still visible.

Inspect the original AI mistake · Follow the recorded corrections · Read the raw-run review

An optional morning final-answer checker now requests separate unknown/failed checks, follow-up purposes, selected-account facts and handoff gaps. It compares those fields with observed calls and renders a separate summary on agreement. The first actual run with this mode separates the evidence states but is rejected for including an issue report among readiness citations. The ID is real and correctly scoped; its evidentiary purpose is wrong for that field. No correction is attempted, and free prose still requires review.

An optional final-context reduction removes superseded host planning messages while preserving every tool result and an exact omission audit. Scripted checks verify content preservation and interrupted requests; no new model run or speedup is claimed yet.

Comparison screenshot from the same source revision. The assessment is an AI-assisted review of a saved attempt; open the replay to locate each field in its raw trace.

Choose what to inspect

For presentation details, use the interview walkthrough. For a portable replay, download the fixed v0.29.8 interview package; the package guide distinguishes that release from newer source changes. Sharing instructions cover direct links, platform briefs and practice exports.

Tools

All eight MCP tools are read-only. The two prompts compose a morning review and a Product-feedback discussion.

MCP setup and prompts · Actual protocol transcript · Architecture and decisions

What the evaluation actually shows

Known failures remain visible, including indirect wording, qualified negation and quoted topics. There is no independent human validation, measured customer impact or general model-reliability estimate.

Methodology and results · Verification scope · Before/after triage explorer · Detailed development history

Portfolio rationale

This project connects customer communication, business analytics and process controls. It makes a chain of decisions inspectable: identify the account, separate evidence from assumptions, ask a focused question, prepare a handoff, and review what an AI explanation adds.

Contribution transparency: Yuchi selected the direction, intended audience and CSM workflow, and commissioned the build. Implementation, tests, documentation and synthetic cases used substantial AI assistance. Unaided coding, completed personal mastery exercises and independent customer validation are not claimed.

The learning path asks you to predict a result, change a copied input, inspect the output and explain the difference. Practice keeps initial reasoning separate from later reflection, with local save/resume and export. Those responses can provide a concrete basis for discussing your own learning.

Run locally

To present the committed browser replay, follow the local preview commands; no package installation or model is needed. To execute the workflow, use Python 3.11–3.13 and install the Python package, then print a morning summary or export English/Chinese briefs. The full verification sequence includes JavaScript checks, package checks and PDF builders.

Repository map

src/csm_copilot/   Evidence schema, workflow engine, MCP server and fixtures
demo/             Interactive replay, downloadable briefs and PDFs
tests/, tests-js/  Authored behavior and boundary checks
evaluation/       Synthetic diagnostic cases and evaluators
evidence/         Workflow outputs, protocol transcript and model recordings
docs/             Sources, methods, setup, interview and learning guides
scripts/          Builders and protocol/package verification

Public sources are paraphrased and linked in the source register, reviewed 2026-10-14. Project policies are labeled separately from official product guidance. See limitations and data handling before extending the prototype. Code is MIT licensed; referenced vendor documentation retains its own terms.