boshu2/agentops

DevOps makes software delivery repeatable through shared practices, automated checks and feedback. AgentOps brings that discipline to agentic coding sessions: paved paths your agents can follow from intent to checked results, with memory to carry useful context into the next task.

Those paths combine skills with the ao CLI and your existing engineering tools. Plan clarifies the expected behavior; Test helps check it; Validate adds independent judgment where needed. Skills guide the agent’s decisions; executable gates check specific contracts. Beads preserves the work graph, decisions, results and project memories. Memory curates supported findings into linked project knowledge, connecting lessons to the work and sources behind them. The next session can use that record instead of starting over.

The skills are a kit, and you are expected to change it. Each skill is one Markdown file of instructions. Keep the ones that fit how you work, rewrite the ones that almost fit, delete the rest, and add skills you write yourself or find in other libraries. What you end up with is your own operating model for agents, which is the reason this project exists. Make it yours shows how.

Use these paths with Claude Code, Codex, Cursor, OpenCode, Gemini CLI, Pi and other coding agents, or personal assistants such as OpenClaw and Grok Bot. Start with one useful task; follow the operational loop for a change or the goal workflow for work across sessions. When you need a larger software factory, extend the same practices with Gas City or tools from the Agentic Coding Flywheel.

Why use AgentOps?

A software factory makes engineering practices reusable through tools, guidance and repeatable workflows. AgentOps brings that approach into the coding session: clear intent, automated checks, recorded decisions and a feedback loop into later work. Pick the path your task needs and use the same practices when you change agents.

Behavior-driven development (BDD) starts by agreeing on what the software should do in a concrete situation. For retries, that could mean: “If a completed job arrives again, return its result and charge the customer only once.” The same behavior guides the implementation, its tests and any review. Plan helps you settle those expectations before the agent starts editing.

Quickstart

Pick one method per agent: a plugin plus npx on the same agent gives you every skill twice.

Start a new session so the skills load. Most skills need only your coding agent; a few use the ao CLI when it is installed. Invocation names vary by agent: this README shows Claude Code’s /agentops:; Codex uses $agentops:.

The operational loop

Each change is shaped, built and checked. Use one fresh review when requested, when a mistake cannot be cheaply undone, or when no deterministic check covers the changed behavior. You (or Plan) write intent as behavior (BDD), using one word per concept (DDD’s ubiquitous language). For a system that calls queued work a Job, in Gherkin:

Feature: Job redelivery is idempotent
  A Job is one unit of queued work. Delivering it again never repeats its side effect.

  Scenario: A completed Job is delivered again
    Given Job "J-42" completed and charged the customer $20
    When the worker receives Job "J-42" again
    Then it returns the completed result of "J-42"
    And the customer has been charged $20 exactly once

  Scenario: A Job that failed before charging is delivered again
    Given Job "J-43" failed before charging the customer $20
    When the worker receives Job "J-43" again
    Then Job "J-43" completes
    And the customer has been charged $20 exactly once

The feature defines the domain term once; each scenario has concrete data, one action and an observable result. Keep scenarios in the issue or conversation; no .feature file is required.

Enter at the step you need. An existing change can enter at Validate when an acceptance judgment is needed. Ordinary changes finish on checks and CI. Review is one round: repair findings and confirm with affected checks; review again only when you ask. Report NOT_PROVEN with its gaps. The author cannot issue its own PASS. Merging and releasing follow your repo’s rules.

Goals

rpi runs Plan → Implement → checks, with Validate where a mistake is costly, for one outcome without check-ins (your agent’s permission prompts still apply) and stops at acceptance, a blocker or a spent limit. Bigger work becomes a goal (it needs Beads: brew install beads, then bd init in your repo):

1. Interview. One question at a time, each with a recommended answer. You settle the outcome, its examples, domain terms, non-goals, authority and budget before agents go autonomous.
2. Craft Goal. Returns SAFE_TO_CREATE plus a prompt to paste into /goal (Claude Code or Codex), USE_PLAN (one change: Plan or native execution), or UNSAFE_GOAL plus what’s undecided. It creates nothing itself.
3. Navigate each round. Picks a few ready work items (beads); each gets one RPI, checks and CI, with one fresh Validate when requested, hard to undo, or uncovered by a deterministic check. The goal ends ACHIEVED, NOT_ACHIEVED or NEEDS_OPERATOR.

Beads remembers the work. Beads (bd) keeps the goal and its history outside any conversation. The root epic holds acceptance; child beads carry the scoped work, design decisions, handoffs and result notes. Record which checks ran and link any review evidence to the same bead. That connects the original intent to the checked output, so the next session can continue from the record instead of reconstructing the chat. bd ready finds unblocked work; search and history help explain earlier choices.

Beads also has a persistent memory store. bd remember saves project facts, bd memories searches them, bd recall retrieves one by key, and bd prime brings stored memories into session context. The work graph and the memory store serve different needs: one preserves what happened; the other keeps selected facts easy to find again.

Memory helps you use that material: recall relevant context, examine recorded decisions and outcomes, or draft a report with links to its evidence. When you want a reusable project document, it curates supported findings into reviewed .context/ pages or an external document bundle. Beads keeps the work record; Git and the original evidence locations keep the code and check artifacts. Reusable documents link back to those sources.

bd search "redelivery" --status all     # include finished work
bd show                         # intent, notes and relationships
bd history                      # how the record changed
bd remember "Redelivery returns the stored Job result" --key job-redelivery
bd recall job-redelivery                # retrieve the saved project fact

One bead per worker. When the goal delegates, the orchestrator holds the graph and results, and each worker starts with one bead instead of the orchestrator’s transcript. Reviewers, when needed, start fresh.

bd create "Job redelivery is idempotent" -t epic
bd create "Return the completed result on redelivery" --parent 
bd dep add           # real ordering only
bd ready --parent                 # the frontier

Navigate shows bd commands; another tracker with status, dependencies and notes works if you map them. AgentOps never builds a second work index.

Try it

Start read-only in any repo, then swap the Job example for your own change.

# First look (changes nothing)
/agentops:research how does this repo validate input? cite files and lines, change nothing

# One change
/agentops:plan make Job redelivery return the completed result without repeating the side effect
/agentops:implement
/agentops:validate     # when requested, hard to undo, or uncovered by checks; use a new session

# One outcome, end to end
/agentops:rpi make Job redelivery return the completed result without repeating the side effect

# A goal
/agentops:interview make the job worker safe under redelivery, retries and crash recovery
/agentops:craft-goal   # then paste its prompt into /goal

Skills at a glance

All skills are optional. Load one when it answers a specific question. Full catalog: docs/SKILL-ROUTER.md.

Make it yours

Every skill here is a folder with one SKILL.md: plain instructions an agent loads when a task calls for them. No skill depends on the full set, and coding with none of them still works. That makes the library easy to take apart, and you should. The 31 skills are a starting point for an operating model that fits your work.

• Start small. Install two or three skills that match work you already do. Add another when a real task asks for it.
• Cut what you override. If you or the agent keep ignoring a rule, change the rule or remove the skill. npx skills lets you pick skills per agent, and ao skills link --skill links an exact subset from a checkout.
• Rewrite what almost fits. Fork this repository, edit the SKILL.md and install from your fork with the same commands, or link a checkout so every agent on your machine reads your edits. A plugin update replaces the installed copy, so keep your changes in a repository you control.
• Write your own. When you have explained the same thing to an agent three times, it is a skill. Skill Builder drafts the package, and tells you when a note in an existing file is enough.
• Mix libraries. Run these next to your company’s skills, the Agentic Coding Flywheel tools or any other library. ao skills link never replaces a skill it did not install.
• Change the workflows too. The operational loop and the goal workflow are defaults. Skip the steps your work does not need, reorder them, or write your own.
• Measure what you change. claude plugin eval compares an agent with and without a plugin on the same request. To measure one edit, run the old and the new version as two plugins. Skill Eval covers how to read the result, and Evidence shows the cases this repository runs; copy them for your own skills.

Evidence

A skill is a page of instructions, so the test is whether an agent does anything differently with it installed. AgentOps runs that test on Claude Code’s own evaluator, claude plugin eval: one realistic request per skill, graded against four or five criteria taken from the practice the skill teaches. For Test, one criterion is that a regression test is shown failing without the fix.

On Claude Opus 5.5, measured 2026-10-22 and 2026-10-23 over five rounds; the 3.10 column is the stored 2026-10-05 result:

Sixteen of the 32 skills moved their case by 0.15 or more. Claude Exec, Skill Builder, Plan, Craft Goal and Using GC gained the most. Seven cases had no room to move: the agent met every criterion with no plugin.

Read the limits before you quote these numbers:

• One request per skill, six runs with 3.11, three with 3.10 and three without, one model. A difference under 0.15 is noise.
• The criteria come from each skill’s own rules. A pass shows the rule landed on that request. It says nothing about the outcome of a real task, and an earlier coding pilot found no end-to-end difference.
• Loading is the weak point. 3.11 loads its skills less often than 3.10 did, and Implement and Research never load on their own. Four tested causes do not explain it. Name the skill when you want its rules applied: RPI, Postmortem, Interview and Craft Goal loaded on 22 of 24 requests that named them.

The full report has every case, each round’s rule and result, and what was rerun. The cases live in evals/plugin-eval/: run them against your own changes, or copy the layout to test your own skills.

Where AgentOps fits

AgentOps grew from applying DevOps experience and established engineering practice to agents. The Practice Registry records the lineage; how it works covers responsibilities.

ao CLI (optional)

Most skills need only your coding agent. With ao installed, Validate binds the exact change it judges to a content manifest; without it, Validate names the commit and the changed paths.

brew tap boshu2/agentops
brew trust --tap boshu2/agentops
brew install agentops
ao version

With Go installed: go install github.com/boshu2/agentops/cli/cmd/ao@latest. ao init is optional evidence setup; ao config --show inspects configuration; ao gate check runs this repository’s own gates (mainly for contributors). See the command reference and installation guide.

Updating and advanced setup

Troubleshooting

Limits

Skills guide agents; installation alone does not enforce their instructions. A green test suite or an agreeing model can still miss a defect. Missing proof stays NOT_PROVEN. Saving notes does not establish improved outcomes or automatic knowledge compounding. See product evidence and limits.

FAQ

Beyond AgentOps

Start with one agent and a paved path. When work needs several agents, a factory can add scheduling, coordination and runtime supervision. Install AgentOps skills into its agent runtimes to carry the same behavior, testing and review practices into each task. The factory runs the agents; your checks and any required independent judgment establish what the result proves.

Gas City

Gas City, from the organization behind Beads, provides configurable building blocks for multi-agent software factories: runtime providers, work routing, workflow formulas, packs and supervision. AgentOps ships a Gas City adapter and an executor integration pack. Use the adapter to work through Gas City’s native coordinator and observe its runs; workers can use AgentOps skills for implementation, tests and validation.

Agentic Coding Flywheel

Agentic Coding Flywheel is Jeffrey Emanuel’s (Dicklesworthstone) ecosystem of tools and workflows for operating coding agents. Its setup project assembles a multi-agent development environment. You can also adopt individual tools alongside AgentOps:

For example, use CASS to find how a recurring migration failure was resolved. Use Memory to examine the supported lesson and curate project context. If the method belongs in a repeatable path, Skill Builder can help package it and Skill Eval can measure whether it helps later work. This connects session history to guidance you can inspect and improve.

Gas City, the Flywheel and these tools are independent projects. AgentOps ships its own Gas City adapter and pack; install external tools and their skills from their authors. Follow their native workflows, integrations and distribution terms.

Contributing

Contributions are welcome: documentation fixes, reproducible bug reports, tests, CLI improvements and skills. Read the contribution guide; to work on skills from a checkout, link them with ao skills link.

Licensed under Apache-2.0.