AI Skills for Production Engineering

These are Research backed agent skills that close the gap between prototype and production.

This library complements widely used public skill collections (for example, Matt Pocock’s) by concentrating on lifecycle gaps that routinely separate demos from systems you can run under load, audit, and operate: self-audit, security, performance, regression, accessibility, and operational resilience. The skills are written so they can be composed: small, repeatable procedures with explicit evidence requirements rather than open-ended brainstorming.

Use this repository as a human-curated, research-anchored playbook. Each skill cites a narrow set of named sources; extend it only when you can attach equivalent evidence (see CONTRIBUTING.md).

If a skill produces a report, treat that report like a code review: every finding should be reproducible from a command, a file path, or a quoted snippet.

---

The Research Behind These Skills

Self-audit and validation skill

Modern agents benefit from structured reflection and verification loops that treat outputs as hypotheses until checked.

• Renze & Guven (Johns Hopkins University, 2024) — Self-Reflection in LLM Agents: Effects on Problem-Solving Performance. Self-reflection improved LLM accuracy with p < 0.001 across GPT-4, Llama 2 70B, and Gemini 1.5 Pro.
• Dhuliawala et al. (Meta FAIR, ACL 2024) — Chain-of-Verification Reduces Hallucination in Large Language Models. CoVe reduced list-generation hallucinations from 2.95 to 0.68 entities and improved biography factuality (FACTSCORE) from 55.9 to 71.4.
• Zhang et al. (NTU / Skywork AI, 2025) — Incentivizing LLMs to Self-Verify Their Answers. Models trained to self-verify achieved verification accuracy on par with GPT-4o and Claude-3.7-Sonnet.
• Li et al. (BIGAI / Peking University, ACL 2025) — ReflectEvo: Improving Meta Introspection of Small LLMs by Reflection Learning. Iterative reflection improved Llama-3-8B from 52.4% to 71.2% and Mistral-7B from 44.4% to 71.1%.
• Ma et al. (2025) — S²R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning. Qwen2.5-math-7B improved from 51.0% to 81.6% using iterative self-verification.

Security audit

Security review for agent-generated code must assume unsafe defaults and implicit trust boundaries.

• Trail of Bits Security Skills Suite (community standard; 2,439+ stars) — covers static analysis, insecure defaults, variant analysis, and differential security review.
• OWASP Top 10 for Agentic AI (2025) — documents prompt injection, insecure output handling, and excessive agency as critical AI security risks.

Performance optimization

Performance work needs measurable baselines; guessing “feels fast” fails under real traffic.