Optimize AI agent skills using DSPy and Agno
概览
A powerful framework for optimizing AI agent skills (prompts, tools, configurations) using optimization algorithms and agents. Transform underperforming prompts into high-quality, production-ready skills with data-driven optimization. After installation, the skill-optimizer command is available globally: : If you prefer not to install globally, you can use uv run skill-optimizer instead. : If you don't have enough examples, generate them using an LLM (requires an API key): : Export optimized skills to Agno Knowledge Base to let your agents retrieve relevant examples at runtime: This generates a code snippet you can paste directly into your Agno agent application. : The included SkillOptimizerAgent now supports: - : Remembers conversation context across sessions (SQLite backend). - : Returns analysis results as Pydantic models. - : Uses Chain-of-Thought for complex optimization tasks.
README
以下为原文 README 节选，完整内容请查看原文。
🚀 Skill Optimizer
A powerful framework for optimizing AI agent skills (prompts, tools, configurations) using DSPy optimization algorithms and Agno agents.
Transform underperforming prompts into high-quality, production-ready skills with data-driven optimization.
✨ Features
| Feature | Description |
|---|---|
| Skill Analysis | Evaluate skills before optimization to understand improvement potential |
| Multiple Strategies | BootstrapFewShot, MIPROv2, BootstrapRS optimizers |
| Auto-Training Data | Generate synthetic training examples using LLMs |
| Side-by-Side Comparison | Compare original vs optimized skills with detailed metrics |
| Flexible Formats | Support for YAML and Markdown skill definitions |
| CLI & Programmatic API | Use via command line or integrate into your Python code |
📦 Installation
Prerequisites
- Python 3.13+
uvpackage manager
Quick Start
# Clone the repository
git clone https://github.com/Ash-Blanc/skill-optimizer.git
cd skill-optimizer
# Install as a CLI tool (recommended)
uv tool install .
# Set up environment variables
cp .env.example .env
# Add your OPENAI_API_KEY, MISTRAL_API_KEY, or GEMINI_API_KEY
After installation, the skill-optimizer command is available globally:
skill-optimizer --help
Note: If you prefer not to install globally, you can use
uv run skill-optimizerinstead.
🎯 Quick Usage
CLI Commands
# Analyze a skill's optimization potential
skill-optimizer analyze email_summarizer
# Generate training data for a skill
skill-optimizer generate email_summarizer --count 10
# Optimize a skill
skill-optimizer optimize email_summarizer --strategy bootstrap_fewshot
# Compare two skill versions
skill-optimizer compare email_summarizer email_summarizer_optimized
# Create a new skill template (Creates skills/my_new_skill/SKILL.md)
skill-optimizer create my_new_skill
# Generate synthetic training data (saves to skills/my_new_skill/TRAINING.json)
skill-optimizer generate my_new_skill --count 10
# Optimize skill (automatically selects best strategy)
skill-optimizer optimize my_new_skill
🛠️ Common Commands
Generate Training Data: If you don’t have enough examples, generate them using an LLM (requires an API key):
# Generate 10 examples and save to the default TRAINING.json
skill-optimizer generate email_summarizer --count 10
# Save to a specific file (useful for creating validation sets)
skill-optimizer generate email_summarizer --count 20 --output validation_set.json
✨ Agno Integration
Dynamic Few-Shot Learning: Export optimized skills to Agno Knowledge Base to let your agents retrieve relevant examples at runtime:
skill-optimizer export-knowledge email_summarizer --vector-db lancedb
This generates a code snippet you can paste directly into your Agno agent application.
Enhanced Agent:
推荐工具
换一个关键词，或者移除筛选条件。
安装
npx skillfish add ash-blanc/skill-optimizer