bhanunamikaze/ai-dataset-generator
Developer toolsЭта страница переведена автоматически. Технические детали смотрите в оригинале. Открыть оригинал
Tool-native dataset generation skill for Codex, Claude Code, and Antigravity with SFT/DPO pipelines, verification, deduplication, and flexible export.
Обзор
An LLM-first dataset generation skill for agent IDEs and AI coding assistants, with 14 specialized sub-skills, 19 pipeline entry scripts, and 15 shared utility modules that turn topics, URLs, or raw files into SFT and DPO training datasets. For detailed installation guidance, example prompts, command reference, generation workflow, reports, and the full script inventory, see the . The installer ships native formats for each tool — not just a generic copy: - Specialized sub-skills: 14 - Pipeline entry scripts in scripts/: 19 (audit.py, augment.py, browser_collect.py, build_loop.py, collect.py, coverage.py, dedup.py, draft_self_check.py, export.py, generate.py, grounding.py, judge_insights.py, quality_report.py, record_history.py, research.py, review_batch.py, status.py, verify.py) - Shared utility modules in scripts/utils/: 15 - Internal canonical schema: 1 (resources/internal-schema/canonical_schema.
README
Показан фрагмент оригинального README. Открыть оригинал.
AI Dataset Generator Skill (Claude / Codex / Antigravity / Cursor / Windsurf / Copilot)
An LLM-first dataset generation skill for agent IDEs and AI coding assistants, with 14 specialized sub-skills, 19 pipeline entry scripts, and 15 shared utility modules that turn topics, URLs, or raw files into SFT and DPO training datasets.
For detailed installation guidance, example prompts, command reference, generation workflow, reports, and the full script inventory, see the Wiki.
IDE Compatibility
The installer ships native formats for each tool — not just a generic copy:
| Tool | Install location | Native format |
|---|---|---|
| Claude Code | ~/.claude/skills/dataset-generator |
Skill directory |
| Codex CLI | ~/.codex/skills/dataset-generator |
Skill directory |
| Antigravity IDE | /.agent/skills/dataset-generator |
Skill directory |
| Claude Cowork | /.claude/skills/dataset-generator |
Project-scoped skill (commit to git) |
| Cursor | /.cursor/rules/dataset-generator.mdc + .cursor/skills/dataset-generator/ |
MDC rule |
| Windsurf | /.windsurf/rules/dataset-generator.md + .windsurf/skills/dataset-generator/ |
Windsurf rule |
| Continue.dev | /.continue/prompts/dataset-generator.prompt + .continue/skills/dataset-generator/ |
Slash command |
| GitHub Copilot | /.github/copilot-instructions.md + .github/skills/dataset-generator/ |
Repo instructions |
| Cline | /.clinerules + .cline/skills/dataset-generator/ |
Project rules |
📦 Current Inventory
- Specialized sub-skills:
14 - Pipeline entry scripts in
scripts/:19(audit.py,augment.py,browser_collect.py,build_loop.py,collect.py,coverage.py,dedup.py,draft_self_check.py,export.py,generate.py,grounding.py,judge_insights.py,quality_report.py,record_history.py,research.py,review_batch.py,status.py,verify.py) - Shared utility modules in
scripts/utils/:15 - Internal canonical schema:
1(resources/internal-schema/canonical_schema.json) - Preset export schemas:
3(inresources/target-schemas/: OpenAI messages, HuggingFace dataset, CSV columns)
Key Script Inventory
The README only highlights the scripts most users reach for first. See the full inventory with purpose notes in the Script Inventory wiki.
| Script | Best for |
|---|---|
build_loop.py |
End-to-end batch build: import drafts, verify, coverage check, dedup, per-batch drift detection, and live progress to workspace/build_loop_progress.json. |
generate.py |
Import canonical draft JSONL into SQLite with --source-type, --dedup-threshold, and injection-tolerant handling for adversarial corpora. |
collect.py |
Multi-backend web search + URL/local-file fallback that emits canonical JSONL the agent can draft from. |
Рекомендуемые инструменты
Попробуйте другой запрос или уберите фильтр.
Установка
npx skillfish add bhanunamikaze/ai-dataset-generator