bhanunamikaze/ai-dataset-generator

本页由 AI 翻译，技术细节请以原文为准。查看原文

21 stars 质量 69 趋势 18MIT

具备Codex、Claude代码和Antigravity工具原生数据集生成技能，支持SFT/DPO流水线、验证、重复去重和灵活导出。

概览

这是一项面向代理IDE和AI编码助手的LLM首要数据集生成技能，包含14个专业子技能、19个流水线输入脚本和15个共享工具模块，将主题、URL或原始文件转换为SFT和DPO训练数据集。有关详细安装指导、示例提示、命令引用、生成流程、报告及完整脚本清单，请参见。安装程序会为每个工具提供本地格式——而不仅仅是通用的副本： - 专业子技能：14 - 脚本中的流水线输入脚本/：19（audit.py、augment.py、browser_collect.py、build_loop.py、collect.py、coverage.py、dedup.py、draft_self_check.py、export.py、generate.py、grounding.py、judge_insights.py、quality_report.py、record_history.py、research.py、review_batch.py、status.py、verify.py）- 脚本/工具中的共享工具模块：15 - 内部规范模式：1（资源/内部模式/模式canonical_schema。

概述（AI 翻译）

这是一项面向代理IDE和AI编码助手的LLM首要数据集生成技能，包含14个专业子技能、19个流水线输入脚本和15个共享工具模块，将主题、URL或原始文件转换为SFT和DPO训练数据集。

有关详细的安装指导、示例提示、命令引用、生成工作流程、报告以及完整的脚本清单，请参见Wiki。

IDE 兼容性

安装程序会为每个工具提供本地格式——而不仅仅是通用的副本：

工具	安装地点	原生格式
克劳德代码	`~/.claude/skills/dataset-generator`	技能目录
军规库 CLI	`~/.codex/skills/dataset-generator`	技能目录
反重力IDE	`/.agent/skills/dataset-generator`	技能目录
Claude Cowork	[ `/.claude/skills/dataset-generator`	项目范围技能（提交到git）
光标	`/.cursor/rules/dataset-generator.mdc` + `.cursor/skills/dataset-generator/`	MDC规则
风帆冲浪	`/.windsurf/rules/dataset-generator.md` + `.windsurf/skills/dataset-generator/`	风帆冲浪规则
Continue.dev	`/.continue/prompts/dataset-generator.prompt` + '。

本页由 AI 翻译，技术细节请以原文为准。

README

以下为原文 README 节选，完整内容请查看原文。

AI Dataset Generator Skill (Claude / Codex / Antigravity / Cursor / Windsurf / Copilot)

An LLM-first dataset generation skill for agent IDEs and AI coding assistants, with 14 specialized sub-skills, 19 pipeline entry scripts, and 15 shared utility modules that turn topics, URLs, or raw files into SFT and DPO training datasets.

For detailed installation guidance, example prompts, command reference, generation workflow, reports, and the full script inventory, see the Wiki.

IDE Compatibility

The installer ships native formats for each tool — not just a generic copy:

Tool	Install location	Native format
Claude Code	`~/.claude/skills/dataset-generator`	Skill directory
Codex CLI	`~/.codex/skills/dataset-generator`	Skill directory
Antigravity IDE	`/.agent/skills/dataset-generator`	Skill directory
Claude Cowork	`/.claude/skills/dataset-generator`	Project-scoped skill (commit to git)
Cursor	`/.cursor/rules/dataset-generator.mdc` + `.cursor/skills/dataset-generator/`	MDC rule
Windsurf	`/.windsurf/rules/dataset-generator.md` + `.windsurf/skills/dataset-generator/`	Windsurf rule
Continue.dev	`/.continue/prompts/dataset-generator.prompt` + `.continue/skills/dataset-generator/`	Slash command
GitHub Copilot	`/.github/copilot-instructions.md` + `.github/skills/dataset-generator/`	Repo instructions
Cline	`/.clinerules` + `.cline/skills/dataset-generator/`	Project rules

📦 Current Inventory

Specialized sub-skills: 14
Pipeline entry scripts in scripts/: 19 (audit.py, augment.py, browser_collect.py, build_loop.py, collect.py, coverage.py, dedup.py, draft_self_check.py, export.py, generate.py, grounding.py, judge_insights.py, quality_report.py, record_history.py, research.py, review_batch.py, status.py, verify.py)
Shared utility modules in scripts/utils/: 15
Internal canonical schema: 1 (resources/internal-schema/canonical_schema.json)
Preset export schemas: 3 (in resources/target-schemas/: OpenAI messages, HuggingFace dataset, CSV columns)

Key Script Inventory

The README only highlights the scripts most users reach for first. See the full inventory with purpose notes in the Script Inventory wiki.

Script	Best for
`build_loop.py`	End-to-end batch build: import drafts, verify, coverage check, dedup, per-batch drift detection, and live progress to `workspace/build_loop_progress.json`.
`generate.py`	Import canonical draft JSONL into SQLite with `--source-type`, `--dedup-threshold`, and injection-tolerant handling for adversarial corpora.
`collect.py`	Multi-backend web search + URL/local-file fallback that emits canonical JSONL the agent can draft from.

在 GitHub 查看完整 README

安装

npx skillfish add bhanunamikaze/ai-dataset-generator

新手？先看 mcpskill.org 教程 →