bhanunamikaze/ai-dataset-generator
Developer tools이 페이지는 AI로 자동 번역되었습니다. 기술적인 세부 사항은 원문을 확인하세요. 원문 보기
Tool-native dataset generation skill for Codex, Claude Code, and Antigravity with SFT/DPO pipelines, verification, deduplication, and flexible export.
개요
An LLM-first dataset generation skill for agent IDEs and AI coding assistants, with 14 specialized sub-skills, 19 pipeline entry scripts, and 15 shared utility modules that turn topics, URLs, or raw files into SFT and DPO training datasets. For detailed installation guidance, example prompts, command reference, generation workflow, reports, and the full script inventory, see the . The installer ships native formats for each tool — not just a generic copy: - Specialized sub-skills: 14 - Pipeline entry scripts in scripts/: 19 (audit.py, augment.py, browser_collect.py, build_loop.py, collect.py, coverage.py, dedup.py, draft_self_check.py, export.py, generate.py, grounding.py, judge_insights.py, quality_report.py, record_history.py, research.py, review_batch.py, status.py, verify.py) - Shared utility modules in scripts/utils/: 15 - Internal canonical schema: 1 (resources/internal-schema/canonical_schema.
README
원문 README의 일부입니다. 원문 보기.
AI Dataset Generator Skill (Claude / Codex / Antigravity / Cursor / Windsurf / Copilot)
An LLM-first dataset generation skill for agent IDEs and AI coding assistants, with 14 specialized sub-skills, 19 pipeline entry scripts, and 15 shared utility modules that turn topics, URLs, or raw files into SFT and DPO training datasets.
For detailed installation guidance, example prompts, command reference, generation workflow, reports, and the full script inventory, see the Wiki.
IDE Compatibility
The installer ships native formats for each tool — not just a generic copy:
| Tool | Install location | Native format |
|---|---|---|
| Claude Code | ~/.claude/skills/dataset-generator |
Skill directory |
| Codex CLI | ~/.codex/skills/dataset-generator |
Skill directory |
| Antigravity IDE | /.agent/skills/dataset-generator |
Skill directory |
| Claude Cowork | /.claude/skills/dataset-generator |
Project-scoped skill (commit to git) |
| Cursor | /.cursor/rules/dataset-generator.mdc + .cursor/skills/dataset-generator/ |
MDC rule |
| Windsurf | /.windsurf/rules/dataset-generator.md + .windsurf/skills/dataset-generator/ |
Windsurf rule |
| Continue.dev | /.continue/prompts/dataset-generator.prompt + .continue/skills/dataset-generator/ |
Slash command |
| GitHub Copilot | /.github/copilot-instructions.md + .github/skills/dataset-generator/ |
Repo instructions |
| Cline | /.clinerules + .cline/skills/dataset-generator/ |
Project rules |
📦 Current Inventory
- Specialized sub-skills:
14 - Pipeline entry scripts in
scripts/:19(audit.py,augment.py,browser_collect.py,build_loop.py,collect.py,coverage.py,dedup.py,draft_self_check.py,export.py,generate.py,grounding.py,judge_insights.py,quality_report.py,record_history.py,research.py,review_batch.py,status.py,verify.py) - Shared utility modules in
scripts/utils/:15 - Internal canonical schema:
1(resources/internal-schema/canonical_schema.json) - Preset export schemas:
3(inresources/target-schemas/: OpenAI messages, HuggingFace dataset, CSV columns)
Key Script Inventory
The README only highlights the scripts most users reach for first. See the full inventory with purpose notes in the Script Inventory wiki.
| Script | Best for |
|---|---|
build_loop.py |
End-to-end batch build: import drafts, verify, coverage check, dedup, per-batch drift detection, and live progress to workspace/build_loop_progress.json. |
generate.py |
Import canonical draft JSONL into SQLite with --source-type, --dedup-threshold, and injection-tolerant handling for adversarial corpora. |
collect.py |
Multi-backend web search + URL/local-file fallback that emits canonical JSONL the agent can draft from. |
추천 도구
다른 키워드를 입력하거나 필터를 제거해 보세요.
설치
npx skillfish add bhanunamikaze/ai-dataset-generator