mit-lcp/croissant-baker

Croissant Baker

Automatically generate Croissant JSON-LD metadata for ML datasets — e.g. for PhysioNet, NeurIPS Datasets & Benchmarks submissions, or any platform that benefits from standardized dataset metadata.

📖 Documentation

---

Installation

pip install croissant-baker

or with uv:

uv add croissant-baker

Quick start

croissant-baker \
  --input /path/to/dataset \
  --creator "Your Name,[email protected]" \
  --description "My ML dataset" \
  --license "CC-BY-4.0" \
  --output my-dataset-croissant.jsonld

Or try with the bundled MIMIC-IV Demo test data:

git clone https://github.com/MIT-LCP/croissant-baker.git && cd croissant-baker
uv sync --group dev
croissant-baker \
  --input tests/data/input/mimiciv_demo/physionet.org/files/mimic-iv-demo/ \
  --creator "Alistair Johnson,[email protected],https://physionet.org/" \
  --creator "Tom Pollard,[email protected],https://physionet.org/" \
  --name "MIMIC-IV Clinical Database Demo" \
  --description "Demo subset of MIMIC-IV containing 100 de-identified patients from Beth Israel Deaconess Medical Center" \
  --url "https://physionet.org/content/mimic-iv-demo/2.2/" \
  --license "https://opendatacommons.org/licenses/odbl/1-0/" \
  --rai-data-biases "Single-site cohort from a US academic medical centre" \
  --rai-data-limitations "Demo subset limited to 100 patients" \
  --output mimic-iv-demo-croissant.jsonld
croissant-baker validate mimic-iv-demo-croissant.jsonld

Use with an agent

croissant-baker ships a local MCP server and an Agent Skill. The server needs the optional mcp extra:

pip install 'croissant-baker[mcp]'

Then point your MCP client at the croissant-baker mcp command. Most clients take an entry like this one:

{
  "mcpServers": {
    "croissant-baker": {"command": "croissant-baker", "args": ["mcp"]}
  }
}

The server speaks stdio only and offers three tools: dry_run, bake and validate. It reads local files and makes no network request. Pass absolute paths, since a relative one resolves against the directory the client started the server in. The skill ships inside the package at croissant_baker/skills/croissant-baker/SKILL.md, and the server also serves it as the resource croissant-baker://skill.

Supported formats

Any of these may arrive wrapped in .gz, .bz2 or .xz — compression is resolved before the format is read, so cells.parquet.gz is described exactly as cells.parquet is. WFDB is the exception: a record spans several files located by path, so a compressed .hea is reported rather than described. .zip, .tar and .tgz are reported as archives, not opened. Files nothing describes are reported with a reason rather than skipped in silence.

Key features

• Automatic type inference for all supported formats
• RAI metadata via --rai-* CLI flags or --rai-config rai.yaml
• Genomic sample identifiers withheld by default: VCF and BCF report a sample count and BAM, CRAM and SAM a read-group count; the identifiers themselves are listed only with --genomic-sample-ids. File and directory names are emitted either way, so files or directories named by sample (SJ001234_D1.bam) still show their sample in every contentUrl
• Validation against the Croissant spec via mlcroissant
• Dry-run mode, include/exclude glob filters, multiple creators

See the documentation for full CLI reference, examples, and RAI configuration.

Contributing

See CONTRIBUTING.md for guidelines and DEVELOPMENT.md for setup, testing, releases, and how to add new file handlers.

Citation

Croissant Baker was accepted at NeurIPS 2026 (Evaluations & Datasets track). The camera-ready version is coming soon. Until then, please cite our arXiv preprint:

@misc{attrach2026croissantbakermetadatageneration,
      title={Croissant Baker: Metadata Generation for Discoverable, Governable, and Reusable ML Datasets},
      author={Rafi Al Attrach and Rajna Fani and Sebastian Lobentanzer and Joan Giner-Miguelez and Debanshu Das and Varuni H. K. and Nobin Sarwar and Rajat Ghosh and Anwai Archit and Surbhi Motghare and Christina Conrad Parry and Luis Oala and Lara Grosso and Joaquin Vanschoren and Steffen Vogler and Sujata Goswami and Eric S. Rosenthal and Marzyeh Ghassemi and Matthew McDermott and Tom Pollard},
      year={2026},
      eprint={2605.15079},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2605.15079},
}

License

MIT License - see LICENSE file.