The AI-Ready Data Framework

Introduction

The AI-Ready Data Framework is an open standard that defines what “AI-ready” actually means — six factors, 62 measurable requirements, and five workload profiles that apply to any data platform.

The AI-Ready Data Agent is an installable skill that puts the framework to work. Point it at your data estate, tell it your workload — RAG, agents, feature serving, or training — and it scores every requirement, surfaces gaps, and guides you through remediation.

The framework defines what to measure. The agent measures it and fixes what doesn’t pass.

Background

The contributors to this framework include practicing data engineers, ML engineers, and platform architects who have built and operated AI systems across industries.

This repo synthesizes our collective experience building data infrastructure that can reliably power AI. Our goal is to help data practitioners design infrastructure that produces trustworthy AI decisions.

Who should use this repo?

• Data engineers building pipelines that power AI systems.
• Platform teams designing infrastructure for ML and AI workloads.
• Architects evaluating whether their stack can support RAG, agents, or real-time inference.
• Data leaders who need to assess organizational AI readiness and communicate gaps to their teams.
• Coding agents building the data infrastructure they will eventually consume.

The Six Factors of AI-Ready Data

1. Clean: Clean data is consistently accurate, complete, valid, and free of errors that would compromise downstream consumption.
2. Contextual: Meaning is explicit and colocated with the data. No external lookup, tribal knowledge, or human context is required to take action on the data.
3. Consumable: Data is served in the right format and at the right latencies for AI workloads.
4. Current: Data reflects the present state, with freshness enforced by infrastructure rather than assumed by convention.
5. Correlated: Data is traceable from source to every decision it informs.
6. Compliant: Data is governed with explicit ownership, enforced access boundaries, and AI-specific safeguards.

These factors apply to any data system powering AI applications, regardless of tech stack.

Requirements

Each factor is backed by a set of measurable requirements — specific, platform-agnostic criteria that define what must be true of your data. Requirements describe the what, not the how. The full canonical list lives in the skill manifest.

The factor markdown files above describe the why and what of each factor in prose. The manifest provides the machine-readable counterpart: every requirement has a unique key, a description, a factor, and a scope (schema, table, or column). All tests return a normalized score between 0 and 1, making it straightforward to build automated assessments or dashboards on top of the framework.

AI-Ready Data Agent (Skill)

An installable skill for coding agents. Use it to scan your data estate for prioritization, assess specific assets against a profile, and get a scored report across the six factors of AI-ready data with guided remediation.

Currently supports Snowflake. Looking for contributions to extend to other platforms.

Quick Start

Install as a skill

npx skills add Snowflake-Labs/ai-ready-data -a cortex

Standalone

Clone or add this repo as workspace context. The agent reads skills/ai-ready-data/SKILL.md automatically.

Start assessment

After installing, ask your coding agent:

Assess my [data assets] for RAG readiness.

The agent asks your platform and scope, loads the RAG profile, runs checks, and presents a scored report. From there you can drill into failures and remediate stage-by-stage.

For estate-level prioritization:

Scan my data estate for AI readiness.

The agent sweeps across all schemas in a database with lightweight readiness proxies and presents a comparative ranking.

How It Works

Three phases, from light to deep: Scan, Assess, Remediate.

1. Choose a platform: Snowflake, Postgres, etc
2. Discovery: tell the agent your database, schema, and tables, or scan your entire estate
3. Choose a profile: RAG, feature serving, training, agents, full assessment, or pick specific requirements
4. Adjust: skip, set, or add requirements before running
5. Coverage: see what’s runnable on your platform before executing
6. Assess: platform-specific checks score each requirement 0–1
7. Remediate: for failures, the agent presents platform-specific fixes for your approval

Factor Stages

Every assessment is organized into six stages, one per factor of AI-ready data:

All scores are 0–1 where 1.0 is perfect. Requirements pass when score >= threshold.

Built-In Profiles

Each profile selects a different subset of the total requirements, with thresholds tuned for the use case. Every assessment uses the same six stages.

Overrides

Before running, you can adjust any profile on the fly:

• skip : exclude a check entirely
• set : override a threshold
• add : include a check not in the base profile

For repeatability, save overrides as a custom profile using extends:

name: my-rag-profile
extends: rag
overrides:
  skip:
    - embedding_coverage
  set:
    chunk_readiness: { min: 0.70 }
  add:
    row_access_policy: { min: 0.50 }

Key Concepts

• Factor: one of six categories of AI-ready data (Clean, Contextual, Consumable, Current, Correlated, Compliant). Factors define the dimensions along which data is evaluated.
• Requirement: a platform-agnostic criterion that must be true of the data. Requirements define what to measure, not how. All requirements live in a single manifest (requirements/requirements.yaml).
• Check: a platform-specific markdown file (check.md) containing prose context and SQL that measures a requirement, returning a normalized 0–1 score. Context, constraints, and variant guidance are co-located directly above the SQL they apply to.
• Diagnostic: a platform-specific markdown file (diagnostic.md) containing prose context and SQL that provides detail drill-downs on check results.
• Fix: a platform-specific markdown file (fix.md) containing remediation options — executable SQL and/or organizational process guidance. A single file can contain multiple remediation paths with prose explaining when to use each. Fixes are executed only with explicit user approval.
• Profile: a curated collection of requirements with thresholds, organized into the six factor stages. Profiles can target a workload (RAG, training, feature-serving, agents) or an estate-level scan. Five built-in, unlimited custom.
• Assessment: the guided flow where the agent discovers scope and profile, runs tests, and produces a scored report.
• Scan: estate-level sweep using the lightweight scan profile across many schemas for comparative prioritization. Scans turn into assessments when the user drills into a specific schema.
• Platform Reference: everything the agent needs to operate on a specific platform, including capabilities, nuances, permissions, and dialect notes.
• Override: skip/set/add adjustments applied to a profile before running an assessment.

Extending

Adding a Requirement

1. Add an entry to requirements/requirements.yaml with: description, factor, scope, placeholders, implementations.
2. Create requirements/{name}/{platform}/ with three markdown files:
   • check.md (required) — context + SQL returning a value score 0–1
   • diagnostic.md (required) — context + SQL for detail drill-down
   • fix.md (required) — remediation SQL and/or organizational guidance
3. Add the requirement to relevant profile YAML(s) under the matching factor stage.

Adding a Profile

Create profiles/{name}.yaml with six stages, or use extends to derive from an existing one.

Adding a Platform

1. Create platforms/{PLATFORM}.md covering capabilities, dialect, permissions, and nuances.
2. Add requirement files under requirements/{key}/{platform}/.

Demos

See demos/README.md for available demo walkthroughs. Start with Scan + Agents for the full estate scan → deep assessment → remediation flow, or RAG Readiness for a focused single-schema assessment.

Structure

factors/                            # The six factors of AI-ready data (prose + requirements)
skills/
  ai-ready-data/
    SKILL.md                        # Orchestration protocol (Scan, Assess, Remediate)
    platforms/                      # Platform references
      {PLATFORM}.md                 # Capabilities, nuances, permissions, dialect
    requirements/                   # Requirement manifest + implementation directories
      requirements.yaml             # Single manifest (all requirement metadata)
      {requirement_key}/
        {platform}/
          check.md                # Context + check SQL (read-only, returns 0–1 score)
          diagnostic.md           # Context + diagnostic SQL (read-only detail)
          fix.md                  # Context + remediation SQL/guidance (mutating)
    profiles/                       # Assessment profiles
      scan.yaml                     # Estate-level scan (lightweight)
      rag.yaml
      feature-serving.yaml
      training.yaml
      agents.yaml

Contributors

License

All content and images are licensed under a CC BY 4.0 License

Code is licensed under the Apache 2.0 License