• 동기 부여
• 개요
• AI Agents를 위한 UC Metadata의 실제적인 이점
• 사용 가능한 도구 및 기능
• 설치
• 시스템 요구사항
• 설치
• 권한
• 서버 실행
• 독립 형태
• Cursor 사용
• 예제 사용 워크 플로우 (LLM Agent 용)
• Metadata를 Terraform으로 관리
• 핸들링
• 연혁

동기 부여

Databricks Unity Catalog (UC)는 카탈로그, 스키마, 테이블 및 열을 포함한 데이터 자산의 상세한 문서를 허용합니다. 이 자산을 완전히 문서화하면 시간의 투자가 필요합니다. 일반적인 질문은: 이 상세한 metadata 입장의 실제적인 이점은 무엇입니까?

이 MCP 서버는 그 노력에 대한 강력한 단정을 제공합니다. Big Language Models (LLMs)를 직접 액세스하고 Unity Catalog 메타데이터를 활용할 수 있습니다. 더 포괄적으로 데이터는 UC에 기술되어 더 효과적으로 LLM 에이전트가 Databricks 환경을 이해할 수 있습니다. 이 더 깊은 이해는 데이터 요청을 충족하기 위해 더 지능적이고 정확한 SQL 쿼리를 구성하는 에이전트에 중요합니다.

개요

Databricks MCP Server

• Motivation
• Overview
• Practical Benefits of UC Metadata for AI Agents
• Available Tools and Features
• Setup
  • System Requirements
  • Installation
• Permissions Requirements
• Running the Server
  • Standalone Mode
  • Using with Cursor
• Example Usage Workflow (for an LLM Agent)
• Managing Metadata as Code with Terraform
• Handling Long-Running Queries
• Dependencies

Motivation

Databricks Unity Catalog (UC) allows for detailed documentation of your data assets, including catalogs, schemas, tables, and columns. Documenting these assets thoroughly requires an investment of time. One common question is: what are the practical benefits of this detailed metadata entry?

This MCP server provides a strong justification for that effort. It enables Large Language Models (LLMs) to directly access and utilize this Unity Catalog metadata. The more comprehensively your data is described in UC, the more effectively an LLM agent can understand your Databricks environment. This deeper understanding is crucial for the agent to autonomously construct more intelligent and accurate SQL queries to fulfill data requests.

Overview

This Model Context Protocol (MCP) server is designed to interact with Databricks, with a strong focus on leveraging Unity Catalog (UC) metadata and enabling comprehensive data lineage exploration. The primary goal is to equip an AI agent with a comprehensive set of tools, enabling it to become independent in answering questions about your data. By autonomously exploring UC, understanding data structures, analyzing data lineage (including notebook and job dependencies), and executing SQL queries, the agent can fulfill data requests without direct human intervention for each step.

Beyond traditional catalog browsing, this server enables agents to discover and analyze the actual code that processes your data. Through enhanced lineage capabilities, agents can identify notebooks and jobs that read from or write to tables, then examine the actual transformation logic, business rules, and data quality checks implemented in those notebooks. This creates a powerful feedback loop where agents not only understand what data exists, but also how it’s processed and transformed.