CM

confluentinc/mcp-confluent

开发工具
164 stars 0 forks 质量 70 趋势 70

An open-source MCP server that enables AI assistants to interact with Confluent Cloud, Confluent Platform, and standalone Apache Kafka deployments through natural language.

概览

An open-source MCP server that enables AI assistants to interact with Confluent Cloud, Confluent Platform, and standalone Apache Kafka deployments through natural language.

README

Confluent MCP Server

An open-source MCP server that enables AI assistants to interact with Confluent Cloud, Confluent Platform, and standalone Apache Kafka deployments through natural language. It provides 50+ tools across Kafka, Flink SQL, Schema Registry, Connectors, Tableflow, and more – usable from any MCP-compatible client including Claude Desktop, Claude Code, Cursor, VS Code, Goose, and Gemini CLI.

[!TIP] Already a Confluent Cloud customer? Confluent offers a fully managed MCP server with no local server to run and no dependencies to install. It provides access to your Confluent Cloud resources with AI-powered connector diagnostics, governed by your existing RBAC permissions. Use this open-source server if you need Confluent Platform / self-managed Kafka support, or want to customize and extend the toolset.

NOTE: The open-source MCP server is a community-supported project. Confluent doesn’t provide dedicated support for it, and support is best-effort only, with no service level commitments. If you run into an issue or want to contribute, open an issue or pull request directly in this repository.

Quick Start

Prerequisites: Node.js 22.19.0+. If you want to interact with Confluent Cloud, you need to create an account first.

  1. Generate a quick config.yaml file in your project root:
npx @confluentinc/mcp-confluent --init-config
  1. Edit the config.yaml file with your connection details, then:
npx @confluentinc/mcp-confluent --config ./config.yaml

See Getting Started for full setup instructions and Configuring MCP Clients for integration with your preferred AI tool.

Table of Contents

Available Tools

Tools are auto-enabled based on which service blocks are present in your resolved configuration; see CONFIGURATION.md for the full block-to-tool mapping.

You can list all available tools via the CLI:

npx -y @confluentinc/mcp-confluent --list-tools

Always-Available Tools

These tools need no service blocks or authentication — they’re enabled even on a bare config, regardless of which deployment the rest of your config targets.

Category Tools Description
Documentation search-product-docs, get-product-doc-page Search Confluent product docs and fetch full page content
Diagnostics explain-disabled-tools, list-configured-connections, config-help, describe-configured-connection Explain why tools are absent, list configured connections and the tools enabled on each, suggest the YAML to enable a specific tool, and describe one connection’s config and tool availability

Available Tools for Confluent Cloud

These tools require endpoints and authentication against specific Confluent Cloud components. Refer to config.example.yaml for the full set of configuration variables. Categories marked with ¹ also work with OAuth authentication — sign in via your browser instead of provisioning API keys.

Category Tools Description
Kafka ¹ list-topics, create-topics, delete-topics, produce-message, consume-messages, list-consumer-groups, describe-consumer-group, get-consumer-group-lag, alter-topic-config, get-topic-config Manage topics, produce/consume messages, inspect consumer groups, configure topic settings
Flink SQL ¹ create-flink-statement, list-flink-statements, get-flink-statement-results, delete-flink-statements, get-flink-statement-exceptions, list-compute-pools Create and manage Flink SQL statements; discover Flink compute pools
Flink Catalog ¹ list-flink-catalogs, list-flink-databases, list-flink-tables, describe-flink-table, get-flink-table-info Explore Flink catalogs, databases, and table schemas
Flink Diagnostics ¹ check-flink-statement-health, detect-flink-statement-issues, get-flink-statement-profile Health checks, issue detection, and query profiling
Connectors ¹ list-connectors, get-connector-config, get-connector-offsets, get-connector-status, get-connector-tasks, get-connector-error-summary, get-connector-error-recommendations, get-connector-logs, create-connector ², delete-connector, pause-connector, resume-connector, restart-connector, update-connector-config Inspect and manage Kafka Connect connectors
Schema Registry ¹ list-schemas, create-schema, delete-schema List, inspect, create, and delete data schemas
Catalog & Tags ¹ search-topics-by-tag, search-topics-by-name, create-topic-tags, delete-tag, remove-tag-from-entity, add-tags-to-topic, list-tags Organize and search topics using tags
Organizations, Environments & Clusters ¹ list-organizations, list-environments, read-environment, list-clusters Discover Confluent Cloud resources
Tableflow ¹ create-tableflow-topic, list-tableflow-topics, read-tableflow-topic, update-tableflow-topic, delete-tableflow-topic, list-tableflow-regions Manage Tableflow-enabled topics
Tableflow Catalog ¹ create-tableflow-catalog-integration, list-tableflow-catalog-integrations, read-tableflow-catalog-integration, update-tableflow-catalog-integration, delete-tableflow-catalog-integration Manage Tableflow catalog integrations (e.g., AWS Glue)
Metrics ¹ list-available-metrics, query-metrics Discover and query Confluent Cloud operational metrics
Billing ¹ list-billing-costs Query billing and cost data

¹ Also available under OAuth — see OAuth Authentication for Confluent Cloud for setup and caveats. Categories not marked currently require a direct connection with static API keys; OAuth migration is in progress.

² Individual tool not available under OAuth; requires a direct connection with static API keys.

Available Tools for local deployments

These tools only require Kafka or Schema Registry endpoints - no Confluent Cloud API key/secret is needed. Ideal for local development with self-managed clusters, including Confluent Platform.

# minimal config.yaml for local development
connections:
  local:
    type: direct
    kafka:
      bootstrap_servers: "localhost:9092"
    schema_registry:
      endpoint: "http://localhost:8081"

Ready-to-use variants live in sample_configs/.

Category Tools Description
Kafka list-topics, create-topics, delete-topics, produce-message, consume-messages, list-consumer-groups, describe-consumer-group, get-consumer-group-lag Manage topics, produce/consume messages, inspect consumer groups
Schema Registry list-schemas, create-schema, delete-schema List, inspect, create, and delete data schemas

Using with Confluent Platform

mcp-confluent runs against a self-managed Confluent Platform (CP) cluster the same way it runs against any local Kafka + Schema Registry deployment: point a direct connection at your brokers and Schema Registry. A CP connection exposes the same tools as any other local deployment — see Available Tools for local deployments. The Confluent Cloud tools (Flink, Tableflow, Billing, Metrics, and the rest) require a Confluent Cloud account and stay disabled on CP. The only differences from a localhost:9092 setup are authentication and TLS.

Sample YAML config

sample_configs/confluent-platform.yaml is a copy-pasteable starter. It assumes PLAIN over SASL_SSL for Kafka and HTTP Basic Auth for Schema Registry. Customize the broker and Schema Registry URLs, and inject credentials via the ${KAFKA_API_KEY} / ${KAFKA_API_SECRET} / ${SCHEMA_REGISTRY_API_KEY} / ${SCHEMA_REGISTRY_API_SECRET} environment variables. If your cluster uses SCRAM or another SASL mechanism, override security.protocol and sasl.mechanisms through the kafka.extra_properties map in that file.

TLS trust (internal CAs)

CP clusters frequently sit behind an internal CA. If you see TLS handshake failures against the broker or Schema Registry, point Node at your CA bundle when starting the server:

NODE_EXTRA_CA_CERTS=/path/to/internal-ca.pem pnpm run start -- --config path/to/config.yaml

End-to-end smoke test

A docker-compose stack (docker-compose.cp-test.yml) brings up a local CP Kafka (KRaft, SASL_PLAINTEXT/PLAIN) plus an unauthenticated Schema Registry. The matching integration tests are tagged @cp and live next to their handlers as *.cp.integration.test.ts:

docker compose -f docker-compose.cp-test.yml up -d
# Wait ~30s for Kafka + SR to become ready, then:
CP_KAFKA_USERNAME=mcp CP_KAFKA_PASSWORD=mcp-secret \
  pnpm run test:integration --tags-filter=@cp
docker compose -f docker-compose.cp-test.yml down -v

The tests skip cleanly when those env vars are unset, so pnpm run test:unit and a default pnpm run test:integration against your real Confluent Cloud account are unaffected if you don’t have the docker stack running.

Getting Started

Prerequisites

  • Node.js 22.19.0 or later – we recommend using NVM to manage versions:
    nvm install 22
    nvm use 22
    
  • pnpm – only needed to build from source (the npx quick start above does not require it). On macOS the simplest install is Homebrew; npm works cross-platform:
    brew install pnpm        # macOS
    # or, cross-platform:
    npm install -g pnpm
    
    See pnpm’s installation guide for other options. The exact pnpm version is pinned in the packageManager field of package.json, and pnpm automatically runs that pinned version (via its built-in package-manager version management), so a recent pnpm install is all you need – no separate Corepack setup required.
  • A local environment with Kafka or Schema Registry running, or a Confluent Cloud account with appropriate API keys or login credentials if using OAuth to authenticate.

General Setup Steps

This MCP server is designed to be used with various MCP clients, such as Claude Desktop, Copilot, or Goose CLI/Desktop. The specific configuration and interaction will depend on the client you are using.

The MCP server can authenticate to Confluent Cloud via OAuth (PKCE) in addition to static API keys defined in the YAML config. See OAuth Authentication For Confluent Cloud for more details.

The general steps to configure (if not using OAuth) and run this MCP are:

  1. Create a configuration file: Copy the provided config.yaml example file to the root of your project. You can use the CLI to bootstrap one in your current directory — no git checkout required:
npx @confluentinc/mcp-confluent --init-config
  1. Populate the file: Fill in the necessary values for your Confluent Cloud environment. See CONFIGURATION.md for the full reference; only fill in the service blocks you need (each one enables a group of tools).

  2. Start the Server: You can run the MCP server in one of two ways:

    • From source: Follow the instructions in the Contributing Guide to build and run the server from source. This typically involves:

      • Installing dependencies (pnpm install)
      • Building the project (pnpm run build or pnpm run dev)
    • With npx: You can start the server directly using npx, no build required:

      npx @confluentinc/mcp-confluent --config /path/to/myconfig.yaml
      
  3. Configure your MCP Client: Each client (e.g., Claude, Goose) will have its own way of specifying the MCP server’s address and any required credentials. You’ll need to configure your client to connect to the address where this server is running (likely localhost with a specific port). The port the server runs on is set via server.http.port in config.yaml.

  4. Start your MCP Client: Once your client is configured to connect to the MCP server, you can start your MCP client and on startup it will stand up an instance of this MCP server locally. This instance will be responsible for managing data schemas and interacting with resources on your behalf.

  5. Interact with your resources through the Client: Once the client is connected and configured, you can use the client’s interface to interact with Confluent Cloud or local resources. The client will send requests to this MCP server, which will then interact with the available connections on your behalf.

Configuration

The full configuration reference — YAML schema, every service block, env-var interpolation, OAuth and HTTP/SSE auth setup, the (deprecated) legacy env-var table, and tool-to-block mapping — lives in CONFIGURATION.md.

Compatibility note. This release ships full parity between YAML (-c config.yaml) and the legacy env-var path (-e config.env) for a single connection. The env-var-only path will emit a startup warning in a near-future release and be removed a release or two later. Defining multiple connections (or none) is YAML-only — the env-var path can express only a single connection. See CONFIGURATION.md → Two paths, one configuration and CONFIGURATION.md → Multiple connections (and zero connections).

Prerequisites & setup for Tableflow commands

Tableflow tools interact with cloud storage (e.g. AWS S3) and a metadata catalog (e.g. AWS Glue) on your behalf via the Flink runtime in Confluent Cloud. The Flink runtime needs IAM permissions on your cloud account, and those have to be granted and linked into Confluent Cloud before any Tableflow tool will succeed.

Follow the Tableflow quick start with custom storage & Glue to set up the roles, policies, and provider integrations. Skipping this step leads to authorization errors when mcp-confluent tries to provision or manage Tableflow-enabled tables.

OAuth Authentication for Confluent Cloud

The MCP server can authenticate to Confluent Cloud via OAuth (PKCE) instead of static API keys. On the first tool call that needs Confluent access, the server opens your browser to the Confluent Cloud sign-in page; subsequent tool calls reuse the resulting session. No API keys to provision.

Setup

npx @confluentinc/mcp-confluent --init-oauth-config
# edit ./config.yaml if needed, then:
npx @confluentinc/mcp-confluent --config ./config.yaml

--init-oauth-config drops a starter config.oauth.example.yaml into ./config.yaml. The whole file is essentially:

connections:
  ccloud-oauth:
    type: oauth

See CONFIGURATION.md → Authentication modes for the full schema and ergonomics.

The ¹-marked categories in Available Tools for Confluent Cloud work under OAuth today; everything else still needs a direct connection with static API keys.

CLI Usage

The MCP server provides a flexible command line interface (CLI) for advanced control. The CLI lets you pick the config file, transports, and fine-tune which tools are enabled or blocked.

Basic Usage

You can view all CLI options and help with:

npx @confluentinc/mcp-confluent --help

Example: Deploy using all transports

npx @confluentinc/mcp-confluent -c config.yaml --transport http,sse,stdio

Example: Allow Only Specific Tools

npx @confluentinc/mcp-confluent -c config.yaml --allow-tools produce-message,consume-messages

Only the specified tools will be enabled; all others will be disabled.

Example: Block Certain Tools

npx @confluentinc/mcp-confluent -c config.yaml --block-tools produce-message,consume-messages

All tools except the specified ones will be enabled.

Example: Use Tool Lists from Files

You can also maintain allow/block lists in files (one tool name per line):

npx -y @confluentinc/mcp-confluent -c config.yaml --allow-tools-file allow.txt --block-tools-file block.txt

Example: List All Available Tools

npx -y @confluentinc/mcp-confluent --list-tools

Tip: The allow-list is applied before the block-list. If neither is provided, all tools are enabled by default.

Configuring MCP Clients

Please refer to the following guides for step-by-step instructions on setting up and using this MCP server with your preferred client:

Telemetry

This MCP server collects usage data to help make improvements. You can opt out by setting DO_NOT_TRACK=true in your environment. See telemetry.md for full details on what is collected.

Troubleshooting

“Node.js version not supported” – This project requires Node.js 22.19.0 or later. Check your version with node -v and upgrade if needed.

Tools not appearing – Each tool requires specific service blocks in your config.yaml. Run --list-tools to see which tools are active, or invoke the explain-disabled-tools MCP tool from your client for a per-tool reason. The block-to-tool mapping lives in CONFIGURATION.md.

Authentication errors on HTTP/SSE – Generate an API key with npx @confluentinc/mcp-confluent --generate-key and add it to your config.yaml under server.auth.api_key. See CONFIGURATION.md → HTTP/SSE transport security.

Connection refused / port conflicts – The default HTTP port is 8080. Set server.http.port in your config.yaml to change it.

Tableflow authorization errors – Tableflow tools require specific IAM permissions in your cloud environment. See Prerequisites & setup for Tableflow commands.

Contributing

Bug reports and feedback is appreciated in the form of Github Issues. For guidelines on contributing please see CONTRIBUTING.md

Pre-release testing

To run the MCP server against a pre-release version for beta testing or early feedback, download the release tarball file to a local directory. Then, when running any of the npx commands above, replace @confluentinc/mcp-confluent with the path to that tarball, e.g. npx @~path/to/my/tarball --list-tools

View this README on GitHub

安装

npx @confluentinc/mcp-confluent --init-config