MR

mxsm/rocketmq-rust

Developer tools
1.5Kย stars 0 forks Quality 67 Trend 67

๐Ÿš€ Apache RocketMQ built in Rust ๐Ÿฆ€ โ€” faster, safer, and more memory-efficient. Powering high-performance messaging for the AI era, with AI Agent, MCP, and AI-SRE capabilities.

Overview

๐Ÿš€ A high-performance, reliable, and feature-rich of Apache RocketMQ, designed to bring enterprise-grade message middleware to the Rust ecosystem. is a complete reimplementation of Apache RocketMQ in Rust, leveraging Rust's unique advantages in memory safety, zero-cost abstractions, and fearless concurrency. This project aims to provide Rust developers with a production-ready distributed message queue system that delivers exceptional performance while maintaining full compatibility with the RocketMQ protocol.

README

RocketMQ-Rust

๐Ÿš€ A high-performance, reliable, and feature-rich unofficial Rust implementation of Apache RocketMQ, designed to bring enterprise-grade message middleware to the Rust ecosystem.


โœจ Overview

RocketMQ-Rust is a complete reimplementation of Apache RocketMQ in Rust, leveraging Rustโ€™s unique advantages in memory safety, zero-cost abstractions, and fearless concurrency. This project aims to provide Rust developers with a production-ready distributed message queue system that delivers exceptional performance while maintaining full compatibility with the RocketMQ protocol.

๐ŸŽฏ Why RocketMQ-Rust?

  • ๐Ÿฆ€ Memory Safety: Built on Rustโ€™s ownership model, eliminating entire classes of bugs like null pointer dereferences, buffer overflows, and data races at compile time
  • โšก High Performance: Zero-cost abstractions and efficient async runtime deliver exceptional throughput with minimal resource overhead
  • ๐Ÿ”’ Thread Safety: Fearless concurrency enables safe parallel processing without the risk of race conditions
  • ๐ŸŒ Cross-Platform: First-class support for Linux, Windows, and macOS with native performance on each platform
  • ๐Ÿ”Œ Ecosystem Integration: Seamlessly integrates with the rich Rust ecosystem including Tokio, Serde, and other modern libraries
  • ๐Ÿ“ฆ Production Ready: Battle-tested architecture with comprehensive error handling and observability

๐Ÿ—๏ธ Architecture

RocketMQ-Rust implements a distributed architecture with the following core components:

  • Name Server: Lightweight service discovery and routing coordination
  • Broker: Message storage and delivery engine with support for topics, queues, and consumer groups
  • Producer Client: High-performance message publishing with various sending modes
  • Consumer Client: Flexible message consumption with push and pull models
  • Store: Efficient local storage engine optimized for sequential writes
  • Controller: Advanced high availability and failover capabilities

๐Ÿ“š Documentation

๐Ÿš€ Quick Start

Prerequisites

  • Rust toolchain 1.95.0
  • A shell with cargo available
  • Separate terminals for the NameServer, Broker, and client examples

See the toolchain and dependency trust policy for the pinned toolchain, dependency admission rules, and local validation contract.

1. Build the Workspace

git clone https://github.com/mxsm/rocketmq-rust.git
cd rocketmq-rust
cargo build --workspace

If you only want to use the client SDK from your own application, add the current release to Cargo.toml:

[dependencies]
rocketmq-client-rust = "1.0.0"
rocketmq-model = "1.0.0"
rocketmq-protocol = "1.0.0"

2. Start the NameServer

cargo run --bin rocketmq-namesrv-rust

The default NameServer endpoint is 127.0.0.1:9876. To bind explicitly:

cargo run --bin rocketmq-namesrv-rust -- --ip 127.0.0.1 --port 9876

3. Start the Broker

The Broker requires ROCKETMQ_HOME. Point it at an existing RocketMQ home or create a local runtime directory for quick testing.

Linux/macOS:

export ROCKETMQ_HOME="$(pwd)/.rocketmq"
mkdir -p "$ROCKETMQ_HOME/conf"
cargo run --bin rocketmq-broker-rust -- -n 127.0.0.1:9876

Windows PowerShell:

$env:ROCKETMQ_HOME = "$PWD\.rocketmq"
New-Item -ItemType Directory -Force "$env:ROCKETMQ_HOME\conf" | Out-Null
cargo run --bin rocketmq-broker-rust -- -n 127.0.0.1:9876

Use cargo run --bin rocketmq-broker-rust -- --help to inspect configuration flags such as --configFile, --namesrvAddr, and config printing options.

4. Send and Receive Messages

Start the consumer example first:

cargo run -p rocketmq-client-rust --example consumer

Then send messages from another terminal:

cargo run -p rocketmq-client-rust --example producer

The quickstart examples use 127.0.0.1:9876 and TopicTest by default. For more messaging patterns, see:

๐Ÿ“ฆ Components & Crates

RocketMQ-Rust is organized into deployable services, reusable protocol/runtime crates, and operational applications. The tables below focus on responsibility and integration boundaries instead of per-crate maturity labels.

Core Runtime Services

Crate Responsibility
rocketmq-namesrv NameServer implementation for broker registration, topic routing, and service discovery.
rocketmq-broker Broker implementation for message storage, dispatch, delivery, and consumer coordination.
rocketmq-controller Controller service for broker coordination and high availability workflows.
rocketmq-proxy Proxy layer for gateway-style client access and protocol integration.
rocketmq-proxy-core Stable proxy contracts, use cases, and ingress-independent models.
rocketmq-proxy-cluster Cluster-mode proxy adapter with keyed execution and remote Broker access.
rocketmq-proxy-local Local-mode proxy adapter.

Client, Protocol, and Shared Libraries

Crate Responsibility
rocketmq-client Async producer, consumer, and admin SDK for application integration.
rocketmq-protocol RocketMQ wire commands, headers, serialization, and compatibility contracts.
rocketmq-transport Runtime-owned network sessions and canonical per-session writers.
rocketmq-model Shared message, route, configuration, and domain models.
rocketmq-auth Authentication, authorization, ACL evaluation, and request context support.
rocketmq-security-api Runtime-neutral authentication, authorization, signing, and maintenance contracts.
rocketmq-filter Message filtering support, including tag and expression-based filtering.

Storage, Runtime, and Observability

Crate Responsibility
rocketmq-store Durable local storage engine for commit logs, consume queues, and message indexes.
rocketmq-store-api Backend-neutral append, read, lifecycle, replication, checkpoint, and health capabilities.
rocketmq-store-local Local CommitLog, ConsumeQueue, Index, mapped-file, and recovery implementation.
rocketmq-store-rocksdb RocksDB-backed store implementation.
rocketmq-tieredstore Tiered storage abstractions for extending message data beyond local disks.
rocketmq-runtime Async runtime abstractions and runtime-friendly coordination utilities.
rocketmq-error Shared error types and result conventions across workspace crates.
rocketmq-macros Procedural macros used by RocketMQ-Rust crates and examples.
rocketmq-observability Metrics and tracing integration for service and client instrumentation.

The cross-cutting ownership, cancellation, error, limit, compatibility, and failure-mode contracts are documented in the core capability contracts.

Tools, Examples, and Dashboards

Project Responsibility
rocketmq-example Standalone examples covering producer, consumer, request/reply, ordering, delay, and transaction flows.
rocketmq-tools Command-line tools and operational utilities.
rocketmq-admin-cli Command-line administration interface for cluster and broker operations.
rocketmq-admin-core Shared admin functionality used by CLI and terminal interfaces.
rocketmq-admin-tui Terminal UI for interactive administration workflows.
rocketmq-store-inspect Storage inspection utilities for broker data files.
rocketmq-mcp Model Context Protocol server for deny-by-default RocketMQ diagnostics and administration.
rocketmq-dashboard Dashboard workspace for desktop, web, and shared management UI components.
rocketmq-dashboard-common Shared dashboard models and reusable dashboard infrastructure.
rocketmq-dashboard-gpui GPUI-based desktop dashboard.
rocketmq-dashboard-tauri Tauri-based cross-platform dashboard shell and backend.
rocketmq-dashboard-web Web dashboard frontend and backend project.

๐Ÿ’ก Capabilities

RocketMQ-Rust focuses on RocketMQ-compatible messaging services and Rust-native integration points.

Area What it provides
Messaging services NameServer, Broker, Controller, and Proxy services for routing, storage, delivery, coordination, and gateway access.
Client integration Async producer, consumer, admin, request/reply, batch, ordered, delayed, and transactional messaging APIs.
Protocol compatibility RocketMQ remoting command models, headers, serialization, route discovery, and client/broker interoperability.
Storage engine Durable commit log, consume queue, index, checkpoint, and tiered storage building blocks.
Security and governance Authentication, authorization, ACL evaluation, request context, and broker/client-side integration points.
Operations Metrics, tracing, admin tools, storage inspection utilities, and dashboard projects for cluster visibility.

๐Ÿงช Build & Validation

Quick Start covers the first local run. For regular development and review, use the root workspace commands below.

Task Command
Build the workspace cargo build --workspace
Run workspace tests cargo test --workspace
Run a focused crate test cargo test -p rocketmq-client
Format Rust code cargo fmt --all
Run clippy with workspace features cargo clippy --workspace --no-deps --all-targets --all-features -- -D warnings
Build local API documentation cargo doc --workspace --no-deps

Standalone projects under rocketmq-example/ and rocketmq-dashboard/ are validated from their own project roots.

๐Ÿค Contributing

We welcome contributions from the community! Whether youโ€™re fixing bugs, adding features, improving documentation, or sharing ideas, your input is valuable.

How to Contribute

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Contribution Guidelines

  • Follow Rust best practices and idiomatic patterns
  • Add tests for new functionality
  • Update documentation as needed
  • Ensure CI passes before submitting PR
  • Use meaningful commit messages

For detailed guidelines, please read our Contribution Guide.

Repository Activity

โ“ FAQ

๐Ÿ‘ฅ Community & Support

Contributors

Thanks to all our contributors! ๐Ÿ™

๐Ÿ“„ License

RocketMQ-Rust is licensed under the Apache License 2.0.

See LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0.

๐Ÿ™ Acknowledgments

  • Apache RocketMQ Community for the original Java implementation and design
  • Rust Community for excellent tooling and libraries
  • All Contributors who have helped make this project better

Built with โค๏ธ by the RocketMQ-Rust community

View this README on GitHub

Install

This server does not publish a one-line install command.

Open the repository installation guide