zylon-ai/private-gpt

PrivateGPT 是开源的 API 层，将本地模型转化为生产型 AI 应用。

---

本地运行模型只是第一步。要构建有用的人工智能应用，你需要一套更高级的构建模块。PrivateGPT作为一个基于Claude API模型的开源API，提供该层——这样你就可以构建私有AI产品，而无需从零重建相同的后端原语，也无需依赖云API。

经过生产测试：PrivateGPT为Zylon提供支持，Zylon是本地部署的AI平台，为全球企业提供私有AI。

PrivateGPT 不运行模型本身。它通过 OPENAI_API_BASE 连接到任何兼容 OpenAI 的推理服务器。如果它实现了 /v1/chat/completions 和 /v1/models，它就能正常工作。

PrivateGPT 内置了用于测试和演示的工作台界面，可在 /ui 获取。API 是实际产品。

---

PrivateGPT 给你的

• 标准消息API（流式流、异步、令牌计数）
• 文件和文物摄取
• 检索，附引用和代理RAG
• 内置镜像 Claude API 的工具（网页搜索、网页抓取、代码执行）
• 自定义工具和MCP连接器

---

Running a model locally is only the first step. To build useful AI applications you need a set of higher-level building blocks. PrivateGPT provides that layer as an open-source API following the Claude API model — so you can build private AI products without rebuilding the same backend primitives from scratch, and without depending on cloud APIs.

Production-tested: PrivateGPT powers Zylon, the on-premise AI platform providing Private AI to enterprises across the globe.

Your app / agent / workflow / UI
              |
        PrivateGPT API
              |
OpenAI-compatible inference server (Ollama, llama.cpp, vLLM, …)              

PrivateGPT does not run models itself. It connects to any OpenAI-compatible inference server via OPENAI_API_BASE. If it implements /v1/chat/completions and /v1/models, it works.

PrivateGPT ships a built-in workbench UI for testing and demos, available at /ui. The API is the actual product.

---

What PrivateGPT gives you

• Standard messages API (streaming, async, token counting)
• File and artifact ingestion
• Retrieval with citations and agentic RAG
• Built-in tools mirroring the Claude API (web search, web fetch, code execution)
• Custom tools and MCP connectors
• Structured access to databases and CSVs
• Embeddings and orchestration

---

Quickstart

For Docker, full installation options, and model configuration see the full Quickstart guide.

Prerequisites: You need a running OpenAI-compatible LLM server. Ollama is the easiest starting point.

1. Install PrivateGPT

# macOS
brew tap zylon-ai/tap
brew install private-gpt

# Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

uv tool install --python 3.11 \
  --find-links https://wheels.privategpt.dev/packages/ \
  "private-gpt[core]"