destrayon/connapse

21 stars 0 forks Quality 58 Trend 31

Open-source, self-hosted knowledge backend for AI agents — hybrid search (vector + keyword), MCP server, 5 connectors, Docker-ready

Overview

Stop losing context between AI sessions. Give your agents persistent, searchable memory. Your AI agents forget everything between sessions. Connapse fixes that. Every time you start a new conversation, your AI agent starts from zero — no memory of past research, no access to your documents, no accumulated knowledge. Connapse is an open-source knowledge backend that gives agents persistent, searchable memory. Upload documents or point it at your existing Amazon S3 buckets or local filesystems. Agents query and build their own research corpus via 11 MCP tools, REST API, or CLI. Container-isolated, hybrid search (vector + keyword), self-hosted and private. Deploy in 60 seconds with Docker. Built on .NET 10. 🤖 AI Agent Integration — Claude queries and builds your knowledge base via MCP AI agents query your knowledge base through the MCP server, receiving structured answers with source citations from your documents.

README

Stop losing context between AI sessions. Give your agents persistent, searchable memory.

Your AI agents forget everything between sessions. Connapse fixes that.

Every time you start a new conversation, your AI agent starts from zero — no memory of past research, no access to your documents, no accumulated knowledge. Connapse is an open-source knowledge backend that gives agents persistent, searchable memory. Upload documents or point it at your existing Amazon S3 buckets or local filesystems. Agents query and build their own research corpus via 11 MCP tools, REST API, or CLI. Container-isolated, hybrid search (vector + keyword), self-hosted and private. Deploy in 60 seconds with Docker. Built on .NET 10.

📦 Quick Start

git clone https://github.com/Destrayon/Connapse.git && cd Connapse && docker-compose up -d
# Open http://localhost:5001

Prerequisites

Docker & Docker Compose
.NET 10 SDK (for development)
(Optional) Ollama for local embeddings

Run with Docker Compose

# Clone the repository
git clone https://github.com/Destrayon/Connapse.git
cd Connapse

# Set required auth environment variables (or use a .env file)
export [email protected]
export CONNAPSE_ADMIN_PASSWORD=YourSecurePassword123!
export Identity__Jwt__Secret=$(openssl rand -base64 64)

# Start all services (PostgreSQL, MinIO, Web App)
docker-compose up -d

# Open http://localhost:5001 — log in with the admin credentials above

The first run will:

Pull Docker images (~2-5 minutes)
Initialize PostgreSQL with pgvector extension and run EF Core migrations
Create MinIO buckets
Seed the admin account (from env vars) and start the web application

Development Setup

# Start infrastructure only (database + object storage)
docker-compose up -d postgres minio

# Run the web app locally
dotnet run --project src/Connapse.Web

# Run all tests
dotnet test

# Run just unit tests
dotnet test --filter "Category=Unit"

Command-line client

The connapse CLI is published separately at Destrayon/connapse-cli.

Install:

dotnet tool install -g Connapse.CLI
# or: download a binary from https://github.com/Destrayon/connapse-cli/releases/latest

Get started:

connapse auth login --server https://localhost:5001
connapse --help

Using with Claude (MCP)

Connapse includes a Model Context Protocol (MCP) server for integration with Claude and any MCP client.

Setup: Create an agent API key via the web UI (Settings → Agent API Keys), then add the config snippet for your client:

The MCP server exposes:

Tool	Description
`container_create`	Create a new container for organizing files
`container_list`	List all containers with document counts
`container_delete`	Delete a container
`container_stats`	Get container statistics (documents, chunks, storage, embeddings)
`upload_file`	Upload a single file to a container
`bulk_upload`	Upload up to 100 files in one operation
`list_files`	List files and folders at a path
`get_document`	Retrieve full parsed text content of a document
`delete_file`	Delete a single file from a container
`bulk_delete`	Delete up to 100 files in one operation
`search_knowledge`	Semantic, keyword, or hybrid search within a container

Full reference: See docs/mcp-tools.md for parameter tables, return formats, error cases, and usage examples.

Containers and sources: container_list tags each scope managed or source. A source is an external system Connapse mirrors read-only — searchable, but never browsable and never writable. list_files and every write tool resolve containers only; pass a source id and they report it as not found. See docs/connectors.md.

🚀 Features

🗂️ Container-Isolated Knowledge — Each project gets its own vector index, storage connector, and search configuration. No cross-contamination between projects, teams, or clients.
🔍 Hybrid Search — Vector similarity + keyword full-text with configurable fusion (convex combination, DBSF, AutoCut). Get results that pure vector search misses.
🧠 Multi-Provider AI — Swap between Ollama, OpenAI, Azure OpenAI, and Anthropic for both embeddings and LLM — at runtime, per container, without restarting.
🔌 Index Your Existing Storage — Connect MinIO, local filesystem (live file watching), or Amazon S3 (IAM auth). Your files stay where they are.
🤖 4 Access Surfaces — Web UI, REST API, CLI (native binaries), and MCP server for Claude. Built for humans, scripts, and AI agents equally.
🔐 Enterprise Auth — Multi-tier RBAC (Cookie + OAuth 2.1 + PAT + JWT) with AWS identity linking. Per-user filtering of search results by AWS permissions is enforced once identity linking is configured (opt-in; unrestricted until then).
🐳 One-Command Deploy — Docker Compose with PostgreSQL + pgvector, MinIO, and optional Ollama. Structured audit logging and rate limiting built in.

🎯 Who Is Connapse For?

AI agent developers who need a knowledge backend their agents can both query and build — upload research, curate a corpus, and search it via MCP or REST API
.NET / Azure teams who want a RAG platform that fits their existing stack and cloud identity
Enterprise teams who need project-isolated knowledge bases with proper RBAC and audit trails
Anyone tired of re-uploading files — point Connapse at your existing Amazon S3/filesystem storage

🏗️ Architecture

┌──────────────────────────────────────────────────────────────────────┐
│                         Access Surfaces                              │
│  Web UI (Blazor)  │  REST API  │  CLI  │  MCP Server                │
└─────────────┬────────────────────────────────────────────────────────┘
              │
┌─────────────▼────────────────────────────────────────────────────────┐
│                       Core Services Layer                            │
│  Document Store  │  Vector Store  │  Search  │  Ingestion           │
└─────────────┬────────────────────────────────────────────────────────┘
              │
┌─────────────▼────────────────────────────────────────────────────────┐
│                        Connectors Layer                              │
│  MinIO  │  Filesystem  │  Amazon S3  │                               │
└─────────────┬────────────────────────────────────────────────────────┘
              │
┌─────────────▼────────────────────────────────────────────────────────┐
│                        Infrastructure                                │
│  PostgreSQL+pgvector  │  MinIO (S3)  │  Ollama (optional)           │
└──────────────────────────────────────────────────────────────────────┘

Data Flow: Upload → Search

[Upload] → [Parse] → [Chunk] → [Embed] → [Store] → [Searchable]
              ↓
         [Metadata]
              ↓
        [Document Store]

Target: < 30 seconds from upload to searchable.

Key Technologies:

Database: PostgreSQL 17 + pgvector for vector embeddings
Object Storage: Managed Storage abstraction — MinIO (S3-compatible) by default, overridable per deployment
Backend: ASP.NET Core 10 Minimal APIs
Frontend: Blazor Server (interactive mode)
Embeddings: Ollama (default), OpenAI, Azure OpenAI (configurable)
LLM: Ollama, OpenAI, Azure OpenAI, Anthropic (configurable)
Search: Hybrid vector + keyword with convex combination fusion
Connectors: Managed Storage (MinIO default), Filesystem, Amazon S3

📚 Documentation

Architecture Guide - System design and component overview
API Reference - REST API endpoints and examples
Connectors Guide - Connector types, configuration, and background sync
AWS Setup - Roles Anywhere access, IAM Identity Center, S3 Access Grants, troubleshooting
Deployment Guide - Docker and production setup
Security Policy - Security limitations and roadmap
Contributing Guidelines - How to contribute

🗺️ Roadmap

Connapse is pre-1.0. Major design work is tracked in Discussions.

v0.1.0 — Foundation (Complete)

✅ Document ingestion pipeline (PDF, Office, Markdown, text)
✅ Hybrid search (vector + keyword with convex combination fusion)
✅ Container-based file browser with folders
✅ Web UI, REST API, CLI, MCP server

v0.2.0 — Security & Auth (Complete)

✅ Three-tier auth: Cookie + Personal Access Tokens + JWT (HS256)
✅ Role-based access control (Admin / Editor / Viewer / Agent)
✅ Invite-only user registration (admin-controlled)
✅ First-class agent entities with API key lifecycle
✅ Agent management UI + PAT management UI
✅ Audit logging (uploads, deletes, container operations)
✅ CLI auth commands (auth login, auth whoami, auth pat)
✅ GitHub Actions release pipeline (native binaries + NuGet tool)
✅ 256 passing tests (unit + integration)

v0.3.0 — Connector Architecture (Complete)

✅ 3 connector types: Managed Storage (MinIO default, provider-abstracted), Filesystem (FileSystemWatcher), Amazon S3 (IAM-only)
↩️ Azure Blob Storage connector (managed identity) — shipped in v0.3.0, since removed (#476): it was unverified and torn down for a clean rebuild
✅ Per-container settings overrides (chunking, embedding, search, upload)
↩️ Cloud identity linking: Azure AD (OAuth2+PKCE) — shipped in v0.3.0, since removed (#476) alongside the Azure Blob connector
↩️ AWS IAM Identity Center (device auth flow) — shipped in v0.3.0, since removed (#435): the device flow could not carry a per-user identity through to a token
⚠️ IAM-derived scope enforcement — the v0.3.0 CloudScopeService was built but never wired into search (#422) and has since been removed; per-user AWS search filtering was rebuilt on ISearchScopeResolver + AWS S3 Access Grants (epic #436) — opt-in, fail-closed once AWS identity linking is configured
✅ Multi-provider embeddings: Ollama, OpenAI, Azure OpenAI
✅ Multi-provider LLM: Ollama, OpenAI, Azure OpenAI, Anthropic
✅ Multi-dimension vector support with partial IVFFlat indexes per model
✅ Cross-model search: automatic Semantic→Hybrid fallback for legacy vectors
✅ Background sync: FileSystemWatcher for local, 5-min polling for cloud containers
✅ Connection testing for all providers (Amazon S3, MinIO, LLM, embeddings)
✅ 457 passing tests (unit + integration)

v0.3.2 — Hardening & Polish (Complete)

✅ Input validation hardening: filename length, path depth, control characters, search params, agent fields
✅ Security fixes: empty API key auth bypass, path traversal, security headers middleware
✅ Unified upload pipeline (IUploadService) shared by API and MCP
✅ File type allowlist for uploads
✅ Rate limiting middleware (per-user and per-IP)
✅ Bulk MCP tools: bulk_upload and bulk_delete
✅ CLI improvements: files commands, container stats, --pre updates, --help flags
✅ Self-hosted fonts (no CDN dependencies)
✅ Docker release package on ghcr.io

Future

v0.4.0: Communication connectors (Slack, Discord). (Per-user AWS search permissions — epic #436, via IAM Identity Center SAML identity linking, not Cognito — has already shipped.)
v0.5.0: Knowledge platform connectors (Notion, Confluence, GitHub)
v1.0.0: Production-ready stable release

❓ FAQ

Does Connapse require internet access? — No. Use Ollama for fully offline embeddings and search.

How many documents can it handle? — Thousands per container. Built on PostgreSQL + pgvector.

Which MCP clients work with Connapse? — Any client supporting Streamable HTTP transport — Claude Desktop, Claude Code, VS Code, Cursor, and others.

Is my data private? — Fully self-hosted. With Ollama, nothing leaves your machine. Cloud providers (OpenAI, Azure) are optional.

What embedding providers are supported? — Ollama (local), OpenAI, and Azure OpenAI. Switch at runtime without re-deploying.

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Quick contribution checklist:

Fork the repo and create a feature branch
Follow code conventions in CONTRIBUTING.md
Write tests for new features (xUnit + FluentAssertions)
Ensure all tests pass: dotnet test
Update documentation if needed
Submit a pull request

Good first issues: Check issues labeled good-first-issue

📄 License

This project is licensed under the MIT License - see LICENSE for details.

You are free to:

✅ Use commercially
✅ Modify
✅ Distribute
✅ Sublicense
✅ Use privately

The only requirement is to include the copyright notice and license in any substantial portions of the software.

💬 Support & Community

📖 Documentation: docs/
🐛 Bug Reports: GitHub Issues
💡 Feature Requests: GitHub Discussions
🔒 Security Issues: See SECURITY.md

🙏 Acknowledgments

Built with:

.NET - Application framework
Blazor - Web UI
PostgreSQL + pgvector - Vector database
MinIO - S3-compatible object storage
Ollama - Local LLM inference

⭐ If you find this project useful, please star the repository to show your support!

View the full README on GitHub

Install

This server does not publish a one-line install command.

Open the repository installation guide New to MCP? Start with the mcpskill.org guide →

Configuration

{
  "mcpServers": {
    "connapse": {
      "transport": "streamable-http",
      "url": "http://localhost:5001/mcp",
      "headers": {
        "X-Agent-Api-Key": "YOUR_API_KEY"
      }
    }
  }
}