voitta-rag

A self-hosted knowledge base that indexes your documents and code repositories, making them searchable through semantic (RAG) search. Connect it to AI coding assistants like Claude Code via MCP, or use the built-in web UI to browse and search across all your indexed content.

Useful for teams and individuals who want to:

• Search across codebases and docs — index Git repos, Google Drive folders, Jira boards, and office documents in one place
• Give AI assistants context — expose your indexed knowledge to Claude Code or other MCP-compatible tools
• Keep everything local — runs on your infrastructure with Qdrant for vector storage, no data leaves your network

Table of Contents

• Features
• Search Scope
• Prerequisites
  • Python
  • Dependencies Note
• Quick Start
  • Option A: Docker (recommended)
    • Mounting local directories (Mapped Paths)
  • Option B: Local development
• Configuration
  • Authentication
• MCP Server (for Claude Code integration)
  • Claude Code Plugin (automated setup)
  • Manual Claude Code Configuration
  • Available MCP Tools
  • Importing Claude Code Session History
• Concepts
  • Projects
  • Toggle Switches
  • Anamnesis
• Bulk Repository Import
  • Config format
  • Running against Docker
  • Customizing
  • Exporting from a running instance
  • Round-trip between machines
• Advanced
  • SQLite journal mode on Docker bind-mounts
  • Git post-commit hook

Features

• File browser with real-time updates
• Folder creation and file upload
• Data source connectors: Filesystem (local path mapping), Git, Google Drive, SharePoint, Azure DevOps, Jira, Confluence, Box
• Jira/Confluence support for both Cloud and Server/Data Center deployments
• Per-user search scope control per folder
• Automatic document indexing (DOCX, PPTX, XLSX, ODT, ODP, ODS, GDOC, GSHEET, GSLIDES) — see Google Drive indexing for how Google Workspace files become searchable
• Vector search with Qdrant (hybrid semantic + keyword, with time range filtering)
• MCP server for Claude Code integration
• Source URL resolution: resolve Google Docs/Sheets/Slides URLs to indexed content via MCP
• Anamnesis: persistent RAG memory for AI assistants (create, retrieve, like/dislike memories)
• File change detection via content hashing
• Global file/folder metadata
• Dark/light theme support

Search Scope

Each folder in the file browser has a Search toggle that controls whether its content is included in search results.

When the toggle is on (green), the folder’s documents are returned by search queries. When off (grey), the folder remains indexed but is invisible to search.

This is useful when you work on multiple unrelated projects. For example, suppose you have two folders indexed: a-project (your current client engagement) and b-project (an internal tool). While working on a-project, you don’t want search results polluted with code and docs from b-project – unrelated hits add noise and can mislead an AI assistant that consumes the results. Toggle b-project off, and searches only return content from a-project. When you switch contexts, flip the toggles.

The setting is per-user and per-project – each user can have their own search scope without affecting others.

Prerequisites

Python

Requires Python 3.11+

If using pyenv, ensure these system libraries are installed first:

# Ubuntu/Debian
sudo apt-get install -y build-essential libssl-dev zlib1g-dev \
  libbz2-dev libreadline-dev libsqlite3-dev curl git \
  libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev \
  libffi-dev liblzma-dev

# Then install Python
pyenv install 3.12

Dependencies Note

The transformers library must be version 4.x (not 5.x) due to compatibility with sentence-transformers. This is already constrained in requirements.txt:

transformers>=4.36.0,<5.0.0

If you encounter ModuleNotFoundError: Could not import module 'PreTrainedModel', downgrade transformers:

pip install "transformers>=4.36.0,<5.0.0"

Quick Start

Option A: Docker (recommended)

Runs both voitta-rag and Qdrant in containers. No Python installation needed on the host. The web UI is at port 58000, the MCP endpoint at /mcp/mcp on the same port.

cp .env.example .env
make docker-build
make docker-up

Open http://localhost:58000 in your browser. Stop with make docker-down. Rebuild the image after code changes with make docker-build.

By default, ~/.ssh is mounted read-only into the container for SSH-based git access. Override with:

SSH_KEY_DIR=/path/to/ssh/keys docker compose up -d --build

Mounting local directories (Mapped Paths)

In Docker mode, local directories are mounted into the container via docker-compose.override.yml (gitignored, merged automatically by Docker Compose). Each mounted directory appears automatically in the UI as a folder with a “Mapped Path” badge. The directory name becomes the folder name, so use descriptive names.

For example, if Google Drive syncs to ~/g/gdrive:

services:
  voitta-rag:
    volumes:
      - ~/g/gdrive:/data/fs/gdrive:ro

The folder appears as “gdrive” in the UI. Enable indexing on it like any other folder. The file watcher detects changes automatically.

Multiple directories:

services:
  voitta-rag:
    volumes:
      - ~/g/gdrive:/data/fs/gdrive:ro
      - ~/Dropbox/Projects:/data/fs/dropbox-projects:ro

Mapped Path folders cannot be created or deleted from the UI – they are managed entirely through volume mounts. Upload is also disabled for these folders since the source of truth is the host directory. Restart with make docker-up after changing mounts.

Note: symlinks inside mounted volumes won’t work – Docker doesn’t resolve symlink targets across mount boundaries. Use volume mounts instead.

This approach complements the Bulk Repository Import — use volume mounts for local/synced folders and the import script for Git repositories.

Option B: Local development

Runs voitta-rag directly with Python on your machine. Requires Python 3.11+ (see Prerequisites). Qdrant still runs in Docker. The web UI is at port 8000, the MCP endpoint at /mcp/mcp on the same port.

# Start Qdrant (vector database)
mkdir -p qdrant_storage
docker run -d --name qdrant \
  -p 6333:6333 -p 6334:6334 \
  -v $(pwd)/qdrant_storage:/qdrant/storage \
  qdrant/qdrant

# Install Python dependencies and run
make install
cp .env.example .env
make run

Open http://localhost:8000 in your browser.

Key difference from Docker mode: In local mode, voitta-rag has direct filesystem access — no volume mounts needed. Set VOITTA_ROOT_PATH in .env to the directory where indexed data should be stored.

Configuration

Key settings in .env:

# Root folder for managed files
ROOT_PATH=/mnt/ssddata/data/voitta-rag-data

# Qdrant connection
QDRANT_HOST=localhost
QDRANT_PORT=6333

# Embedding model (uses GPU if available)
EMBEDDING_MODEL=intfloat/e5-base-v2

# Chunking settings
CHUNK_SIZE=512
CHUNK_OVERLAP=50

# Indexing worker poll interval (seconds)
INDEXING_POLL_INTERVAL=10

# MCP server port
MCP_PORT=8001

# Microsoft login (Azure AD / Entra ID) — optional
MS_AUTH_TENANT_ID=
MS_AUTH_CLIENT_ID=
MS_AUTH_CLIENT_SECRET=

# Google login (OAuth2) — optional
GOOGLE_AUTH_CLIENT_ID=
GOOGLE_AUTH_CLIENT_SECRET=

# Base URL for OAuth redirect callbacks
VOITTA_BASE_URL=https://your-domain.com

Authentication

The web UI supports optional OAuth login via Microsoft (Azure AD) and/or Google. Set the corresponding env vars to enable each provider. When any provider is configured, the landing page shows login buttons instead of the user picker.

For Microsoft, you need an Azure AD app registration with redirect URI {VOITTA_BASE_URL}/auth/microsoft/callback. For Google, create OAuth credentials in Google Cloud Console with redirect URI {VOITTA_BASE_URL}/auth/google/callback.

The MCP server validates tokens independently via X-Auth-Token-Microsoft and X-Auth-Token-Google headers — calling Microsoft Graph /me and Google userinfo respectively. Every tool response includes an _auth block with per-provider validation status.

MCP Server (for Claude Code integration)

The MCP server runs embedded in the main app (no separate process needed) and exposes RAG capabilities via the MCP protocol.

Claude Code Plugin (automated setup)

voitta-rag ships as a Claude Code plugin. This repo is its own marketplace, so install with:

/plugin marketplace add voitta-ai/voitta-rag
/plugin install voitta-rag@voitta-rag

The MCP server URL depends on your deployment (Docker vs local), so after installing the plugin run the setup script to configure it:

# Docker mode (port 58000)
bash claude-plugin/setup.sh --docker

# Local mode (port 8000)
bash claude-plugin/setup.sh

# With session memory hook (saves a session summary on exit)
bash claude-plugin/setup.sh --docker --with-hook

See claude-plugin/README.md for details.

Manual Claude Code Configuration

Add to ~/.claude.json under mcpServers (global) or in your project settings:

{
  "mcpServers": {
    "voitta-rag": {
      "type": "http",
      "url": "http://localhost:58000/mcp/mcp"
    }
  }
}

Note: The URL path is /mcp/mcp — FastMCP creates its endpoint at /mcp inside the app, which is itself mounted at /mcp. If running locally (not Docker), replace 58000 with 8000.

Available MCP Tools

Importing Claude Code Session History

Import your past Claude Code sessions as searchable memories:

python3 scripts/import_claude_history.py

This parses ~/.claude/history.jsonl, groups prompts by session, and creates one memory per session. Filter what gets imported:

# Only sessions in a specific project
python3 scripts/import_claude_history.py --project /path/to/project

# Only sessions after a date
python3 scripts/import_claude_history.py --after 2025-06-01

# Only sessions mentioning a keyword
python3 scripts/import_claude_history.py --keyword "DynamoDB"

# Preview without importing
python3 scripts/import_claude_history.py --dry-run

Note: Only user prompts are stored locally by Claude Code; assistant responses are not available. Despite this, user prompts provide good semantic search targets for recalling past work.

Concepts

Projects

The project dropdown in the toolbar lets you organize which indexed folders are included in MCP search results. Each user gets a Default project automatically. You can create additional projects to group folders for different contexts — e.g., “Backend”, “Client Docs”, “Research”.

• Switching projects changes which folders’ search toggles are active. A folder can be search-active in one project but not in another.
• Default project stores search-active states globally in the user’s folder settings. Non-default projects store their own independent set of search-active states.
• The active project persists across sessions and is used by the MCP server to determine which folders to search.
• Selecting “Manage Projects…” opens a modal to create or delete projects. The Default project cannot be deleted.

Toggle Switches

Two independent toggles control folder behavior:

Enable for Indexing (sidebar toggle) — controls whether the background indexing worker processes a folder. When enabled, the folder is queued with “Pending” status and the worker will index its files into the vector store. Disabling the toggle stops future indexing but does not remove already-indexed content.

Search (inline checkbox in the file list) — controls whether a folder’s indexed content is included in MCP search results. This is project-scoped: toggling it applies only to the currently active project. Toggling a parent folder applies recursively to all subfolders. A folder must be both indexed and search-active to appear in search results.

Anamnesis

Anamnesis (Greek for “remembering”) is a persistent memory system for AI assistants. Each user has an Anamnesis// folder containing markdown files with YAML frontmatter that store memories created and managed through MCP tools (create_memory, get_memory, like_memory, etc.).

The Anamnesis folder is read-only in the web UI — you cannot upload, create, delete, or modify files through the browser. It is managed exclusively through MCP tools. The UI shows the contents for browsing and inspection only. Memories can be liked or disliked to influence their relevance in search results.

Bulk Repository Import

Import multiple Git repositories at once using a JSON config file:

python3 scripts/import_repos.py [path/to/config.json]

Defaults to scripts/import_repos.json. An example config is provided at scripts/import_repos.example.json showing the format (per-host auth plus folders-of-repos).

Copy it to get started:

cp scripts/import_repos.example.json scripts/import_repos.json
python3 scripts/import_repos.py

Config format

The config specifies per-host auth and folders with repo lists:

{
    "hosts": {
        "github.com": {"auth_method": "ssh"},
        "git.example.com": {
            "auth_method": "token",
            "username": "your-username",
            "token": "your-pat-token"
        }
    },
    "folders": {
        "my-repos": [
            {"repo": "[email protected]:org/repo.git"},
            {"repo": "https://git.example.com/team/project.git", "branch": "develop"}
        ]
    }
}

Folders map to top-level folders in the voitta-rag UI. A common pattern is one folder per GitHub organization, so repos appear as org-name/repo-name in the file browser.

Auth methods:

• ssh — uses SSH keys mounted into the container (default: ~/.ssh). No credentials in the config file.
• token — uses a personal access token. Include username and token fields in the host entry.

Branch is auto-detected from the remote when not specified (prefers main, then master, then develop).

Running against Docker

When running voitta-rag via Docker Compose (make docker-up), the script connects to localhost:58000 by default (reads DOCKER_PORT from .env):

python3 scripts/import_repos.py scripts/import_repos_personal.json

The script waits for the server to be ready, creates folders, configures sync, and polls until each repo finishes syncing. Already-imported repos are skipped automatically.

Customizing

To add or remove repos, edit the folders entries. Each repo entry needs at minimum a repo URL:

{"repo": "[email protected]:your-org/new-repo.git"}

To use a specific branch instead of auto-detection, add "branch": "branch-name".

Both import_repos.json and import_repos_personal.json are gitignored since they may contain credentials. Only scripts/import_repos.example.json is committed.

Exporting from a running instance

The inverse of import. Walks the current voitta-rag folders, finds subfolders with a GitHub sync source, and writes a JSON file in the same format import_repos.py consumes:

python3 scripts/export_repos.py [output_path]

Defaults to scripts/import_repos_personal.json (gitignored). Only github sync sources are exported – non-github connectors (Google Drive, SharePoint, Jira, etc.) are skipped with a count in the summary.

Secrets are never written. The output records only auth_method per host. If you use token auth, you must re-add username and token to each hosts entry on the target machine before running import_repos.py.

Round-trip between machines

Use the export/import pair to move your repo set from one voitta-rag instance to another:

# Machine A (source): export current state
python3 scripts/export_repos.py /tmp/repos.json
scp /tmp/repos.json user@machine-b:/tmp/

# Machine B (target): import
cd /path/to/voitta-rag
make docker-build && make docker-up
python3 scripts/import_repos.py /tmp/repos.json

If the target uses token auth, edit /tmp/repos.json on machine B to fill in username and token under each hosts entry before running the import. For SSH auth, make sure the target machine’s SSH key is present on ~/.ssh (mounted into the container) and authorized on GitHub.

Advanced

SQLite journal mode on Docker bind-mounts

For operators only. Skip this unless you hit the symptom below.

voitta-rag stores its operational metadata in a SQLite database (voitta.db). By default SQLite picks rollback journaling (mode delete), which is the safe choice when the file lives on a Docker bind-mount from a macOS or Windows host. The application does not force a journal mode — it leaves whatever the file header specifies.

Symptom: under sustained use on a bind-mounted setup (Docker Desktop, Rancher Desktop, OrbStack on macOS or Windows), some HTTP endpoints intermittently fail with:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) disk I/O error

while sqlite3 voitta.db "PRAGMA integrity_check;" returns ok and the host can read the same query without issue. Restarting the container temporarily fixes it, then it comes back.

Root cause: the database is using SQLite’s WAL (write-ahead log) journal mode. WAL relies on a -shm shared-memory page that does not cross the host/VM file-sharing boundary cleanly. Stale -shm mappings — left over from a previous container session, laptop sleep, or a host-side process that touched the file — surface as EIO on the next query.

Fix: force the database to use rollback journaling (mode delete) once. The setting is persisted in the SQLite file header and survives container restarts.

# 1. Stop the voitta-rag container so nothing has the DB open.
docker compose stop voitta-rag

# 2. Truncate any pending WAL and switch journal_mode on the file.
sqlite3 $VOITTA_ROOT_PATH/voitta.db "PRAGMA wal_checkpoint(TRUNCATE);"
sqlite3 $VOITTA_ROOT_PATH/voitta.db "PRAGMA journal_mode = DELETE;"

# 3. Confirm the change stuck.
sqlite3 $VOITTA_ROOT_PATH/voitta.db "PRAGMA journal_mode;"   # prints: delete

# 4. Bring the container back up.
docker compose start voitta-rag

Trade-off: rollback journaling serializes writes against reads, so on a single-writer / many-reader workload like voitta-rag the cost is modest. If your workload is write-heavy enough that you actually need WAL throughput and you are not on a bind-mount (e.g. you’ve moved the DB into a Docker named volume), you can force WAL the same way:

sqlite3 voitta.db "PRAGMA journal_mode = WAL;"

See also the SQLite docs on journal mode and WAL mode caveats on networked filesystems.

Git post-commit hook

For operators only. Skip this if your Git sources reindex on a schedule and you don’t need fresh data after each commit.

If you have a voitta-rag Git source pointed at a repository you actively work on, you can wire a post-commit hook in that repository so every local commit immediately triggers a sync (which also re-runs the optional llm-tldr static analysis from issue #15). This keeps the indexed snapshot — both raw code chunks and structural analysis chunks — in lockstep with HEAD on your machine without waiting for the next scheduled sync.

1. Enable the hook endpoint on the voitta-rag server.

The hook calls POST /api/sync/_hook/sync/{folder_path} which is authenticated by a shared secret in the X-Voitta-Hook-Secret header, not by browser cookie. The route is disabled (returns 403) when the secret env var is unset, so this is strictly opt-in.

In your voitta-rag .env:

VOITTA_HOOK_SECRET=

Restart the container so the new env var is picked up:

docker compose restart voitta-rag

2. Install the hook in your repo.

Copy scripts/git-hooks/post-commit from this repo into your target repo’s .git/hooks/ directory and make it executable:

cp /path/to/voitta-rag/scripts/git-hooks/post-commit \
   /path/to/my-repo/.git/hooks/post-commit
chmod +x /path/to/my-repo/.git/hooks/post-commit

If you maintain shared hooks across multiple repos via core.hooksPath or a tool like husky, drop the script there instead.

3. Tell the hook how to reach your voitta-rag.

The hook reads three environment variables. Set them in your shell rc file (~/.bashrc, ~/.zshrc, etc.) so they’re available whenever you commit:

export VOITTA_RAG_URL=http://localhost:58000
export VOITTA_RAG_FOLDER=my-repo              # the voitta-rag folder_path
export VOITTA_HOOK_SECRET=

The hook bails out silently when any of these is unset, so the same script can live in repos that aren’t synced to voitta-rag.

4. Verify.

Make a commit. You should see a line like:

[voitta-rag] {"folder_path":"my-repo","status":"syncing","message":"Sync started"}

Set VOITTA_HOOK_QUIET=1 to suppress that line if your commit output must stay clean.

5. Watch the sync progress.

Sync runs in the background on the voitta-rag side; the hook returns immediately. Tail the server’s app log to confirm:

docker exec  tail -f logs/app.log | grep -E "sync|llm-tldr"

Notes

• The hook exits 0 unconditionally on any error so a voitta-rag outage never blocks a commit.
• The hook never reads your commit content; it just pings the API.
• The endpoint is HTTP only — terminate TLS at a reverse proxy if your voitta-rag listens on the public network.