LLM Proxy

One gateway for every LLM. Any client protocol in, any provider out — with smart routing, cost control, and a full admin console.

LLM Proxy is a self-hostable LLM API gateway. Point your existing OpenAI or Anthropic SDK at it and instantly gain access to every configured provider — with automatic failover, cost-aware routing, per-key budgets, request logging, and an admin dashboard to manage it all without restarts.

Documentation

**Full documentation: **

Why LLM Proxy?

• Write once, run on any provider — Your app speaks one protocol (OpenAI or Anthropic); the proxy translates bidirectionally to every upstream. Switch providers by changing a model name, not your code.
• Never hard-down — Priority-based provider fallback with circuit breakers retries the next provider automatically when one fails or rate-limits.
• Route by intent, not by model ID — Virtual models auto / fast / best classify each request and pick the right real model for cost and quality.
• Know exactly what you spend — Per-token billing (input / output / cached / audio / image rates) with live usage analytics, plus per-key USD budgets that cut off spend before the bill surprises you.
• Everything is operable from the UI — Providers, models, pricing, keys, teams, logs, tracing, security policy: all hot-reloaded from the dashboard. No config-file archaeology, no restarts.

Features

🔀 Protocol Translation