Cordy Gateway

Cordy Gateway is a self-hosted LLM gateway. It exposes a single OpenAI-compatible endpoint and routes each request to one of 100+ upstream providers, while adding authentication, per-key quotas, cost metering, and automatic failover on top. You run it on your own infrastructure: your API keys, your Postgres, your logs. No third party sees your prompts.

Point any OpenAI SDK at the gateway by changing one line — the base_url — and keep using the chat completions, embeddings, and models APIs you already know.

What it is

• One endpoint, many providers. Clients call a single OpenAI-compatible URL. Behind it, the gateway routes to OpenAI, Anthropic, Azure OpenAI, Bedrock, Gemini, Groq, Together, OpenRouter, self-hosted vLLM, and more.
• Authenticated and metered. Every request carries a wnx_ API key. Keys have scopes, token quotas, rate limits, and concurrency caps.
• Cost-controlled. Per-model pricing, prepaid credit balances, and usage analytics let you cap and attribute spend.
• Self-hosted and single-tenant. Cordy Gateway is a privately deployed product. You operate one instance for your own organization; it is not a multi-tenant managed SaaS.

Who it's for

Cordy Gateway is for teams that want a single internal LLM endpoint they control:

• Platform and infrastructure teams consolidating many provider keys behind one auditable gateway.
• Product teams that need per-key quotas, cost attribution, and failover without building it themselves.
• Organizations with data-control requirements that need prompts and keys to stay on their own infrastructure.

Core capabilities

• OpenAI-compatible chat/completions, completions (legacy), embeddings, and models endpoints.
• 100+ providers through LiteLLM, plus a direct passthrough for any OpenAI-compatible upstream (vLLM, Volcengine Ark, and similar).
• Per-key authentication with scopes, monthly token quotas, requests-per-minute rate limits, and per-key in-flight concurrency caps.
• Routing strategies (weighted, cheapest, fastest, region-aware) with automatic same-model failover across channels.
• Cost metering per provider and model, prepaid credit balances, and a self-service usage analytics endpoint.
• Upstream credentials encrypted at rest (Fernet), append-only audit logging, and role-based access in the Admin control plane.
• Streaming (SSE), tool/function calling, and idempotent retries.

Honest limits

• Text only. Image and audio (multimodal) content is rejected today. Requests must be text.
• Not a managed SaaS. It is self-hosted and single-tenant by design.
• No compliance certifications. Cordy Gateway does not claim SOC 2, ISO 27001, HIPAA, FedRAMP, or any other certification. See Security for the honest trust model.

Architecture at a glance

Cordy Gateway runs as two cooperating services over a shared PostgreSQL database and Redis:

                       ┌─────────────────────────────┐
   Admins ───────────► │  Control plane (Admin)       │  :8001
                       │  Django, owns schema +       │
                       │  migrations, providers,      │
                       │  channels, keys, pricing     │
                       └──────────────┬──────────────┘
                                      │ shared Postgres + Redis
                       ┌──────────────┴──────────────┐
   Clients (OpenAI ──► │  Data plane (Gateway)        │  :8002
   SDK) Bearer wnx_    │  Starlette + Granian,        │ ──► 100+ providers
                       │  raw psycopg, routing,       │     (LiteLLM / direct)
                       │  auth, metering, failover    │
                       └─────────────────────────────┘

• The control plane is a Django Admin where operators register providers, channels, models, pricing, users, and API keys. It owns the database schema.
• The data plane is the high-performance request path clients call. It reads configuration from the shared database and never bootstraps Django on the hot path.

See Concepts for the full request lifecycle and model-routing model.

A note on Cordy's client apps

Cordy also ships separate end-user client apps. Those are distinct products with their own documentation and can optionally be pointed at a Cordy Gateway deployment as their backend, but Cordy Gateway itself is a standalone, headless API — this documentation set covers only the gateway.

Next steps

• Quickstart — deploy the supported stack and make your first request.
• Client integration — point your existing OpenAI SDK at the gateway.
• Concepts — how the control plane, data plane, and model routing fit together.