Providers

Cordy Gateway does not host models. It routes your clients' requests to upstream providers you configure. This page covers the two integration paths, the supported upstreams, and how to wire a provider, channel, and model in the Admin.

Two integration paths

Every channel reaches its upstream through one of two paths:

1. LiteLLM — the default path, giving access to 100+ providers and models through one integration. It normalizes each provider's API to the OpenAI shape.
2. Direct OpenAI-compatible passthrough — for upstreams that already speak the OpenAI Chat Completions API, the gateway forwards the request over a pooled httpx client with no translation. It is enabled per channel by setting openai_passthrough = true in the channel's config, and covers chat (unary and streaming), embeddings, and text completions.

Use the direct path for OpenAI-compatible endpoints such as vLLM, Volcengine Ark, Vercel AI Gateway, or any self-hosted OpenAI-compatible server. Use LiteLLM for everything else.

Supported upstreams

The following provider types are recognized out of the box. Because the provider type is a free-form uppercase token, operators can add new upstreams without a code change as long as the integration is supported by LiteLLM or the direct path.

• OpenAI
• Anthropic
• Azure OpenAI
• AWS Bedrock
• Google Vertex / Gemini
• Cohere
• Groq
• Together AI
• DeepInfra
• OpenRouter
• Mistral
• DeepSeek
• Hugging Face
• Replicate
• Ollama (self-hosted)
• Any OpenAI-compatible server via the direct path (vLLM, Volcengine Ark, and similar)

The configuration chain

Serving a model requires a small set of records, created once in the Admin under Gateway Configuration:

Provider → Public model → Provider model → Channel → Channel-provider-model

See Concepts for how these resolve at request time. The steps below build them in order.

1. Add a provider

Create a Provider for the upstream vendor:

• name — a unique internal name, e.g. openai-main.
• provider_type — an uppercase token, e.g. OPENAI, ANTHROPIC, BEDROCK, or CUSTOM.
• base_url — optional; leave blank to use the provider's default endpoint.
• status — ACTIVE.

A provider is a global system resource. Credentials do not live on the provider — they live on its channels.

2. Define a public model

Create a Public model — the slug your clients will request in the model field:

• slug — the public identifier, e.g. gpt-4o-mini. Unique.
• display_name, description — for the catalog.
• status — ACTIVE, and is_public if it should appear in listings.

This is the only model name your clients ever see. You control it; it need not match any vendor's naming.

3. Map a provider model

Create a Provider model linking the Provider and the Public model to the concrete upstream model:

• provider_model_name — the real vendor model string, e.g. gpt-4o-mini.
• input_price, output_price — used for cost metering and cost-aware routing.
• supports_chat, supports_embeddings, supports_streaming, context_window — capability flags.
• status — ACTIVE.

4. Add a channel

Create a Channel on the provider — a concrete routing target with its own credentials:

• Upstream credential — the provider API key. It is stored Fernet-encrypted at rest; the plaintext is never returned by the Admin after saving. For Bedrock, the AWS secret is stored the same way.
• base_url / endpoint_url — override the upstream endpoint (required for the direct path to a self-hosted server such as vLLM).
• weight — relative share for weighted routing (default 100).
• priority — higher-priority channels are preferred; the weighted pick happens within the top priority band (default 0).
• region — e.g. us-east, for region-aware routing.
• cost_weight — a multiplier on price for cost-aware routing (>1 biases away, <1 biases toward).
• rate_limit, concurrent_limit, timeout_seconds — per-channel guards.
• Cooldown settings — optionally take a flapping channel out of rotation after repeated errors.
• status — ACTIVE.

For a direct OpenAI-compatible upstream, also set openai_passthrough = true in the channel's config.

5. Enable the model on the channel

Create a Channel-provider-model binding the Channel to the Provider model. This is what actually enables the model on that channel. The provider model must belong to the channel's provider — you cannot bind provider B's model to provider A's channel.

Once at least one active channel is bound to the model, the public model becomes routable and appears in GET /v1/models for keys allowed to use it.

Routing across channels

Because a public model can be served by several channels, you can:

• Load-balance across multiple keys or accounts for the same provider using weight.
• Prefer a primary channel with priority, falling back to others automatically.
• Fail over to a different provider entirely when one is down — bind the same public model to channels on different providers.
• Bias by cost using cost_weight with the cheapest strategy.
• Route by region using region with the region_aware strategy.

The routing strategy is set globally by GATEWAY_DEFAULT_ROUTING_STRATEGY (default weighted) and can be overridden per request with the X-Routing-Strategy header:

Strategy	Picks
weighted	Priority band, then weighted-random.
cheapest	Lowest effective price (cost_weight × provider-model price).
fastest	Lowest observed P95 latency; falls back to weighted with no data.
region_aware	Region match first, then weighted.

Failover, cooldown, and circuit breakers are described in Concepts.

Worked example

Serve one public model, gpt-4o-mini, load-balanced across an OpenAI account and a self-hosted vLLM server:

1. Providers: openai-main (provider_type: OPENAI) and vllm-local (provider_type: CUSTOM).
2. Public model: slug gpt-4o-mini.
3. Provider models: one under openai-main (provider_model_name: gpt-4o-mini), one under vllm-local (provider_model_name: your served model name).
4. Channels: openai-main channel with the OpenAI key; vllm-local channel with endpoint_url pointing at the vLLM server and openai_passthrough = true in config.
5. Channel-provider-models: bind each channel to its provider model.

Clients now call model: "gpt-4o-mini" and the gateway balances between OpenAI and vLLM, failing over if one is unavailable.

Credential security

• Clients authenticate with wnx_ gateway keys only. They never see, send, or receive upstream provider keys.
• Upstream credentials are stored Fernet-encrypted in the channel record and decrypted only in memory when a request is dispatched. See Security.
• Rotating an upstream key is a channel edit in the Admin; no client change is needed.

Next steps

• Concepts — how routing and failover work.
• Security — credential encryption and the trust model.
• Deployment — running at scale.