Providers

Cordy Gateway does not host models. It routes your clients' requests to upstream providers you configure. This page covers the two integration paths, the supported upstreams, and how to wire a provider, channel, and model in the Admin.

Two integration paths

Every channel reaches its upstream through one of two paths:

1. LiteLLM — the default path, giving access to 100+ providers and models through one integration. It normalizes each provider's API to the OpenAI shape.
2. Direct OpenAI-compatible passthrough — for upstreams that already speak the OpenAI Chat Completions API, the gateway can forward the request over a pooled httpx client with no translation. The data plane implements chat (unary and streaming), embeddings, and text completions.

Current-source limitation: the data plane expects a custom endpoint in Channel.config.base_url, but the current Channel form only saves endpoint_url and does not write that runtime setting. The form also has no raw channel-config editor, and the supported channel-import command does not accept an endpoint. Therefore this release has no supported Admin or import workflow for enabling a custom/self-hosted endpoint. Do not rely on the direct path until the product connects those fields. The steps below cover built-in provider endpoints.

Supported upstreams

The runtime provider registry recognizes the following common types. The Admin accepts other uppercase provider identifiers, but accepting an identifier does not prove that its endpoint and authentication fields form a usable integration. Use the built-in endpoint workflows below; custom endpoints remain subject to the limitation above.

• OpenAI
• Anthropic
• Azure OpenAI
• AWS Bedrock
• Google Vertex / Gemini
• Cohere
• Groq
• Together AI
• DeepInfra
• OpenRouter
• Mistral
• DeepSeek
• Hugging Face
• Replicate

The runtime also contains adapters for Ollama and direct OpenAI-compatible servers such as vLLM and Volcengine Ark. They require a custom endpoint, so the current Admin wiring limitation prevents a complete supported setup through this release's public operator surfaces.

The configuration chain

Serving a model requires a small set of records, created once in the Admin under Gateway Configuration:

Provider → Public model → Provider model → Channel → Channel-provider-model

See Concepts for how these resolve at request time. The steps below build them in order.

1. Add a provider

Create a Provider for the upstream vendor:

• name — a unique internal name, e.g. openai-main.
• provider_type — an uppercase token, e.g. OPENAI, ANTHROPIC, BEDROCK, or CUSTOM.
• base_url — control-plane metadata in this source snapshot. Leave it blank for a built-in provider endpoint; it does not supply the channel endpoint read by the data plane.
• status — ACTIVE.

A provider is a global system resource. Credentials do not live on the provider — they live on its channels.

2. Define a public model

Create a Public model — the slug your clients will request in the model field:

• slug — the public identifier, e.g. gpt-4o-mini. Unique.
• display_name, description — for the catalog.
• status — ACTIVE, and is_public if it should appear in listings.

This is the only model name your clients ever see. You control it; it need not match any vendor's naming.

3. Map a provider model

Create a Provider model linking the Provider and the Public model to the concrete upstream model:

• provider_model_name — the real vendor model string, e.g. gpt-4o-mini.
• supports_chat, supports_embeddings, supports_streaming, context_window — capability flags.
• status — ACTIVE.

Add a Price history row under that provider model for metering: input/output price per 1,000 tokens, currency, USD conversion rate, source, and an effective start/end window. Price history is the request-cost authority; overlapping windows for the same provider model are rejected.

4. Add a channel

Create a Channel on the provider — a concrete routing target with its own credentials:

• Upstream credential — the provider API key. It is stored Fernet-encrypted at rest; the plaintext is never returned by the Admin after saving. For Bedrock, the AWS secret is stored the same way.
• endpoint_url — shown on the Channel form, but not connected to the data plane in this source snapshot. Leave it blank for built-in endpoints; see the limitation above before planning a custom endpoint.
• weight — relative share for weighted routing (default 100).
• priority — higher-priority channels are preferred; the weighted pick happens within the top priority band (default 0).
• region — e.g. us-east, for region-aware routing.
• cost_weight — a multiplier on the router's cost score (>1 biases away, <1 biases toward).
• rate_limit, concurrent_limit, timeout_seconds — per-channel guards.
• Cooldown settings — optionally take a flapping channel out of rotation after repeated errors.
• status — ACTIVE.

5. Enable the model on the channel

Create a Channel-provider-model binding the Channel to the Provider model. This is what actually enables the model on that channel. The provider model must belong to the channel's provider — you cannot bind provider B's model to provider A's channel.

Once at least one active channel is bound to the model, the public model becomes routable and appears in GET /v1/models for keys allowed to use it.

Routing across channels

Because a public model can be served by several channels, you can:

• Load-balance across multiple keys or accounts for the same provider using weight.
• Prefer a primary channel with priority, falling back to others automatically.
• Fail over to a different provider entirely when one is down — bind the same public model to channels on different providers.
• Bias by cost using cost_weight with the cheapest strategy.
• Route by region using region with the region_aware strategy.

The routing strategy is set globally by GATEWAY_DEFAULT_ROUTING_STRATEGY (default weighted) and can be overridden per request with the X-Routing-Strategy header:

Strategy	Picks
weighted	Priority band, then weighted-random.
cheapest	Lowest router cost score × cost_weight. Current source still reads hidden legacy ProviderModel price columns instead of PriceHistory, so validate this strategy before use.
fastest	Lowest observed P95 latency; falls back to weighted with no data.
region_aware	Region match first, then weighted.

Failover, cooldown, and circuit breakers are described in Concepts.

Worked example

Serve one public model, gpt-4o-mini, across two OpenAI accounts:

1. Provider: openai-main (provider_type: OPENAI, status: ACTIVE).
2. Public model: slug gpt-4o-mini.
3. Provider model: one under openai-main with provider_model_name: gpt-4o-mini, plus a non-overlapping effective Price history row.
4. Channels: openai-account-a and openai-account-b, each with its own encrypted OpenAI key, priority: 0, and weight: 100.
5. Channel-provider-models: bind both channels to the provider model and leave both bindings active.

Clients now call model: "gpt-4o-mini"; the default weighted strategy distributes requests between the two channels and can fail over when one is unavailable. Change their weights to alter the relative share.

Credential security

• Clients authenticate with wnx_ gateway keys only. They never see, send, or receive upstream provider keys.
• Upstream credentials are stored Fernet-encrypted in the channel record and decrypted only in memory when a request is dispatched. See Security.
• Rotating an upstream key is a channel edit in the Admin; no client change is needed.

Next steps

• Concepts — how routing and failover work.
• Admin console — users, keys, quotas, pricing, and monitoring.
• Security — credential encryption and the trust model.
• Deployment — running at scale.