Client integration
Point any OpenAI SDK at Cordy Gateway — authentication, streaming, tool calling, embeddings, routing headers, and the product extras.
Cordy Gateway speaks the OpenAI API. Any client or SDK that can target a custom base_url works without code changes beyond the URL and key. This page shows the common integration patterns end to end.
Connect
Set two things on your client:
- Base URL:
https://<host>:8002/v1(usehttp://only for local no-TLS trials). - API key: a
wnx_-prefixed key, sent as a bearer token.
Authentication accepts either header:
Authorization: Bearer wnx_<key>
or
X-API-Key: wnx_<key>
The standard OpenAI SDKs use the Authorization: Bearer form automatically. API keys must be sent in a header — keys in the URL query string are not accepted.
Python
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["CORDY_API_KEY"], # wnx_...
base_url=os.environ["CORDY_BASE_URL"], # https://<host>:8002/v1
)
JavaScript / TypeScript
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.CORDY_API_KEY, // wnx_...
baseURL: process.env.CORDY_BASE_URL, // https://<host>:8002/v1
});
Throughout, model is the public model slug configured in the Admin, not a vendor model name.
Chat completions
Non-streaming
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a concise assistant."},
{"role": "user", "content": "What is Cordy Gateway in one line?"},
],
temperature=0.7,
max_tokens=256,
)
print(resp.choices[0].message.content)
const resp = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [
{ role: "system", content: "You are a concise assistant." },
{ role: "user", content: "What is Cordy Gateway in one line?" },
],
temperature: 0.7,
max_tokens: 256,
});
console.log(resp.choices[0].message.content);
Every chat request must contain at least one user message; system-only or assistant-only histories are rejected with 400.
Streaming (SSE)
Set stream=True to receive server-sent events as they are produced. The stream ends with a [DONE] sentinel.
stream = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Stream a short poem about routing."}],
stream=True,
)
for chunk in stream:
delta = chunk.choices[0].delta.content
if delta:
print(delta, end="", flush=True)
print()
const stream = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Stream a short poem about routing." }],
stream: true,
});
for await (const chunk of stream) {
const delta = chunk.choices[0]?.delta?.content;
if (delta) process.stdout.write(delta);
}
If the upstream stream closes without a finish reason, the gateway emits an in-band stream_interrupted error event before [DONE], so clients can tell a clean finish from a truncation. Routing headers (X-Channel-Id, X-Provider, X-Failover-Count) are present on the streaming response as well as non-streaming.
Raw SSE with curl:
curl -N "$CORDY_BASE_URL/chat/completions" \
-H "Authorization: Bearer $CORDY_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"hi"}]}'
Tool / function calling
Tool calling uses the standard OpenAI tools and tool_choice fields. If you pass tool_choice, you must also pass a non-empty tools list.
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}
]
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "What's the weather in Paris?"}],
tools=tools,
tool_choice="auto",
)
msg = resp.choices[0].message
if msg.tool_calls:
call = msg.tool_calls[0]
print(call.function.name, call.function.arguments)
const tools = [
{
type: "function",
function: {
name: "get_weather",
description: "Get the current weather for a city.",
parameters: {
type: "object",
properties: { city: { type: "string" } },
required: ["city"],
},
},
},
];
const resp = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: "What's the weather in Paris?" }],
tools,
tool_choice: "auto",
});
const msg = resp.choices[0].message;
if (msg.tool_calls) {
const call = msg.tool_calls[0];
console.log(call.function.name, call.function.arguments);
}
You can round-trip the assistant message back into messages (the canonical multi-turn pattern), append your tool result as a role: "tool" message, and call again to let the model use the result.
Embeddings
emb = client.embeddings.create(
model="text-embedding-3-small",
input=["first document", "second document"],
)
print(len(emb.data), "vectors")
print(len(emb.data[0].embedding), "dimensions")
const emb = await client.embeddings.create({
model: "text-embedding-3-small",
input: ["first document", "second document"],
});
console.log(emb.data.length, "vectors");
input accepts a single string or an array of strings. Empty input is rejected with 400.
Text only
The gateway is text-only today. A chat message content may be a plain string, or an array of {"type": "text", "text": "..."} parts (which are concatenated). Any non-text part — image_url, audio, and similar multimodal types — is rejected with a clear 400 error. Do not send images or audio.
Product extras
Beyond the OpenAI-standard endpoints, the gateway exposes read-only product endpoints that your client can call with the same key (they require the balance scope).
Balance and quota
curl -sS "$CORDY_BASE_URL/balance" -H "Authorization: Bearer $CORDY_API_KEY"
Returns the key's token quota (quota_limit, quota_used, quota_remaining) and the owning user's prepaid credit_balance_usd.
Subscription
curl -sS "$CORDY_BASE_URL/subscriptions" -H "Authorization: Bearer $CORDY_API_KEY"
Returns the user's tier, plan, subscription status, and feature summary.
Package catalog
curl -sS "$CORDY_BASE_URL/packages" -H "Authorization: Bearer $CORDY_API_KEY"
Returns the read-only catalog of platform packages (tiers, quota limits, pricing).
Usage analytics
curl -sS "$CORDY_BASE_URL/account/analytics?period_days=30" \
-H "Authorization: Bearer $CORDY_API_KEY"
Returns the authenticated user's own usage, latency, and cost aggregated over a trailing window. period_days accepts 1–365 (default 30); out-of-range values are clamped. Note: this endpoint is available only under the /v1 prefix.
Request headers
You can influence routing and reliability per request:
| Header | Values | Effect |
|---|---|---|
X-Routing-Strategy | weighted, cheapest, fastest, region_aware | Overrides the routing strategy for this request. See the current cheapest pricing limitation in Providers. |
X-Region | a region code, e.g. us-east | Sets the caller region for region_aware routing. |
Idempotency-Key | any unique string | Replay-protects a POST: a repeat within the TTL returns the stored response instead of calling upstream again. X-Idempotency-Key is also accepted. |
Example — weighted routing with an idempotency key:
curl -sS "$CORDY_BASE_URL/chat/completions" \
-H "Authorization: Bearer $CORDY_API_KEY" \
-H "Content-Type: application/json" \
-H "X-Routing-Strategy: weighted" \
-H "Idempotency-Key: order-4711-attempt-1" \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}'
Passing custom headers with the OpenAI SDKs:
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "hi"}],
extra_headers={"X-Routing-Strategy": "fastest"},
)
const resp = await client.chat.completions.create(
{ model: "gpt-4o-mini", messages: [{ role: "user", content: "hi" }] },
{ headers: { "X-Routing-Strategy": "fastest" } },
);
Response headers
Successful responses include observability headers you can log or surface:
| Header | Meaning |
|---|---|
X-Channel-Id | The channel that served the request. |
X-Provider | The upstream provider used. |
X-Failover-Count | How many channels were tried before success. |
X-Routing-Overhead-Ms | Time spent in routing, excluding the upstream call. |
X-Idempotency-Status | hit when a stored response was replayed; skipped when idempotency did not apply. |
Handling errors
Errors use the OpenAI-style envelope: {"error": {"type", "message", "code", ...}}. The ones your client should handle explicitly:
402 insufficient_credits— the account is out of prepaid credit. Stop and top up rather than retrying; a retry will fail the same way.429withcode: quota_exceeded(monthly token quota) or a rate-limit code — back off and retry later, or raise the limit.401— the key is missing, invalid, revoked, inactive, or expired. Do not retry with the same key.403 scope_denied— the key lacks the scope for that endpoint. Issue a key with the right scope.503 auth_service_unavailable— a transient auth-infrastructure issue. HonourRetry-Afterand retry.503 gateway_overloaded— the worker that received the request is at its capacity cap, when the operator has enabled one. HonourRetry-After(1 second) and retry with backoff and jitter. The request was rejected before any upstream call.429 concurrency_limit_exceeded— this key already has its maximum number of concurrent requests in flight. Wait for some to finish, or lower your client concurrency.
See the API reference for the complete error table.
A note on Cordy's client apps
Cordy's own end-user client apps are separate products. They can optionally be pointed at a Cordy Gateway deployment as their backend, but they are not required — any OpenAI-compatible client works against the gateway directly, as shown above.
Next steps
- API reference — full endpoint, scope, and error tables.
- Providers — how models and routing are configured on the server side.