Deployment

The supported way to run Cordy Gateway in production is Docker Compose with docker-compose.prod.yml. This page covers the production topology, how to scale it, the optional observability overlays, and day-two operations. For first bring-up, start with the Quickstart.

Production topology

docker-compose.prod.yml runs the full stack on one Docker host:

   Clients ──► nginx ──► gateway (1..N replicas) ──► pgbouncer ──► postgres
                 (LB / TLS)         │                                 ▲
                                    └──────────► redis                │
                                                                 worker ┘
   Admins  ──► admin (Django) ─────────────────► postgres, redis

Service	Role	Exposure
nginx	Load balancer and TLS termination point; round-robins across gateway replicas	Publishes GATEWAY_HTTP_PORT (default 8002)
gateway	Data plane; scalable to N replicas	Internal 8002, behind nginx
admin	Control plane (Django)	Publishes ADMIN_HTTP_PORT (default 8001)
pgbouncer	Transaction pooler for the gateway hot path	Internal only
postgres	PostgreSQL 18 + pg_cron	Internal only
redis	Rate limits, quotas, cache, cooldown, idempotency, buffers	Internal only
worker	Background usage settlement and audit upload	Internal only

Postgres, PgBouncer, and Redis are never published to the host. nginx terminates plain HTTP by default — put a 443 server block and certificates in docker/nginx/, or front the stack with your own reverse proxy or load balancer.

Bring it up:

docker compose -f docker-compose.prod.yml --env-file .env up -d --build

Horizontal scaling

The gateway service has no fixed container name and sits behind nginx, so you can run multiple replicas. Scale up or down with Compose:

# Run 4 gateway replicas
docker compose -f docker-compose.prod.yml --env-file .env up -d --scale gateway=4

Because all shared state lives in Redis and Postgres, replicas are safe to add: rate limits, quotas, cooldown, and idempotency are cluster-global. Each replica runs GRANIAN_WORKERS worker processes (default 8); tune workers per replica to match host CPU.

Capacity planning

Throughput depends heavily on your upstream provider latency, hardware, model, and payload sizes — always measure your own workload. As rough guidance from internal benchmarking on a single test host:

• A useful planning figure is on the order of ~500 requests/second per gateway replica for a lightweight workload.
• In an internal benchmark the architecture sustained ~2,151 RPS with zero server-side errors at scale=4, and routing latency stayed bounded as replicas were added.

These are benchmark observations on a constrained test rig, not guarantees. Add replicas and re-measure against your real traffic and providers, and watch the metrics below.

Observability

The gateway exposes metrics for scraping and can export traces. Both are opt-in overlays.

Metrics (VictoriaMetrics + Grafana)

The gateway serves Prometheus-format metrics at /metrics/prometheus. In production this endpoint is denied unless the caller presents GATEWAY_METRICS_TOKEN or comes from a CIDR in GATEWAY_METRICS_ALLOWED_CIDRS — it exposes cost, provider, and breaker internals, so keep it gated.

The bundled monitoring overlay runs VictoriaMetrics (a Prometheus-compatible time-series database), vmalert (alerting rules), and Grafana (dashboards):

docker compose \
  -f docker-compose.prod.yml \
  -f docker-compose.monitoring.yml \
  --env-file .env up -d

Useful metric families include gateway_requests, gateway_requests_error, gateway_tokens, gateway_cost_usd, gateway_failover_success_rate, gateway_routing_overhead_ms, and gateway_provider_circuit_breaker_state.

Tracing (OpenTelemetry)

Set OTEL_TRACING_ENABLED=true and point OTEL_EXPORTER_OTLP_ENDPOINT at an OTLP collector to emit distributed traces. Two backends are provided as overlays:

• Grafana Tempo via docker-compose.tracing.yml (OTLP → Tempo, viewed in Grafana).
• Self-hosted Langfuse via docker-compose.observability.yml for LLM-focused tracing.

Both are optional. See Configuration for the OTEL_* variables.

Backups and upgrades

Day-two operations are provided as make targets that wrap the compose stack:

Task	Command	What it does
Backup	make backup	Logical pg_dump into BACKUP_DIR (default /var/backups/cordy, retention BACKUP_RETENTION).
DR drill	make dr-drill	Backup, restore into a throwaway database, and verify — an RTO/RPO proxy.
Upgrade pre-flight	make preflight	Read-only checks before an upgrade.
Upgrade	make upgrade	Pre-flight → backup → migrate → verify. Use DRY_RUN=1 to preview.
Upgrade rollback	make upgrade-rollback BACKUP=…	Restore the database from an upgrade backup.
Health self-check	make health-check	Customer-side check of Postgres, Redis, gateway, and backups.
Acceptance check	make acceptance-check	Runs the deployment acceptance checklist into a report.

Database migrations are owned by the Admin service and run automatically on Admin startup; upgrades apply new migrations as part of make upgrade.

Kubernetes: reference-only

The Kubernetes manifests under k8s/ are reference-only and unvalidated. They have not been run end-to-end and are not a supported deployment path. Use docker-compose.prod.yml for production.

If you adapt the manifests for a cluster, treat them as a starting point and validate the full path yourself — database wiring (DB_*, not just DATABASE_URL), the Ingress body-size limit, secrets, and migrations. kubectl --dry-run=client only checks YAML syntax; it does not prove the services start or serve traffic.

Operational checklist

• Terminate TLS at nginx (or an external proxy) and set DJANGO_SECURE_SSL_REDIRECT=True.
• Set CHANNEL_ENCRYPTION_KEYS identically on the Admin and Gateway, and store it in a secrets manager.
• Gate /metrics/prometheus with a token or CIDR allow-list.
• Set CLIENT_CORS_ORIGINS explicitly if browsers call the gateway; never use * in production.
• Configure and test backups before going live; enable WAL archiving/PITR and set REQUIRE_PITR=1 when ready.
• Scale gateway replicas to your measured throughput and watch the failover and error metrics.

Next steps

• Configuration — every variable referenced here.
• Security — the trust model and hardening.