Deployment

The supported way to run Cordy Gateway in production is Docker Compose with docker-compose.prod.yml plus the documented docker-compose.production-overrides.yml. This page covers the production topology, how to scale it, connect operator-managed observability, and handle day-two operations. For first bring-up, start with the Quickstart.

Production topology

docker-compose.prod.yml runs the full stack on one Docker host:

   Clients ──► nginx ──► gateway (1..N replicas) ──► pgbouncer ──► postgres
                 (LB / TLS)         │                                 ▲
                                    └──────────► redis                │
                                                                 worker ┘
   Admins  ──► admin (Django) ─────────────────► postgres, redis

Service	Role	Exposure
nginx	Load balancer and TLS termination point; round-robins across gateway replicas	Publishes GATEWAY_HTTP_PORT (default 8002)
gateway	Data plane; scalable to N replicas	Internal 8002, behind nginx
admin	Control plane (Django)	Publishes ADMIN_HTTP_PORT (default 8001)
pgbouncer	Transaction pooler for the gateway hot path	Internal only
postgres	PostgreSQL 18 + pg_cron	Internal only
redis	Rate limits, quotas, cache, cooldown, idempotency, buffers	Internal only
worker	Background usage settlement and audit upload	Internal only

Postgres, PgBouncer, and Redis are never published to the host. nginx terminates plain HTTP by default — put a 443 server block and certificates in docker/nginx/, or front the stack with your own reverse proxy or load balancer.

The upstream production file does not map ENVIRONMENT, CORS/proxy controls, metrics controls, several runtime limits, the Admin portal URLs, or the credit-consume WAL. An .env value has no container effect unless a Compose file references it. Create the required production override before using any command below; the bare production file otherwise starts the Gateway with its development default.

Bring it up:

docker compose \
  -f docker-compose.prod.yml \
  -f docker-compose.production-overrides.yml \
  --env-file .env up -d --build

Horizontal scaling

The gateway service has no fixed container name and sits behind nginx, so you can run multiple replicas. Scale up or down with Compose:

# Run 4 gateway replicas
docker compose \
  -f docker-compose.prod.yml \
  -f docker-compose.production-overrides.yml \
  --env-file .env up -d --scale gateway=4

Because all shared state lives in Redis and Postgres, replicas are safe to add: rate limits, quotas, cooldown, and idempotency are cluster-global. Each replica runs GRANIAN_WORKERS worker processes (default 8); tune workers per replica to match host CPU.

Capacity planning

Throughput depends heavily on your upstream provider latency, hardware, model, and payload sizes — always measure your own workload. As rough guidance from internal benchmarking on a single test host:

• A useful planning figure is on the order of ~500 requests/second per gateway replica for a lightweight workload.
• In an internal benchmark the architecture sustained ~2,151 RPS with zero server-side errors at scale=4, and routing latency stayed bounded as replicas were added.

These are benchmark observations on a constrained test rig, not guarantees. Add replicas and re-measure against your real traffic and providers, and watch the metrics below.

Observability

Production observability uses infrastructure you operate and can reach from the production network. The Gateway exposes authenticated Prometheus-format metrics and can export OTLP/HTTP traces.

Metrics

Set a strong token in .env and leave the CIDR bypass empty for a Gateway behind public nginx:

GATEWAY_METRICS_TOKEN=<metrics-token>
GATEWAY_METRICS_ALLOWED_CIDRS=

Apply it with the production override, then point your existing Prometheus-compatible scraper at the public HTTPS endpoint:

docker compose \
  -f docker-compose.prod.yml \
  -f docker-compose.production-overrides.yml \
  --env-file .env up -d gateway nginx

scrape_configs:
  - job_name: cordy-gateway
    scheme: https
    metrics_path: /metrics/prometheus
    authorization:
      type: Bearer
      credentials: <metrics-token>
    static_configs:
      - targets: [gateway.example.com]

The CIDR check uses the immediate socket peer. If the endpoint is behind public nginx, allowing nginx or its network would permit all public requests it proxies before token validation. Use GATEWAY_METRICS_ALLOWED_CIDRS only when the scraper connects through a direct, isolated trusted path.

Useful metric families include gateway_requests, gateway_requests_error, gateway_tokens, gateway_cost_usd, gateway_failover_success_rate, gateway_routing_overhead_ms, and gateway_provider_circuit_breaker_state.

Tracing (OpenTelemetry)

Configure the full OTLP/HTTP trace URL and headers in .env, as shown in Configuration, then apply the production override. The collector hostname must resolve from the Gateway container, its HTTPS certificate must be trusted there, and network policy must allow egress to it.

Bundled profile status

docker-compose.monitoring.yml, docker-compose.tracing.yml, and docker-compose.observability.yml are development/reference profiles. The monitoring services join the external development network while production nginx and Gateway join cordy-link-prod-network; the tracing and observability collectors likewise do not share the production Gateway network. The observability profile also contains CHANGEME defaults. These files are useful source examples, but they are not production overlays and must not be combined with the production command without a separately engineered and validated network, secret, storage, and access-control deployment.

Credit-consume WAL

The production override mounts the named volume gateway_wal at /var/lib/cordy and writes the Redis-failure fallback to /var/lib/cordy/credit-consume.jsonl. This keeps the append-only evidence across container recreation. The shipped image runs as root and can write the mount; a redistributed image that runs under a custom UID must create the directory and grant that UID write access before serving traffic.

This WAL is a fail-soft billing record, not a queue with an automated consumer. If Redis rejects a consume-buffer write, the request still completes and the cost is appended. If that append also fails, the Gateway counts an under-charge. There is no automated replay tool in the repository; preserve and reconcile WAL records manually as described in Operations.

Backups and upgrades

Day-two operations are provided as make targets that wrap the compose stack:

Task	Command	What it does
Backup	make backup	Logical pg_dump into BACKUP_DIR (script default ./backups; production template /var/backups/cordy; retention BACKUP_RETENTION).
DR drill	make dr-drill	Backup, restore into a throwaway database, and verify — an RTO/RPO proxy.
Upgrade pre-flight	make preflight	Read-only checks before an upgrade.
Upgrade	make upgrade	Pre-flight → backup → migrate → verify. Use DRY_RUN=1 to preview.
Upgrade rollback	make upgrade-rollback BACKUP=…	Restore the database from an upgrade backup.
Health self-check	make health-check	Customer-side check of Postgres, Redis, gateway, and backups.
Acceptance check	make acceptance-check	Runs the deployment acceptance checklist into a report.

Database migrations are owned by the Admin service and run automatically on Admin startup; upgrades apply new migrations as part of make upgrade.

Kubernetes: reference-only

The Kubernetes manifests under k8s/ are reference-only and unvalidated. They have not been run end-to-end and are not a supported deployment path. Use docker-compose.prod.yml for production.

If you adapt the manifests for a cluster, treat them as a starting point and validate the full path yourself — database wiring (DB_*, not just DATABASE_URL), the Ingress body-size limit, secrets, and migrations. kubectl --dry-run=client only checks YAML syntax; it does not prove the services start or serve traffic.

Operational checklist

• Terminate TLS at nginx (or an external proxy) and set DJANGO_SECURE_SSL_REDIRECT=True.
• Set CHANNEL_ENCRYPTION_KEYS identically on the Admin and Gateway, and store it in a secrets manager.
• Apply docker-compose.production-overrides.yml to every create, recreate, and scale command.
• Gate /metrics/prometheus with a bearer token behind public nginx; keep the CIDR bypass empty unless the scrape path is direct and isolated.
• Set CLIENT_CORS_ORIGINS explicitly if browsers call the gateway; never use * in production.
• Configure and test backups before going live; enable WAL archiving/PITR and set REQUIRE_PITR=1 when ready.
• Verify /var/lib/cordy is writable and retain the gateway_wal volume; database WAL/PITR and the credit-consume JSONL WAL solve different recovery problems.
• Scale gateway replicas to your measured throughput and watch the failover and error metrics.

Next steps

• Configuration — every variable referenced here.
• Operations — step-by-step health, diagnosis, backup, recovery, and upgrade commands.
• Security — the trust model and hardening.