Gateway Risks

Every external request passes through the gateway, so if it is down or misconfigured, every API behind it becomes unreachable at once, even though the services themselves are healthy. Its failure modes are also global: a bad route change, an expired TLS certificate, a shared Redis rate limiter failing closed or a connection leak can each take down all traffic in seconds. A compromised gateway exposes everything too. That is why it is tier-0 infrastructure. Redundant instances across availability zones remove the single-machine failure, but not the single-component failure: one bad configuration rolled out to every instance still breaks everything, so change safety and blast-radius limits matter equally.

A gateway becomes a bottleneck when work per request grows or capacity does not keep up with traffic. Common causes are CPU-heavy filters that parse or rewrite JSON bodies, synchronous remote calls per request such as token introspection or a remote rate-limiter check, too-small connection pools towards backends, blocking I/O on a thread-per-request model, and large uploads streamed through it. Each adds latency to every API at once. The fixes are local JWT validation with cached JWKS, streaming instead of buffering bodies, tuned upstream pools with HTTP/2, a reactive or virtual-thread runtime, presigned URLs for uploads, and autoscaling on CPU and connections, verified by load tests.

Organisational coupling means service teams cannot change their own APIs without going through the gateway and the team that owns it. It grows when route configuration is centrally owned, when the gateway transforms payloads so a field change needs a gateway change, when business rules like pricing or validation creep into filters, and when custom plugins need specialist knowledge. The result is a release queue at the platform team, coordinated deployments and shared risk, the same failure pattern as the old ESB era. Remedies are a thin gateway, standard reusable policies, and federated configuration where each team owns its route file in Git, reviewed and deployed through GitOps.

Run several stateless gateway instances spread across at least two, preferably three, availability zones using topology spread constraints, behind a multi-zone load balancer. Size for peak plus the loss of a zone and autoscale with an HPA on CPU and connections, pre-scaling before known events. A PodDisruptionBudget keeps enough instances during node maintenance, and graceful shutdown drains connections. Health checks must reflect real readiness. Availability also depends on change safety and dependencies: validated, tested and canaried configuration with instant rollback, automated certificate renewal with expiry alerts, explicit failure policies for Redis and the identity provider, and early load shedding. Measure the gateway with its own SLOs.

That must be an explicit, documented decision, because both naive options are bad. Failing closed rejects every request whenever Redis fails over, turning a cache problem into a full outage. Failing open removes all abuse protection exactly when an attacker might be causing the trouble. The usual recommendation is a local fallback: each gateway instance switches to an in-memory limiter with its share of the global limit, giving approximate protection while Redis is unavailable. Put a short timeout and a circuit breaker on the Redis call so slow responses do not add latency, alert on degraded mode, and test the behaviour with chaos experiments such as blocking Redis traffic.

Treat gateway configuration as production code, because a single bad route can break every API. Store it in Git and deploy it through GitOps with Argo CD or Flux. In the pull request, validate it against a schema, run route unit tests with WebTestClient that assert which service each path reaches, and lint for risks such as catch-all predicates or public admin routes. Replay synthetic traffic in staging. In production, roll it out as a canary to one instance first, compare 5xx and 404 rates for a few minutes, then progress gradually with automatic rollback on anomalies. Keep rollback under two minutes, and audit every change.

A thin gateway handles only generic, cross-cutting edge concerns: routing, TLS termination, authentication and coarse authorisation, rate limiting, request size limits, CORS, header normalisation and telemetry. It does not read or change business payload fields, call databases, hold business rules like pricing or validation, or shape responses for particular clients; that belongs in the services themselves or in backend-for-frontend services owned by the client teams. The benefit is that the gateway stays fast, stateless apart from rate-limit counters, and rarely needs to change, so it is not a bottleneck or a release queue. A useful review question is whether any filter would change if a business rule changed.

Stop running one global gateway in front of everything. Split gateways by domain, such as orders, catalogue and payments, or by audience, such as partners versus consumer apps, so a failure affects one slice. For the strongest isolation, use cell-based architecture: each cell holds its own gateway, services and databases for a subset of customers, and a very thin global router maps customers to cells, so one bad change hits perhaps five percent of users. Add per-region gateways with DNS failover, roll configuration out cell by cell, and keep emergency paths ready: restricted internal ingress for critical operations, static maintenance pages at the CDN and documented failover runbooks.