Concept notes · Networking

Proxies and redirects

What sits between a client and a service changes the request: by the configuration it actually runs, and by the answers it gives.

A proxy runs the configuration it accepted

A proxy never reads your intent. It runs one configuration in memory and routes by that. New configuration arrives in one of two ways. A reload (nginx, HAProxy) checks the file, starts new workers with it and lets the old ones finish their connections. An API update (Envoy’s xDS, from a control plane or from watched files) delivers listeners, routes, clusters and endpoints as separate resources.

Every update is validated before it takes effect. An update that fails validation is rejected, and the rejected resources keep their last accepted version; other resources in the same update may still have been accepted. Over xDS, Envoy tells the control plane by answering with a NACK that carries the error. So the file or the control plane can show one configuration while the proxy runs another, and nothing in the request path reports the difference.

How it fails. A change is “deployed” but behaviour does not change, or changes only partly, because some resources were accepted and others were not. The reason is logged at the moment of the update, not when requests later fail. A proxy started from scratch with an invalid file has no previous configuration to fall back on and may not start at all.

Observe it.

nginx -t
haproxy -c -f /etc/haproxy/haproxy.cfg

Validate a file before reloading. nginx -T also prints the complete configuration the check read, with every included file.

curl -s 'http://127.0.0.1:9901/stats?filter=update_'

Envoy’s admin interface counts each subscription’s updates: cluster_manager.cds.update_attempt, update_success and update_rejected, and the same for listeners and routes. Use the admin address your deployment configures.

curl -s 'http://127.0.0.1:9901/config_dump?resource=dynamic_active_clusters'

The clusters Envoy is running now. Compare them with what the source says they should be.

What good looks like. Every change increments the success count and leaves the rejected count flat, the running configuration matches its source, and reload and update logs are clean.

Redirects and the method that follows

A redirect is a 3xx response with a Location header; the client decides whether to make a second request there, and with which method. HTTP semantics (RFC 9110) define five common codes. 301 Moved Permanently and 302 Found allow a client to change POST to GET for the next request, which browsers and curl do. 303 See Other means “fetch the result with GET”. 307 Temporary Redirect and 308 Permanent Redirect forbid changing the method: the request is repeated with the same method and body. Clients may cache 301 and 308.

A redirect can come from the application or from anything in front of it that answers on its behalf.

How it fails. A POST that meets a 301, 302 or 303 arrives at the new location as a GET with no body. The server there sees a different request: a read instead of a write, an empty form, or a 404 or 405. The client may still finish with a success code from the last hop, and dashboards that count 5xx responses stay clean, because a 3xx is not an error.

Observe it.

curl -sS -i --data 'item=42' http://shop.example/orders

--data sends a POST; without -L curl does not follow, so you see the status line and Location of the first answer.

curl -sS -L -v --data 'item=42' http://shop.example/orders

Follow the redirects. Lines starting > show the method of each request, and curl notes Switch from POST to GET when it changes it. --post301, --post302 and --post303 keep the method.

curl -sS -o /dev/null -w '%{http_code} %{redirect_url}\n' --data 'item=42' http://shop.example/orders

One line per check: the status and where it points. Then read each hop’s access log for the method and status it saw.

What good looks like. Write endpoints are reached without a redirect, or redirect with 307 or 308, and every hop’s log shows the same method and a body of the expected size.

References