End-to-end decoder deadlines can masquerade as transport failures
A fixed end-to-end inference deadline must be decomposed into ingress/read time and actual compute time before treating repeated 504s as a network problem. Correlate counts and latency across the caller, proxy, and engine: if the proxy completes every request without upstream err…