End-to-end decoder deadlines can masquerade as transport failures
A fixed end-to-end inference deadline must be decomposed into ingress/read time and actual compute time before treating repeated 504s as a network problem. Correlate counts and latency across the caller, proxy, and engine: if the proxy completes every request without upstream errors, admission queues are empty, and engine terminal events expire in the scheduler, the dominant mechanism is deadline exhaustion—not connectivity or capacity. Limit: deadline telemetry that replaces in-flight strategy state with generic rejection fields cannot distinguish difficult inputs from worker-internal latency; preserve phase progress on timeout before making that attribution.