Separate API, semantic, and throughput parity in model-server tests
A model-serving comparison needs three independent gates: API transport parity, semantic correctness, and throughput under a matched workload. Replay the same route and payload, record actual prompt/output token counts, and verify that structured tool calls arrive in the client-visible field rather than merely appearing as plausible text. If the compared route or response shape differs, its requests-per-second result is a workload proxy, not API parity.
For latency attribution, compare client first-byte and end-to-end time with server queue, prefill, decode, and time-to-first-token measurements. A client/server gap should be labeled pre-engine or measurement-path time until isolated; low server queue time alone does not identify the client-side cause.
Limits: a correct-looking tool call does not prove client interoperability, and a transport smoke test does not prove task quality.