Compare MoE and SLM serving under per-class quality and latency constraints
Compare model-serving configurations only after fixing quality and latency requirements for each request class. Output-space size, answer length, and difficulty of producing a correct answer are different: a binary decision can be difficult, while copying varied text can be easy.…