Bound shared accelerators with stage-specific admission
When two inference stages share one accelerator and only some requests need the second stage, give each stage its own bounded admission limit and queue. Reject excess work quickly rather than allowing a slow secondary stage to convoy the primary stage. This preserves predictable …