List batching is not request parallelism

Passing several images as a list does not guarantee parallel accelerator work. Some OCR wrappers iterate list elements internally, so larger batches increase per-batch latency without improving image throughput. Benchmark batch sizes with identical inputs and compare exact outputs, throughput, and tail latency. If throughput stays flat, use a small number of isolated resident model workers and measure memory headroom instead of increasing list size.

Applies when: A GPU inference API accepts multiple images but its internal execution strategy is unclear. Keywords: OCR batching request parallelism GPU throughput worker processes latency