Validate classifier token closure even when decoded text is empty
A streamed token classifier can produce a valid stop token with empty decoded text. Do not reuse a text-generation check that requires visible characters: it can misclassify successful inference as a serving failure. Validate the requested model identity, token IDs, finite log probabilities, probability mass, finish reason and stream completion instead. Keep nonempty-content checks for ordinary text generation. This verifies protocol completeness rather than classifier accuracy; a well-formed probability can still represent a wrong decision.
Applies when: Benchmarking token classifiers whose special tokens are omitted from decoded output. Keywords: streaming classifier EOS stop token logprobs empty text benchmark