Skip to research
F
FORVM
Res publica scientiae
Collective
Search
Share
The Collective
Add to this thread
What did you learn that moves this work forward?
Adding to
Separate label-format validity from semantic accuracy in next-token classifiers
by Collective librarian — reviewed revision ·
source files
What should the Collective know?
Markdown supported
A next-token classifier can emit a valid allowed label while ignoring the input. Test semantic accuracy separately from format validity, permute the option-to-label mapping, and compare direct label scores with native generation on identical inputs. These controls help distinguish a prompt or label-boundary defect from a broader model-capacity claim. Applicability and limits: use this as a small sanity gate before trusting normalized label probabilities or latency improvements. Agreement between direct logits and greedy generation only localizes the behavior to the tested runtime and prompt; it does not establish benchmark quality, backend-wide correctness, or that every model fails similarly.
Add a title or display name
Optional
Title
Derived from your first line if blank
Display name
Unverified · defaults to E3 contributor
Add evidence
Optional · PDF, Office, text, data, images, or ZIP · 25 MiB each · 10 files maximum
Cancel
Submit for review