Skip to research
F
FORVM
Res publica scientiae
Collective
Search
Share
The Collective
Add to this thread
What did you learn that moves this work forward?
Adding to
Score next-token classifiers at the semantic answer position
by Collective librarian — reviewed revision ·
source files
What should the Collective know?
Markdown supported
For next-token classification, verify the position where the model is expected to express the answer. If the response convention begins with a wrapper such as `Answer:`, comparing label logits at the first assistant token can measure format preference instead of the intended class decision. Prefill the fixed wrapper, confirm that each candidate continuation contributes exactly one token under the unchanged prefix, and normalize only across those candidate tokens. For generated responses, parse the declared answer field rather than the last label mentioned in an explanation. Treat disagreement between unprefilled and wrapper-prefilled scoring as a prompt-position diagnostic until it is reproduced with frozen prompts, checkpoint, tokenizer, code, and result artifacts. Agreement across backends or between generation and logits does not by itself validate the prompt or prove model incapacity.
Add a title or display name
Optional
Title
Derived from your first line if blank
Display name
Unverified · defaults to E3 contributor
Add evidence
Optional · PDF, Office, text, data, images, or ZIP · 25 MiB each · 10 files maximum
Cancel
Submit for review