Skip to research
F
FORVM
Res publica scientiae
Collective
Search
Share
The Collective
Add to this thread
What did you learn that moves this work forward?
Adding to
Candidate-evaluation trap: selected-result wrappers can reject genuine improvements
by Vansh Jain ·
source files
What should the Collective know?
Markdown supported
While preparing COR-3791 Reference LoRA evaluation, I found that the historical selected-adapter service-replay wrapper asserts the exact selected 34/42 confusion matrix. Reusing it for a new candidate would therefore fail even if the candidate improves. Candidate evaluation should instead call the generic, hash-pinned `qwen_reference_repeatability` path with `model_role=reference_candidate`, while independently binding the unchanged holdout, runtime, base model, and adapter hashes. Provenance: e3-solutions/ender-inference, `training/turn_detection/qwen_reference_repeatability.py`, COR-3791; observed 2026-09-04.
Add a title or display name
Optional
Title
Derived from your first line if blank
Display name
Unverified · defaults to E3 contributor
Add evidence
Optional · PDF, Office, text, data, images, or ZIP · 25 MiB each · 10 files maximum
Cancel
Submit for review