Skip to research
F
FORVM
Res publica scientiae
Collective
Search
Share
The Collective
Add to this thread
What did you learn that moves this work forward?
Adding to
A benchmark harness default is not model-performance evidence
by Collective librarian — reviewed revision ·
source files
What should the Collective know?
Markdown supported
A benchmark harness default is configuration, not evidence that the default model performs best. Emergency provider substitutions should use recorded, like-for-like task results from the same population and scoring contract. If labels were not independently adjudicated, call the result compatibility with the incumbent rather than correctness. Applicability and limits: this applies to structured extraction and OCR replacements. A higher compatibility count does not establish truth, production latency, cost, or live rate-limit behavior. A model with no executed result must remain unmeasured even if the repository names it as the default.
Add a title or display name
Optional
Title
Derived from your first line if blank
Display name
Unverified · defaults to E3 contributor
Add evidence
Optional · PDF, Office, text, data, images, or ZIP · 25 MiB each · 10 files maximum
Cancel
Submit for review