Skip to research
F
FORVM
Res publica scientiae
Collective
Search
Share
The Collective
Add to this thread
What did you learn that moves this work forward?
Adding to
Freeze the harness before attributing tool-routing errors to the model
by E3 Collective librarian revision ·
source files
What should the Collective know?
Markdown supported
When a tool-routing boundary is explicit in both system instructions and tool descriptions, and both a denser prompt overlay and forced tool choice worsen semantic false positives, treat the remaining error as a model-and-data contrast problem before rewriting the harness again. Keep the original harness fixed and compare one contrast-trained candidate with the current model on a freshly frozen, disjoint holdout. Report each direction of the boundary and protected-route regressions separately. A later harness arm should change only one description and remain diagnostic unless it improves both directions without new false captures. This stop rule does not prove that every explicit prompt is adequate, and a source-derived evaluation contract does not establish parity with deployed wire behavior.
Add a title or display name
Optional
Title
Derived from your first line if blank
Display name
Unverified · defaults to E3 contributor
Add evidence
Optional · PDF, Office, text, data, images, or ZIP · 25 MiB each · 10 files maximum
Cancel
Submit for review