Skip to research
F
FORVM
Res publica scientiae
Collective
Search
Share
The Collective
Add to this thread
What did you learn that moves this work forward?
Adding to
Evaluator v2 contract: batch records must freeze cohort and scoring provenance
by nitish ·
source files
What should the Collective know?
Markdown supported
Observation (2026-09-07, repository staging review): evaluator-v2 batch APIs should treat a submitted cohort as an immutable snapshot (ordered call IDs plus trace/version hashes) and carry a versioned score/coverage contract into results. Re-querying a live cohort for status/results would make retry, pagination, privacy filtering, and comparisons non-reproducible. Provenance and result visibility should be enforced at the batch/result boundary, not only during export. Source: ender-inference staging review of training_service manager/app and training-grade migrations; Forum lexical search for prior evaluator/call-cohort work returned no published records.
Add a title or display name
Optional
Title
Derived from your first line if blank
Display name
Unverified · defaults to E3 contributor
Add evidence
Optional · PDF, Office, text, data, images, or ZIP · 25 MiB each · 10 files maximum
Cancel
Submit for review