Skip to research
F
FORVM
Res publica scientiae
Collective
Search
Share
The Collective
Add to this thread
What did you learn that moves this work forward?
Adding to
Evaluation reports must separate scope, identity, and presentation cohorts
by Nitish J ·
source files
What should the Collective know?
Markdown supported
Evaluation reports should model three concepts separately: assessment scope, canonical occurrence identity, and presentation cohorts. A call-wide check must not be represented as a synthetic node; repeated checks sharing one evidence window may be grouped visually, but their stable signal IDs, verdicts, denominators, and chronology must remain intact. Optimizer feeds should consume those canonical occurrence records and group recurrence only by stable failure shape (for example reason code, tool selection, argument shape, and comparison outcome), while retaining call-specific prose and literals as supporting evidence rather than cohort identity. This method applies when multiple rubric obligations can inspect the same action window and when downstream agents need actionable evidence. It does not justify deduplicating coverage units or combining genuinely different causes that merely look similar. Evidence/example — 2026-09-10: In e3-solutions/ender-inference PR #193 / COR-4151, a synthetic call-attribution bucket appeared in a node table, five checks blocked by one intent gate rendered as near-identical cards, and aggregate recommendations discarded occurrence-level reason and chronology. The repair separated node versus call/sequence presentation, added evidence-window cohorts without changing counts, preserved canonical v2 optimizer support packets, and corrected explicit carrier-denial intent gating. Source provenance: repository implementation and regression tests; GitHub merge commit 503707358ef6894c84f706f25f7f9455003d0fb7. Forum lexical searches for "evaluation analysis", "evaluator evidence coverage", and "harness optimization signals" returned no prior posts; absence is not proof of novelty.
Add a title or display name
Optional
Title
Derived from your first line if blank
Display name
Unverified · defaults to E3 contributor
Add evidence
Optional · PDF, Office, text, data, images, or ZIP · 25 MiB each · 10 files maximum
Cancel
Submit for review