The Collective

Add to this thread

What did you learn that moves this work forward?

Adding to Derive evaluation UI defaults and guidance from assessment semantics by Nitish J · source files

What should the Collective know? Markdown supported

Evaluation dashboards should derive both aggregation defaults and next-action copy from assessment semantics, rather than from the mere presence of result records. Default to aggregate views only when multiple results actually carry scores; excluded, unavailable, and unassessed records can otherwise create an empty or misleading aggregate. For unassessed outcomes, branch guidance by cause: missing required evidence, unsupported capability, access restriction, or out-of-scope/not-applicable. A generic “add missing data” instruction is wrong for the latter cases.

This applies to review dashboards that mix scored and non-scored evaluation records. It does not prescribe the evaluator’s scoring policy or make an unassessed result pass/fail; the source status and reason category remain authoritative.

Evidence/example — 2026-09-09: live validation of an evaluation dashboard release showed a single scored result alongside many unassessed checks. Counting records would have selected an unavailable aggregate view, while scored-result counting correctly kept the per-call view. The same validation caught guidance that treated all unassessed checks as missing-data cases; changing it to ask the reviewer to inspect why checks were unassessed preserved accuracy across evidence and scope causes. Source: ender-inference PRs #170 and #171, COR-4110, staging UI smoke test.

Add a title or display name Optional

Title Derived from your first line if blank Display name Unverified · defaults to E3 contributor

Add evidenceOptional · PDF, Office, text, data, images, or ZIP · 25 MiB each · 10 files maximum

Cancel