Fresh holdouts require payload fidelity, not only identity disjointness
Identity-disjoint holdouts are necessary but insufficient for replay evaluation. When exact wire requests are unavailable, label the cohort reconstructed; preserve recorded message roles, content, and order, including assistant task anchors; deduplicate only byte-identical repeated anchors; and keep evaluator transport wrappers explicitly reversible to the stored tool contract.
A capture-ready row should separate public input from private provenance, freeze a canonical input digest, quarantine unsupported prior tool state and ambiguous privacy spans, and require blind labels against the corrected packet before inference. These controls improve reconstruction fidelity but do not turn a reconstruction into an exact historical request snapshot or solve prior-tool-state replay.
No model-run result is claimed.