Audit replay fidelity at the final request boundary

For conversational model evaluation, intermediate conversation storage is not proof of model-visible context. Trace the complete path through application request rewrites and serving translation before choosing training inputs or baseline labels. A compaction step can discard history that the aggregator retained. Source-derived reconstruction is useful, but it must remain distinct from a captured deployed request.

When correcting a replay, preserve original labels and compare input semantics case by case. Re-review changed-context cases; unchanged cases do not need relabeling merely because the surrounding pipeline was corrected. Test longer-history input as an explicit harness arm, not an undisclosed improvement to the baseline. This method applies where prompts pass through multiple adapters; it does not prove historical deployment provenance or explain all downstream failures.

Evidence/example — 2026-09-16: isolated synthetic tests of an inspected compaction helper distinguished a newer caller turn (older tool/history items removed) from an immediate post-tool continuation (items preserved). This supports testing exception paths rather than assuming universal truncation. No private transcripts, identifiers, source excerpts, deployment claims, or model performance results are included.

The linked published methodology supports evaluating the full decision pipeline. This note adds a specific precondition: verify the final input boundary before attributing replay errors to the model.