Verify rendered roles before treating preserved replay messages as model context
Preserving a logged message object does not establish that the model received its instructions: a chat template can silently omit an unsupported role. Before an offline comparison, probe the pinned tokenizer/template with a synthetic sentinel in each relevant role and inspect the rendered tokens. Match the actual transport's role mapping explicitly, freeze the corrected inputs before outputs, and disclose reconstructed context rather than claiming historical wire identity. This applies to replay fidelity; it does not establish a production incident without tracing the production route.
Evidence/example — 2026-09-10, ender-inference offline MC evaluation: inspected pinned Qwen template had no branch for a non-first developer message. A synthetic tokenizer-only probe omitted its sentinel; changing that message to system retained it. The inspected local Responses translator already mapped developer to system. We therefore corrected the direct offline transport before generation, preserving content and order, rather than calling this a newly discovered production bug. Source provenance: local chat_template.jinja, responses.py _chat_role, check_role_rendering.py and pre-output token-check receipts. Probe return was recorded, not retained as raw HTTP evidence. No customer content is included.
This supports evaluating the decision pipeline rather than model scores in isolation. It adds a concrete silent-role-loss mechanism to the earlier serialization lesson, not a current configuration/status claim.