Rerun the preserved baseline contemporaneously when testing a LoRA target-module change, and…

Rerun the preserved baseline contemporaneously when testing a LoRA target-module change, and audit baseline reproducibility separately from the candidate-versus-baseline score. Identical model hashes, ordered requests, greedy settings, and prompt-token counts are necessary checks but do not by themselves establish identical outputs across serving configurations. A candidate's apparent gain can coincide with a previously correct baseline answer becoming incorrect in the new run. Preserve raw old/new outputs, identify the exact discrepancy, and avoid crediting a stable correction until it is supported.

Evidence/example — September 10, 2026: a bounded offline adapter comparison used unchanged training data and recipe except target coverage. All baseline predictions reproduced on one diagnostic cohort; one prediction did not reproduce on another, despite matching request bytes except alias and matching prompt-token counts. That discrepancy was also the candidate's only gain on that cohort. Adapter-bank size and loaded target coverage changed together, so the mechanism remains unknown; the evidence does not isolate a specific serving flag or numerical cause. Independent raw-response reparse and case-level comparison, not aggregate scores alone, revealed this qualification. Private transcripts and weights are not attached.

Limit: this is a reproducibility-check method and a qualified observation, not a claim that greedy decoding always drifts, that a particular runtime setting caused drift, or that one target-module strategy is generally better.