Separate decision labels from business outcomes and quarantine future evidence in operational SFT

When constructing SFT data from operational tool traces, label the model decision and the downstream result on separate axes, and keep future evidence out of the model input. This applies when later turns or handler outcomes help humans decide whether an earlier action was premature or misgrounded. Its limits are that stored traces may not contain the exact generation-start request or audio, so causal attribution can remain unresolved.

A robust pipeline builds one caller-turn decision episode, links calls/results conservatively, and assigns (1) an expected tool/arguments label from input-visible evidence, (2) a decision-quality label, and (3) a result-family label such as found, genuine not-found, unavailable, validation rejection, or infrastructure failure. Later success and prefix relations may nominate cases and support blinded adjudication, but cannot enter the training/evaluation prompt. Group splits must close over calls, repeated entities, corrected variants, near-duplicates, and prior evaluation exposure before model execution.

Evidence/example — 2026-09-10: A voice-routing corpus had a current deterministic rule treating bare three-digit references as lookup obligations, while operational evidence highlighted one-to-four-digit lookups as possible premature fragments. Reusing that rule would encode the disputed behavior as ground truth. The remedy is human-adjudicated contextual completeness, an ambiguous/withhold class, clean controls, and a frozen group-disjoint lockbox. This is a dataset-design correction, not proof of a universal reference-length rule or production causality.