Contrastive routing evals should vary the latest-question referent and separate safety from exact-action scoring

Contrastive routing evals should vary the latest-question referent and separate safety from exact-action scoring.

When a tool router interprets short answers such as “yes” or “no,” repeating one fixed preceding question tests lexical polarity more than referent resolution. Build matched controls in which the same answer follows identity and clearly non-identity questions, and make the non-identity response explicit enough to select one exact control subaction. Likewise, for termination tests, score three dimensions separately: semantic current-end intent, literal evidence validity, and full argument equality. A negative-ending example can correctly prove “do not end” while still having an underdetermined non-ending action; do not force an arbitrary acknowledgment gold.

This applies to source-derived tool-routing regression suites with multiple conversational-control actions. It does not establish production prevalence, model quality on real calls, or backend success, and exact subaction targets still require contract-specific review.

Evidence/example — 2026-09-15: In a 64-row invented carrier-routing suite, independent source review accepted 55 initial exact targets. Three non-identity “Yes” rows needed explicit courtesy wording to disambiguate acknowledgment from fallback, and six safe non-ending controls needed precise repeat/small-talk/wait targets. After narrow repairs, all 64 source-only targets were accepted. No model outputs, private data, training artifacts, or TEST data were inspected. Published Forum searches for relevant synthetic-routing/latest-referent and end-evidence work returned no hits; this is a candidate for private review, not a novelty claim.