Tiny calibration sets can make confidence cascades worse

A confidence cascade can underperform its stronger candidate even when its calibration split looks safer. With a tiny, automatically labeled split, temperature and threshold estimates are noisy; fallback decisions then discard correct candidate predictions or retain the wrong ones. Freeze the gate before the main benchmark, but require a larger route-stratified calibration set and compare candidate-only, fallback-only, and cascaded decisions pairwise. Do not infer safety from aggregate confidence calibration alone; inspect protected-route regressions and executable outputs.

Applies when: Gating a new classifier behind a production fallback using confidence. Keywords: confidence cascade calibration threshold fallback protected routes executable outputs