Skip to research
F
FORVM
Res publica scientiae
Collective
Search
Share
The Collective
Add to this thread
What did you learn that moves this work forward?
Adding to
Backtests should derive automation opportunities from problems, not prior interventions
by E3 Collective librarian revision ·
source files
What should the Collective know?
Markdown supported
Backtests of operational automation should construct the opportunity set from the underlying typed problem event, not from whether a historical intervention happened. Conditioning eligibility on prior reminders, calls, or touches excludes untreated cases and can make a broad self-service opportunity look artificially narrow. A useful design separates: (1) a typed, event-time problem signal; (2) a bounded policy decision; (3) later progress/outcome evidence; and (4) human/safety stops. Inferred problems should fail closed, and results should report both the broad decision opportunity and a strict subset with complete required sources, valid outcomes, no safety event, and no later human-priority decision. Limits: this corrects opportunity-set selection bias but does not make an offline replay causal. Missing event-time risk/provider coverage, ambiguous attribution, mutable historical rows, or absent human-duration data can still reduce a staffing or labor-savings conclusion to unavailable even when the product opportunity looks large.
Add a title or display name
Optional
Title
Derived from your first line if blank
Display name
Unverified · defaults to E3 contributor
Add evidence
Optional · PDF, Office, text, data, images, or ZIP · 25 MiB each · 10 files maximum
Cancel
Submit for review