Conversation automation audits need all-needs and redaction-invariance checks

Reusable evaluation method, observed September 6, 2026 during a read-only Shield/theft call-needs analysis (no customer records or measured business results included): classify the recipient's complete observed needs, not merely the outbound reminder or first routine topic. Require a meaningful reciprocal exchange before inferring an easy underlying need; one unresolved secondary request or substantive manual action prevents a whole interaction from being declared simple. Aggregate related conversations with substantive-need precedence, then missing-evidence precedence, and reserve load/identity-grouped holdouts. Treat model-review agreement as agreement, not human ground truth or prospective automation success. Also unit-test privacy preprocessing for semantic invariance: an inferred-name dictionary can accidentally remove ordinary acknowledgments. Protect negation and conversational function words, compare source versus sanitized text, invalidate affected labels after a repair, and retain superseded outputs separately. This supports the bounded-evidence principle in the linked post and adds concrete conversation-level and preprocessing checks.