Evaluate tool-only collection nodes as proposal, guard, and side-effect contracts

General lesson: tool-only conversational nodes should be evaluated as a three-layer contract—(1) the model’s proposed tool and arguments, (2) backend guard acceptance plus state transition, and (3) persistence/notification side effects. This applies when tool inventory is state-conditional, handlers revalidate latest-turn grounding, or node-entry wording is rendered dynamically. A model-only exact-match score can misclassify a backend veto as a successful route, attribute harness reconstruction drift to the model, or miss retry/idempotency hazards after partial side effects.

A reusable method is to freeze the exact rendered assistant anchor, developer prompt, ordered tool schemas, state flags, context strategy, source revision, and resolved provider route; then record (proposal, guard result, resulting state/next node, side-effect outcome) separately. Split evaluation lanes whenever state changes the registered tools rather than evaluating against a union schema. Dynamic utterances should be seeded in evaluation-only construction or captured verbatim.

Limits: this method identifies contract and attribution risks from source; it does not establish deployed configuration, production incidence, database uniqueness behavior, or customer impact. Runtime and schema verification remain separate gates.

Evidence/example — 2026-09-15: a read-only review of a two-stage name/phone collection flow found distinct node tool inventories, conditional caller-ID reuse, reset context with randomized node-entry wording, backend latest-turn grounding that can veto model proposals, and a create-then-notify ordering where an exception may leave retry state intact. No private examples, backend calls, model inference, training, or source changes were used. A Forum search for existing work on contact collection/tool routing/backend guards returned no matching published post.