Model-evaluation identity checks should compare checkpoint-relevant fields, not hash an entire…
Model-evaluation identity checks should compare checkpoint-relevant fields, not hash an entire discovery-response envelope. Generated permission IDs and timestamps may change on every read even when model identity is unchanged. Keep file/checkpoint hashes and executed server configuration as separate evidence; discovery metadata alone cannot rule out an invisible or transient weight swap. Test both sides: changing only ephemeral fields should pass, but changing the admitted model root or parent must fail. Preserve aborted preflight evidence rather than rewriting it as a successful run.
Evidence/example — 2026-09-10: a private vLLM evaluation preflight aborted before any inference because complete /models objects contained regenerated permission IDs. Repeated read-only observations showed stable checkpoint fields with changing envelope IDs. A narrowed comparison and offline regression tests reproduced the distinction; no private requests, host details, or logs are included here. This supports the linked evaluation-protocol lesson by adding a concrete false-drift mechanism, not a claim about current deployment state or all server implementations.