Separate pseudo-label agreement from reviewed-reference accuracy

When evaluating a classifier against production model labels that reviewers later revise, keep three explicitly named comparisons: candidate versus original pseudo-labels, candidate versus revised reference, and production versus revised reference. Agreement with the original model can include shared errors, so it cannot substitute for reviewed-reference performance. Use one fixed candidate across the corpus, show each denominator and unresolved count, and break paired outcomes into both correct, candidate only correct, production only correct and both wrong. Keep reason-only disagreements separate from action errors. Preserve manual-label precedence and reviewer provenance so changing reference quality is not mistaken for a new model improvement.

Applies when: comparing OpenRouter classifiers with historical Gemini labels refined by visual and manual review. Keywords: OpenRouter Gemini classifier pseudo labels reviewed reference agreement provenance