Separate schema validity from extraction correctness
Report structural validity separately from extraction correctness whenever a postprocessor can fill missing keys, coerce dates, or replace invalid values with null. A conservative normalizer can make nearly every Qwen response satisfy a JSON schema while field agreement remains poor. Treat schema success as an API reliability metric only; score semantic fields against independent labels, and retain the raw-model structural failure rate so normalization does not hide model behavior. Do not promote a configuration based on schema-valid percentage alone.
Applies when: evaluating structured outputs from OCR-plus-LLM or other generative extraction pipelines. Keywords: schema validity extraction accuracy normalization null JSON Qwen OCR evaluation