03Research artifact · arXiv:2608.14639

verifydoc — per-field selective risk control

Reference implementation for “Valid Per-Field Selective Risk Control for Document Extraction: Three Failure Modes, a Validity Ladder, and When Conditioning Pays.” It shows that the natural per-field accept/review procedure silently violates its own selective-risk guarantee on real documents.

Sole author · Submitted 28 July 2026 · announced August 2026 · arXiv preprint · 14 pages

Paper ↗Source ↗
13,859
fields evaluated
800
CORD receipts
3
failure modes diagnosed
49.0%
base field accuracy

In short

  • Diagnoses three failure modes — including document clustering — proposes a validity ladder, and characterizes when conditioning pays.
  • Evaluated on 13,859 claude-sonnet-5 field extractions from 800 CORD receipts, of which 49.0% were correct.
  • The harness is seed-pinned and regression-gated and released under Apache-2.0, so the numbers in the paper are reproducible rather than reported.
  • A companion benchmark paper — VerifyDocBench, measuring per-field calibration, selective risk, and grounding at scale across models and languages — is under review.
01

The problem

Document extraction pipelines routinely route low-confidence fields to human review and accept the rest, on the assumption that this delivers a bounded error rate on the accepted set. On real documents that guarantee does not hold — it fails silently, which is the worst way for a risk control to fail.

02

Contribution

  • Shows the natural per-field accept/review procedure violates its selective-risk guarantee on real documents.
  • Diagnoses three distinct failure modes, including document clustering — fields are not independent when they come from the same page.
  • Proposes a validity ladder, and characterizes the conditions under which conditioning actually pays.
03

Evidence

13,859 field extractions produced by claude-sonnet-5 over 800 CORD receipts, with a 49.0% correct rate — a regime where selective risk control matters rather than one where the model is already right.

04

Reproducibility

The verifydoc repository is a seed-pinned, regression-gated harness under Apache-2.0: reruns reproduce the published figures, and regressions fail the build rather than drifting quietly.

05

Companion work

VerifyDocBench — measuring per-field calibration, selective risk, and grounding for document extraction at scale, across models and languages — is under review as the benchmark companion to this paper.