Extraction Quality Audit

Most teams cannot tell you how good their extraction is. They can tell you it is better than what they had, which is not the same thing.

The Extraction Quality Audit measures your pipeline against the methodology behind our published benchmarks: where it fails, what kind of failure it is, what it costs you per document, and how close it already sits to the noise floor of your own task.

You get a sealed measurement report and a readout. Fixed price, two to three weeks, starting when your documents and current outputs are in hand.

What the audit measures

Noise floor. The irreducible disagreement in your task. This sets the ceiling, and it stops you chasing accuracy that does not exist.

Cross-family consensus panel. Your documents run through a multi-vendor panel. Where the vendors diverge is where a single-vendor pipeline is silently wrong.

Error taxonomy. Not one number, but what kind of wrong: negation, assertion status, variable cardinality, boundary errors. This is the part you can act on.

Cost ladder. What your extraction costs per document across model families, and what quality each rung actually buys.

The deliverable is a measurement report and a readout call, written so it can be handed to an auditor or a board.

When people come to us

Nobody buys this out of curiosity. They call when something has already happened.

• An audit wall. EU AI Act high-risk obligations become enforceable in August 2026. Or 21 CFR 11, or GxP, and somebody has asked how the output is validated.
• A pilot nobody trusts. The extraction shipped, and the people downstream quietly stopped relying on it.
• Diligence. You are buying, or being bought, and the product depends on extraction nobody has measured.
• An inherited pipeline. New CDO, new VP of Data, and a system you did not build is now yours to defend.

What it does not include

Rebuilding or improving the pipeline, which is the follow-on engagement. Labeling a full gold corpus. Infrastructure or security review. Vendor selection, though the cost ladder informs it.

What we need from you

A sample of real documents, your current extraction outputs on those documents, the target schema, and about two hours of a domain expert. The clock starts when those are in hand, not when the contract is signed.