A large ecommerce marketplace has a vendor base in the hundreds of thousands. Every delivery produces a proof of delivery (POD), usually a scanned PDF. Before the vendor is paid, that POD has to be checked against the item received note (IRN) and the purchase order (PO), and turned into one invoice the ERP will accept.
That's a whole job, not one extraction. The invoice has a header, carrying vendor, invoice number, dates and totals, and a body of line items. The ERP won't take it unless both are right. A single POD lists dozens of lines, and the largest in our test set lists 559. Each one has to match the right line on the IRN and the PO. Get one wrong and the invoice doesn't reconcile.
This is a three-way match, run at scale, on scanned documents. We gave the same 60 cases to seven systems, scored every one of them with the same scorer, and measured all of it: the header fields, every line, and what the run cost.
The workflow
We built this with an ecommerce customer as a Cadel workflow. The design idea is simple: use the right tool for each step instead of one model for everything.
One workflow, the right tool at each step
The PO and the IRN are structured files, a spreadsheet and a CSV, so we parse them as data. The scanned POD needs a vision model to read its line items. Pairing those OCR'd lines to the clean catalog rows is the hard step, so it gets a focused model call with the candidate rows in front of it, not a general agent re-reading the raw files. Joining to the PO and assembling the invoice is deterministic code.
The workflow mixes models rather than betting on one. Gemini 2.5 Pro is the main model: in our default configuration it reads every scan, recovers lines the first pass missed, and runs the matching calls that drive reconciliation. The two cheaper configurations run those same model calls on smaller models, Gemini 3.5 Flash-Lite and GPT-5.6-luna. The deterministic half of the workflow, parsing the structured files, building candidate lists, merging matched lines into the final invoice, is identical in every configuration.
Seven ways to do the job
The obvious question is whether you need the workflow at all. Could you hand the raw POD, IRN and PO to a strong coding agent and let it produce the invoice? We tested that, against three configurations of our own workflow.
Model versions matter and they move fast, so to be precise: these are the versions we ran on 26 and 27 July 2026, each on the same 60 cases, scored by the same scorer.
- Our workflow on Gemini 2.5 Pro, our default configuration
- Our workflow on Gemini 3.5 Flash-Lite, and again on GPT-5.6-luna, our two cheaper configurations
- GPT-5.6-sol and GPT-5.6-luna, run through the Codex CLI
- Claude Opus 5 and Claude Sonnet 5, run through Claude Code
What the numbers say
How we measured
- Data: 60 cases from a public synthetic set, no customer data. 2,629 ground-truth line items, the largest single document carrying 559 of them. Scanned, image-only PODs. Seed 20260703. This is not a synthetic-only result: the same workflow runs in production on real customer documents. The synthetic set exists so we can publish exact numbers without exposing anyone's invoices.
- One scorer for every system. The composite weights line-item F1 at 0.70, line-field accuracy at 0.20 and header accuracy at 0.10. Identifying the right set of lines is the hard part of the task, so it carries most of the weight.
- We report the median, not the mean. The composite is computed per case, giving 60 scores per system, and the chart plots the median of those 60. The per-case distributions are skewed, and a mean would let a few bad cases swing the comparison.
- Line items are matched to the ground truth by content, and scored on F1 rather than recall, so a system is penalized for lines it invents as well as lines it misses. An errored case counts as zero.
- The coding agents ran in their real harnesses (Codex CLI, Claude Code) in a Docker sandbox. Each got the scanned POD as a PDF, the IRN as text and the PO parsed to text, and had to output the reconciled invoice. Our rows ran the actual product workflow end to end. Same scorer for all seven. Harness versions: inspect_ai 0.3.241, inspect_swe 0.2.63. The Codex rows ran the GPT-5.6 models on Codex's pinned 5.5 scaffold; its 5.6 profiles are code-mode only and our harness cannot select them.
- Cost is each system's total LLM spend for all 60 cases, reported relative to our Gemini 2.5 Pro configuration (1×) rather than in absolute dollars.
Why it costs less
Look at the two Claude Code runs. Sonnet 5 is the cheaper model per token and it produced the more expensive run, because it spent 2.8 times the output tokens doing the same job. Cost on this workload is set by how many tokens you spend, not by what a token costs.
Controlling that is what the structure is for. Structured files are parsed, not prompted, and the matching step works from a short candidate list, so the only document a model reads in full is the scan. A general agent re-reads the raw documents in its loop on every run. The gap on the cost axis is mostly that.
Only one agent undercuts our default configuration at all, Codex on GPT-5.6-luna at 0.77×, and its median sits 0.9 points lower. What the structure buys is the headline of the chart: the best median in the field at 5 to 7 times lower cost than the agents that tie it, and a configuration 0.8 median points back at 33 times lower cost than Opus 5. That's what AP automation at this scale takes.
