Automating revenue recognition includes two steps. First, understanding the contract, and then calculating the actual revenue schedule. The first step includes not just extracting the contract start date, end date, line items and price, but also nuances such as billing terms, cancellation rights, performance obligations and milestone definitions. The second step is about applying structured accounting logic to the extracted data to calculate the revenue schedule.
Our hypothesis was that the second step can be relatively straightforward for a coding agent harness if all the relevant context is provided. The first step is the harder part: identifying all the information that matters across contracts and related documents, and doing that accurately enough for the accounting logic to work on top of it.
We wanted to measure this in a more systematic way, so we built a benchmark around contract understanding for ASC 606. We split the task into two stages: contract understanding, where we measure how accurately a system identifies the relevant contract terms, and revenue scheduling, where we measure how accurately it applies the accounting logic to those terms and produces the final schedule.
We created a synthetic dataset by sampling real-world complex scenarios and different variations of final revenue schedule outputs. We first defined the underlying commercial terms and the expected revenue schedule for each scenario, then generated the corresponding input documents from those terms. The correct answer was known before the documents were created.
The portfolio
The dataset contains 37 order forms, 18 master service agreements and 10 amendment documents, covering 16 customers across eight European jurisdictions. It carries many real-world complex scenarios: terms starting mid-month, a single fee that has to be split across multiple performance obligations, order-level discounts, early terminations, metered usage lines, renewals that combine two or three expiring agreements, and one services line that is bundled into the subscription in one contract and severable in its twin. These scenarios test whether a system can identify not just individual fields, but the terms and relationships across documents that change the final revenue schedule.
| The portfolio | |
|---|---|
| Order forms | 37 |
| Master agreements | 18 |
| Amendments | 10 |
| Documents in total | 65 |
| Customers | 16 |
| European jurisdictions | 8 |
| Recognizable revenue | $12,658,554 |
| Monthly recognition entries | 1,107 |
The coding agents were not given this business's close policy: how it counts a term in months, when it defers a service fee, how it allocates a combined charge. Cadel's engine encodes that policy.
Results
| System | Overall | Variance | Contract extraction | Revenue scheduling | Renewal reconciliation | Cost per run |
|---|---|---|---|---|---|---|
| Cadel | 96.2% | 1.2 | 98.1% | 95.7% | 95.8% | 1.0× |
| Claude Code · Sonnet 5 | 72.7% | 5.3 | 98.6% | 62.6% | 76.9% | 2.0× |
| Claude Code · Opus 5 | 61.6% | 11.3 | 98.6% | 44.8% | 75.1% | 3.2× |
| Codex · GPT-5.6-luna | 54.6% | 3.0 | 97.0% | 27.6% | 93.3% | 0.1× |
| Codex · GPT-5.6-sol | 52.6% | 3.5 | 98.0% | 26.5% | 85.7% | 3.1× |
Every row is the mean of three runs. Variance is the gap between a system's best and worst run on identical inputs, in points. Cost is relative to Cadel.
Contract extraction is term dates, quantities, rates, billing cadence, and the termination or cancellation clauses that change when revenue stops. Revenue scheduling is whether each dollar landed in the right period against the right performance obligation. Renewal reconciliation is whether an arriving renewal was tied to the agreements it replaces and those agreements closed out.
The gap is between 24 and 44 points, almost entirely in the scheduling column: extraction is near-perfect everywhere, so the differences come from scheduling, not from reading the contracts.
One divergence is easy to see in the totals. Every agent decided that an implementation fee whose delivery was never confirmed had been earned, and recognised it. Across twelve runs and four models that is $552,700 of implementation and services fees booked on signature, on 27 of the 37 contracts, and the amount over-recognised is the whole fee: $88,000 on one, $76,000 on another, $33,000, $31,000, $27,500.
That is only 4.4% of the portfolio, so it is not what drives the scheduling column down. The score is computed per month, per obligation, and most of the damage is in where the money lands rather than how much of it there is.
Total contract value is right on the same 7 of 37 contracts in every single run, whichever model produced it. The judgments in play are the same ones every time.
They fail on the same contracts, but not in the same way. Codex sol lands within $70,000 of the correct portfolio total and still scores worst on scheduling at 26.5%, because it puts roughly the right amount of money in the wrong months against the wrong obligations. Codex luna has the strongest renewal column of any agent at 93.3%. It works out which contract supersedes which, then schedules the revenue into the wrong periods.
Why the difference
Our hypothesis was backwards: extraction turned out to be the easy half, and every system is near-perfect there. Scheduling is where they diverge. The calculation itself is simple; what it needs are inputs the contracts do not carry.
Take one contract. Its subscription lines are priced at catalogue rates, and its implementation line is sold at $20,000 against a $25,000 catalogue price. Claude Code · Opus 5 found that gap, identified it as a bundle discount, checked whether the discount could be attributed to the implementation line alone, concluded the documents did not carry the evidence that carve-out requires, and allocated the $5,000 proportionately across all three obligations. It cited the paragraph it was relying on. That is what ASC 606 tells you to do by default.
Our answer key does the other thing: the discount stays on the line it was given. Both readings are defensible. This business chose one and wrote it into the engine, so Opus was marked down for reasoning correctly to a different conclusion. Because allocation is proportional, that one decision moved every line and every month on the contract, and it recurred on 20 of the 25 multi-line contracts.
Claude Code · Sonnet 5 scores higher because it never asked. Nine turns to Opus's twenty-two, no mention of standalone selling price, no allocation considered: it read the stated totals off the order form and divided by the months. When the firm's conventions are not written in the documents, deeper reasoning tends to move a model further from them, not closer.
These models can do the accounting: Opus correctly recognized that a restated master agreement had no accounting effect, treated an amendment priced at catalogue as a separate contract rather than a modification, declined to record a renewal between two overlapping contracts, and flagged its own delivery-date assumption for a reviewer to check. What it could not know is which of several acceptable treatments this business applies; that is not in the contracts.
Cadel's engine holds those answers. The model reads the contract and reports what it says; the engine decides what to book, the same way every run, which is why our variance is 1.2 points against 3.0 to 11.3. The agents' lower scores in the tables come from one thing they were never given: the firm's close policy, which the Cadel workflow encodes.
How we measured
- Data: the portfolio in the table above. The corpus is synthetic, with a fictional vendor. A benchmark built on real customer contracts cannot be published, re-run by a sceptical reader, or shown free of answers that leaked into training data. This one regenerates from seed and its answer key is recomputed against the documents on every run. The same workflow runs in production on real customer contracts.
- What each system was given. The same PDFs and the same product catalogue, with no pre-parsing for anyone, plus the accounting date and the requirement to follow ASC 606. Nothing about how this business closes its books.
- How each was run. Cadel processed the portfolio as one job, the way it runs a close. The agents worked a master agreement at a time: the corpus splits into 17 batches along master-agreement boundaries, so no session sees more than four contracts and no renewal chain is cut in half. Each agent's outputs are merged back into one portfolio answer and scored against the whole key.
- One scorer for every system, against the answer key that produced the documents. Overall weights revenue scheduling at 0.60, contract extraction at 0.20 and renewal reconciliation at 0.20.
- Three runs each, with caching disabled and document contents varied so nothing could be served from a previous run.
