Decision Engines for Accounting Automation

Generative models are useful when accounting work requires explanation, reconstruction, or multi-step reasoning. But many accounting workflow decisions are more specific: identify what's holding up a customer's payment, assign an invoice line to an account, or check whether a charge meets a stated policy. We tested whether TypeSafe's Jev decision model could handle those bounded judgments without sending every case through a generative LLM.

The result depends less on the accounting function than on the shape of the decision. A classification with clearly defined choices and complete evidence can work well as a direct decision-engine call. A classification that hides missing company policy inside hundreds of plausible choices still benefits from a reasoning model. “Just pick an account” can be asking quite a lot. In that second case, the decision engine proved more useful as a router.

Two accounting decisions with very different shapes

We chose tasks that already appear in accounting workflows, using structured inputs and historical decisions as reference labels. We share summary measurements here.

AR collections classification. Read a customer's collection history, balance and aging information, then select the category that best describes the current issue or status. Categories include payment commitments, disputes, document requests, difficulty reaching the customer, and payments awaiting reconciliation. The history matters: a customer requesting an invoice copy is in a different situation from one who has received it and promised to pay. This multi-class classification helps the collections team decide what needs attention next.

AP account coding. Assign every line of a complete invoice to an account in the company's chart of accounts. This is a multi-class classification task at the line-item level, with all lines classified together in a single invoice-level request. The input includes the full account master, invoice descriptions and amounts, available department and channel context, and historical account assignments from earlier postings. The output is an account selection for each line. Several accounts may look equally reasonable; choosing between them can depend on company-specific accounting policy that the invoice itself does not explain.

Where direct classification was competitive

On the AR cases requiring model interpretation, Jev came close to Luna and Gemini. Here are the results for the 97-case semantic subset, excluding cases handled by deterministic rules. Agreement measures matches with historical decisions.

The speed and cost difference was larger. Jev's median semantic-decision latency was 0.45 seconds and its estimated total cost was $0.00812, compared with 2.54 seconds and $0.05439 for Luna. That makes direct classification worth considering here. The model has defined categories, the notes it needs to read, and code taking care of the exact numeric rules.

There is still real interpretation involved. Collection notes accumulate over time: an earlier promise to pay, a later document request, another follow-up. The model has to work out what is relevant now. But it can usually look for the answer in the supplied history. Our reading of this result is that Jev is a useful candidate for this kind of bounded decision, with a small observed agreement gap worth testing on a larger sample.

Where direct classification fell short

AP coding was where the simple replacement idea ran into trouble. The challenge set contained 120 complete invoices, 417 lines and a 554-account master. We score agreement per line; latency measures the complete invoice request.

That is too large a gap for us to swap Jev in as the classifier. The benchmark already supplied structured invoice fields, so reading the document was not the problem being tested. The difficult part was choosing among related accounts when the history was missing or only partly comparable. Reimbursements, duties and ancillary charges also need to be understood in the context of the other invoice lines. A familiar-looking description is not always enough.

In some cases, the accounting policy is only hinted at in the context: a department, a cost center, or how a similar purchase was coded previously. Selecting the account requires connecting those clues and working out which policy distinction applies. Our results suggest that the Jev model we tested struggled with this kind of contextual reasoning, leaving a substantial gap to the larger LLMs.

Where routing worked better

The AP result suggested another experiment. Could Jev recognise which invoices needed the more capable model, even when it could not reliably code them itself? We asked it to route routine invoices with clear supporting context to Luna, and ambiguous or policy-dependent invoices to Gemini. The selected classifier still received the complete invoice and full account master.

We froze the routing prompt and threshold on one calibration set, then tested on a different set. The fresh holdout contained 120 invoices and 227 lines; Jev routed 34 invoices to Gemini and the rest to Luna.

Routing costs include Jev and the selected classifier. Routing latency is composed by adding their separately recorded call times, representing a sequential request path. These results use a different sample from the direct AP comparison above.

For this sample, that was a more useful job for Jev. We gave up some agreement compared with always calling Gemini, but reduced the estimated cost substantially. It is a tradeoff we can measure and tune: the application chooses the routing threshold, the models and the fallback policy.

There is a small catch with routing benchmarks: a convincing route is not the final result. We have to score the account assignments made by the selected model and include both calls in the cost and latency. Otherwise we are measuring how quickly we decided to do the work, which is the easy part.

Where we would use this

Direct classification where the evidence is fully available. When the categories are clearly defined and the input contains enough information to choose between them, a decision engine can handle the classification directly. Our AR collections results make this a promising use case.

Simple validation checks on documents or extracted fields. A decision engine could check whether an extracted description is supported by the document text or whether a note satisfies a stated requirement. These are small judgments with a clear pass, fail or review outcome. We have not benchmarked this use case here.

Routing to different models based on task complexity. A decision engine can assess the supplied context and send routine cases to a smaller model, reserving a more capable model for cases that need additional reasoning. Our AP routing experiment shows how this can reduce cost while retaining much of the stronger model's accuracy.

This fits how we think about the engine beneath Cadel workflows. Code handles known rules, models handle interpretation, and the workflow has a way to escalate. Decision engines add another option to that toolkit. We can choose where to spend the reasoning effort for each case.

Choosing the right decision system

These results suggest that Jev can approach LLM accuracy on well-defined classification tasks where the necessary context is available, with substantial reductions in latency and cost. More complex tasks that require reasoning through subtle accounting distinctions still benefit from larger reasoning models. Decision engines give us another useful option in an accounting workflow, with the choice depending on the task.

Making that choice requires an evaluation framework built around the accounting domain and the specific task. By testing on representative data and comparing accuracy, latency and cost, we can determine where a decision engine is sufficient, where a reasoning model is needed, and where routing between models offers the best tradeoff.

See how Cadel applies this architecture across AP automation and AR intelligence, or get in touch to benchmark it on your workflow.