Invoice anomaly detection and processing¶

Fast facts¶

What it is¶

Manual invoice processing is costly, time-consuming, and prone to error. While predictive AI models can identify patterns and incongruencies in an organization’s invoicing data, generative AI can augment the validation process to generate concise summaries of all detected anomalies and improve invoice approvals.

By crafting clear narratives around these invoice anomalies, generative AI summarization can help better communicate underlying causes and improve certainly around approval or rejection of any given invoice.

This is another example of generative and predictive AI working together to deliver new efficiencies for an organization.

How it works¶

The data from the internal invoicing system, like SAP Concur, gets into the organization’s database (information filled out by the employee submitting the invoice). This information is then fed into a predictive model that utilizes unsupervised learning for anomaly detection by comparing every invoice against historical data from previously labeled invoices (training data, where invoices are categorized as “anomalous” and “non-anomalous”).

Using Prediction Explanations, a generative AI model is instructed through a system prompt to summarize the predictions for any given transaction in a concise and human-readable format, which are then fed back into the original invoicing system where the analyst or the financial manager is able to review the findings and make the decision (reject or approve). The process augments invoicing by improving anomaly detection rates for invoices and explaining the anomalies to the people making the decisions. It eliminates a lot of the manual steps required to review each and every invoice, by automating most of the reasoning processes involved in the review process.

Two simple Python files can easily orchestrate this integration through simple functions and hooks that will be executed each time an invoice requires a prediction and its consecutive analysis. The first file has the credentials to connect with the generative AI model and contains the prompt to summarize the explanations and insights derived from the predictive model. The second file easily orchestrates the whole predictive and generative pipeline through a few simple hooks.

User experience¶

The end user can interact with the invoice anomaly detection solution via the front end user interface. They consume the generated insights within their invoicing system to make the final decision on each individual invoice. Everything in the backend is handled by the predictive model and the generative AI solution.

Why it might benefit your business¶

By automating anomaly detection organizations can accelerate invoice processing workflows, reduce the human capital required to handle manual invoice reviews and minimize disruptions created by invoicing errors. The additional benefit of this process is improved communication with external parties, like employees submitting invoices. Fewer legitimate invoices are being flagged due to the predictive AI pipeline, while more illegitimate ones get through the review process. Those that do get flagged are accompanied by an appropriate narrative explaining the organization’s decision to reject the invoice.

Depending on the size of the organization and its invoicing backlog, the solution can save dozens, hundreds, and even thousands of hours on invoice processing, while also saving the organization money by detecting more anomalous invoices.

Potential risks¶

Risks associated with this use case span both generative and predictive AI components of the solution.

• An inaccurately flagged invoice may lead to an incorrect decision by the user if the system prompt for the generative AI is not fine-tuned to appropriately explain the prediction. In this case, a bad prediction may be masked by a bad summarization with generative AI. The output may look convincing and the users may choose to just trust it to make the decision.

• A system prompt that’s not fine-tuned may lead to unconventionally worded and structured summaries for invoices, which complicates the review and may also impact the user experience. For example, the system outputs an explanation for the prediction that’s too long for the invoicing system to display.

Baseline mitigation tactics¶

• Custom metrics monitoring for the generative AI model that tracks text quality parameters, like readability and complexity.

• Extensive pre-production testing of the solution, such as feature selection, prompt engineering, and various LLMs. This also requires a human in the loop, i.e. the end user should be involved in evaluating the quality of various pipelines to identify the optimal solution. Since such solutions are integrated with existing invoicing tools, it’s important to make sure that the output of the model fits the UI of the system.

• A retraining regiment that uses grounding data to improve the model’s outputs. However, this requires a new process, where the analyst can amend the automed report, which is then fed into a vector database (new infrastructure).