AI pilot project: start from a process that can be measured

AI pilot project: two warehouse staff check an incoming pallet against the delivery note

In a group of three companies that provide ongoing maintenance for industrial plants, the first artificial intelligence project concerned management control. It stopped before reaching a result, because the internal contact person could not find time for it. When the management team reopened the subject, the question had changed: which activity to choose so that an AI pilot project would produce a measurable result, even with a contact person busy elsewhere.

Aesir proposed a check that the accounts department carries out anyway, with or without technology: verifying that every job closed with a work report has actually been invoiced. In the proposal, the pilot works alongside the person who handles that check today. Agent and person work on the same reports, and the comparison between the two lists becomes the yardstick of the project.

Which process should an AI pilot project start from?

An AI pilot project pays off best on a repetitive process with high volumes and a verifiable outcome, one that absorbs hours of qualified staff.

In first meetings with companies the criterion comes back in similar words: find the point where the business devotes many resources to low-value activities, and calculate the return from there. The best candidates have an incoming document and a precise destination. A manufacturer receives around one hundred delivery notes a day, to be entered into the ERP line by line. A plant installer with a growing after-sales team sends around four hundred invoices a month, one at a time, over WhatsApp. That adds up to two and a half days of work every month, and sometimes the number gets blocked for twenty-four hours.

On processes like these, the measure is simple: documents that go through without intervention, documents that need a manual check, and time recovered. When an error occurs, it surfaces at a known point: at the stock count for a badly entered delivery note, at the reminder stage for an invoice that never arrived.

Processes that management redesigns frequently are better stabilised first. An agent works on rules, and rules that change every month leave the pilot without a benchmark.

The pilot works next to the person, on the same documents

The comparison happens on everyday work: in an AI pilot project the agent operates in parallel with the person who already does the task. In the maintenance group described above, the person keeps checking the final accounts as before, while the agent runs the same check on the same reports. The differences between the two results show the agent’s mistakes and, often, rules that nobody had ever written down. Aesir applies the same approach in its own accounts department, with agents working alongside staff to see whether they reach the same accuracy.

Running in parallel lightens the load on the contact person, which had been the weak point of the first attempt. The person carries on with their own work, and the extra time goes into a review of the differences at fixed intervals, held together with an Aesir consultant. How long that review takes depends on the number of differences, and the parallel run itself measures it.

When delivery notes are read automatically, the system recognises the header and the lines and checks that the data are complete. Above a reliability threshold the data go straight into the ERP; below it, a task is created for a person, who confirms or corrects. Calibrating the threshold requires a sample of the company’s real documents, including handwritten ones.

A car workshop, two calendars and a quote that stays with a person

A car repair workshop with a steady rise in contacts managed appointments and quote requests on two different calendars and a shared board. The initial request was for an agent on chat, WhatsApp and phone, able to book appointments without going through the front desk.

The objection came during the demonstration. A quote depends on make, model and parts, and cannot be delegated. Around that limit the scope of the pilot was designed: the agent collects the request, reads the data from the photo of the vehicle registration document sent by the customer, asks for confirmation and routes the case. At the desk, a member of staff completes the quote. As for the workshop’s management software, it enters the project only if it exposes open interfaces; otherwise the data pass through an intermediate database.

The measure was set before the proposal, with four values: average time per request, requests per month, hourly cost of the people handling them and share of requests closed without human intervention. An initial analysis takes a day or a little more. The expected duration is two to three months. Work begins with a single channel and one type of request, and the scope widens after the first results.

Insight. The demonstration also brought out a technical limit: on the voice channel the agent copes less well with callers who interrupt. Starting from chat and WhatsApp, where the photo of the registration document arrives already attached, keeps the pilot on the ground where the measurements are cleanest.

How long does an AI pilot project take, and what does it require from the company?

On a well-defined process, an AI pilot project needs about a day of analysis and two to three months of work.

Before the start, the pilot requires a contact person and three things from the company: real documents or conversations, the annual volumes of the process and access to the ERP interfaces. The access keys are issued by the software vendor, and not always quickly. They are best requested in the week of the analysis, before the project begins.

Estimating the fee requires volumes that have been counted

For a conversational agent, the flow is designed and the number of interactions it will handle in a year is estimated. The fee depends on that volume, plus the initial configuration. For document reading, the reference is the number of documents archived each year. For automatic sending over WhatsApp, the number of messages counts.

Volumes are the figure companies know least. In a recent meeting, the estimate of invoices issued in a year ranged between ten thousand and fifty thousand. A count taken from the ERP data before the proposal makes the fee estimate verifiable, and protects the project from surprises at the first renewal.

Artificial intelligence works inside the ERP, not in its place

In a manufacturer with a few hundred employees and a long-standing, heavily customised ERP, some people internally were suggesting replacing the ERP with artificial intelligence. The IT manager stated the opposite view in one sentence: “We cannot start from AI, because we would start badly.” In his approach, artificial intelligence is a tool inside a process.

In an AI pilot project on delivery notes the principle is clear. The agent reads the document, matches the lines with the orders already in the ERP and flags the differences, while master data, stock and accounts stay where they are. Its value comes from the fact that the ERP already holds the reference against which the document is checked. A similar principle applies to access to company information, described in the article on AI on company data.

Connection goes through the ERP’s web interfaces, staging tables or shared folders. Aesir is a Gold Partner of NTS Informatica for Business Experience, and with other ERPs the available connectors are used or new ones are built. Before the pilot, it is also worth checking where the use case sits under the European regulation on artificial intelligence, which scales obligations according to the level of risk.

The phases of an AI pilot project

Phase What happens Who is involved What is measured
Analysis Choice of process, volume count, sample collection Management, process owner, Aesir Average time per case and annual volumes
Configuration Rules, reliability thresholds, ERP connection Process owner, ERP vendor, Aesir Documents correctly recognised on the sample
Parallel run Agent and person work on the same cases The person who does the task today Differences between the two results
Extension New channels or new document types Management and users Share of cases closed without intervention

Before signing up for an AI pilot project, these points should be clear:

  • the chosen process, with an owner who knows it in detail;
  • annual volumes counted from the data, not estimated from memory;
  • a sample of real documents or conversations, including the difficult cases;
  • the threshold above which data enter the ERP without review;
  • who reviews the differences during the parallel run, and how often;
  • the access keys to the ERP interfaces;
  • the result that, once reached, leads to extending the project.

Agents built around processes, data in certified infrastructure

Aesir Srl designs agents that connect to SharePoint, to the document archive and to the company’s web applications, including with non-public models when data must not leave the company perimeter. ERP, document management and infrastructure are handled by the same people, so the pilot stays connected to the systems it will have to use. We run our own Tier IV data centres in the European Union with replication, and hold ISO 9001, 27001, 27017 and 27018 certifications. We work on projects throughout Italy and, on specific request, in Switzerland, the United Kingdom and the United States.

Questions before starting the pilot

Can artificial intelligence replace the ERP?

It works inside the ERP: it reads documents and routes requests based on the data the ERP already holds. Master data, orders and accounts remain in the ERP.

Are dedicated servers or computing capacity needed?

A pilot on documents and requests uses cloud services, with no hardware purchase. Dedicated infrastructure comes into play when data cannot leave the company perimeter, with a non-public model hosted in a certified data centre.

Who updates the rules after the pilot?

An internal owner decides on changes, starting from the cases passed to a person, and the partner applies them at fixed intervals. The approach is described in the article on the virtual agent after go-live.

Does the pilot change how people work?

During the parallel run the person keeps doing the task as before, and also reviews the differences at fixed intervals. The change comes with the extension, when the agent takes over the cases it handled without errors during the parallel run.

Let’s talk

An AI pilot project reaches a measurable result when the process is stable, the volumes are counted and a person compares the agent’s work with their own. If you would like to explore the subject or assess the situation in your own company, you can fill in the form at the bottom of this page or write to support@aesir-tech.it: we will arrange a free consultation and start from your numbers.

Vuoi parlarne applicato alla tua azienda?

Ti rispondiamo con l’esperienza di progetto, non con una brochure.

Request a call