Why automation projects fail, and the exception nobody scoped

There is a particular kind of meeting we have been in more than once. A business built an automation, it worked in the demo, it worked for two weeks, and then it quietly stopped being trusted. Nobody can quite say when. Somebody started checking its output manually "just to be safe", and now there are two processes instead of one.

The automation was not badly built. It was built for the good day.

Almost everything that goes wrong in these projects falls into six categories, and every one of them is decidable in advance, in a conversation, before a line of code exists. This article is those six categories and the questions that settle them.

The demo problem

When you demonstrate an automation, you use a clean input. Of course you do. You are showing the mechanism, not the mess.

The trouble is that the clean input is also what gets scoped, estimated and signed off. The business sees an invoice go in and a record come out, and reasonably concludes that the problem is solved. Then production arrives with an invoice that has two pages, a handwritten correction, and a supplier name that does not match anything in the system.

At that moment the automation does one of three things:

1. Crashes. Annoying, but honest. Somebody notices.
2. Skips it silently. Dangerous. The record just never appears, and nobody finds out until a reconciliation months later.
3. Guesses. Worst. A plausible wrong value enters the system and is trusted because everything else the automation produced was correct.

The third one is what destroys confidence, and it is specifically the risk that AI-assisted steps introduce, because they will almost always return something. A rules-based parser fails loudly on an unfamiliar format. A model returns a confident answer. That difference has to be designed for.

Failure 1: Missing or malformed data

The most common exception, and the easiest to plan for.

For every field your automation depends on, answer three questions:

• Is it required? If the process genuinely cannot continue without it, that is a hold, not a skip.
• What does invalid look like? A date in the future, a negative quantity, a total that is not the sum of the lines. Write these rules down; they are the cheapest quality control you will ever build.
• What happens when it is wrong? Skip and log, hold for review, or stop the run.

That last choice is a business decision and it differs by process. Moving marketing data between systems can usually skip a bad row and report it. Posting a financial transaction usually cannot.

A validation rule you write down before the build costs minutes. The same rule discovered in production costs a reconciliation.

One practical tip: have the automation record why it rejected something, in plain language, next to the record itself. "Total does not match line items" is actionable. A stack trace in a log file is not, at least not to the person who has to fix it.

Failure 2: The other system is unavailable

Every integration depends on something you do not control. APIs go down, credentials expire, rate limits arrive without warning, and maintenance windows happen at the worst time.

The design questions:

Retry, how many times, and how far apart? Immediate retries are usually useless, because whatever caused the failure is still happening. Back off — wait, then wait longer. And always cap it. An unbounded retry loop against a rate-limited API is a way to get your access suspended.

What happens after the retries run out? Something must hold the work and tell a person. Work that vanishes into a failed run is the most expensive failure mode there is, because you cannot even tell what you lost.

Is the operation safe to repeat? This is the one that gets missed. If your automation creates a record and the connection drops after the record is created but before the confirmation comes back, a retry creates a second one. The fix is to make operations identifiable — check whether this item already exists before creating it, or use the source system's reference as a key. Duplicate invoices are a genuinely serious problem, and this is exactly how they happen.

Have credentials got an expiry? Put a reminder in a calendar. A surprising number of "the automation broke" incidents are an expired token that nobody owned.

Failure 3: The AI result is wrong

If any step involves a model interpreting something, you need an answer to "how do we know it was right?" before you go live, not after.

Three things to build:

A confidence threshold you actually act on. Not just a number in a log. If the extraction is uncertain, the item should route somewhere different. Uncertainty that changes nothing is decoration.

A cross-check that does not use the model. If you are extracting invoice line items, check that they sum to the stated total. If you are classifying requests, check the resulting category against something structural, like the sending domain. These arithmetic and structural checks catch a large share of errors for almost no effort, and they do not depend on the model being honest about its own uncertainty.

A review queue someone owns. With a name against it, and enough context in it that reviewing an item takes seconds rather than requiring the reviewer to open the original document and start over.

There is also a decision to make explicitly: is it worse to miss something or to get something wrong? Tuned one way, more items go to review and people spend time. Tuned the other, more passes through unchecked and errors reach your records. Different processes want different answers, and it should be a conscious choice rather than whatever the default was.

Failure 4: It ran twice

Someone clicks the button twice. A scheduled job overlaps with the previous run that has not finished. A webhook is delivered more than once — most webhook providers guarantee at least once, not exactly once, and people read that too quickly.

Two defences:

• A lock, so a second run cannot start while the first is in progress.
• Idempotency, so that if it does run twice, the second run recognises the work as already done and changes nothing.

Idempotency is the stronger of the two and usually not difficult: give each unit of work a stable identifier derived from the source, and check for it before acting. The awkward part is that it has to be designed in. Retrofitting it after the first duplicate incident is considerably more work than including it.

Failure 5: Nobody is watching

This is the quiet one, and it is the reason automations get abandoned rather than the reason they break.

An automation with no visible signal will eventually stop and nobody will know. Weeks later somebody says "wasn't that meant to be automatic?"

The minimum viable monitoring is genuinely small:

• A run record: it ran, at this time, and processed this many items.
• An alert on failure, sent somewhere a person reads, which usually means email or the chat tool they already have open.
• An alert on silence, which is the one people forget. If the job has not run in twenty-five hours and it should run daily, that needs to be noticed. Failure alerts do not fire when nothing runs at all.
• A weekly summary, so someone sees the trend rather than only the incidents.

Build this at the start. It is a small amount of work up front and it is what makes the difference between an automation that lasts years and one that is quietly replaced by a person again.

Failure 6: The process changed and nobody told the automation

Automations encode assumptions: this field means that, this form has these options, this supplier always sends PDFs. Businesses change those things without connecting the change to the automation, because the automation is invisible.

Two habits help:

Validate assumptions rather than trusting them. If your automation expects one of five categories, have it flag a sixth instead of silently ignoring it. Loud surprises are better than quiet ones.

Write down what the automation depends on, in plain language, and keep it with the process documentation rather than only in the code. When someone proposes changing a form, the dependency should be discoverable by the person making the change.

What good scoping sounds like

The difference between a project that lasts and one that does not is visible in the first conversation. Good scoping sounds like:

• "What happens if the supplier name does not match anything we have?"
• "Who checks the ones the system is unsure about, and how quickly?"
• "If this runs twice, what breaks?"
• "How will you know on Monday that it ran over the weekend?"
• "What in the process might change in the next six months?"

None of those are technical questions. They are business questions with technical consequences, and the answers determine both the cost and whether anyone still trusts the thing in a year.

This is why an honest quote takes a conversation rather than an email. The successful path is often the cheap part. Everything above is the rest of it, and leaving it out does not make the project cheaper — it moves the cost to the people who have to work around the automation later.

If you have an automation that people have quietly started double-checking, that is worth a look. Bring it to a free consultation and we will work out which of these six it is.