← Back to Articles
The short answer

Pilots stall most often because they were designed to prove the technology works rather than to answer the questions that gate a production decision: who owns it, what it costs at scale, how it degrades, and whose workflow absorbs the change. A pilot that only tests feasibility will succeed and still leave the deployment decision unmade, because none of the blocking questions were in scope.

The pattern

A pilot is scoped. A motivated team runs it in one department with vendor support close at hand. It works. The report is positive. Then the conversation about scaling begins, and it surfaces questions nobody was asked to answer: who funds it beyond the pilot, who maintains it, what happens when the champion moves roles, whether the workflow holds in a department that did not volunteer.

The pilot did not fail. It answered a question that was never the blocker.

Four questions pilots usually skip

  • Who owns this in production? Not who runs the pilot. Who holds the budget line, the incident process, and the vendor relationship in year two.
  • What does it cost at full scale? Pilot pricing is frequently discounted or free. Integration effort is usually the larger number and is rarely in the pilot budget.
  • How would we know it is degrading? Over a twelve-week pilot, drift is invisible. Over three years it is the main risk.
  • Does this work for the unenthusiastic? Pilots run on volunteers. Production runs on everyone.

What the ones that scale did

In our experience the distinguishing feature is unglamorous: the successful ones established a baseline before they started, and picked a workflow where someone already owned the outcome. When the pilot ended they could say what changed against a number that existed beforehand, and there was a named person whose job got easier or harder.

The stalled ones usually cannot answer what the process cost before the tool arrived, which means the benefit case is an estimate arguing with a budget.

Our view

We would go further than most: a large share of healthcare AI pilots should not be run at all. If the organisation cannot articulate what decision the pilot will settle, and what result would cause it to walk away, the pilot is a procurement ritual with a technical costume.

The most useful discipline we give clients is to write the stopping condition first. Before the pilot starts, state the result that would make you decline to proceed. Teams find this uncomfortable, which is the point. A pilot with no failing grade is not an experiment.

The second discipline: pick something boring. The pilots that scale tend to address a workflow nobody enjoys, with a measurable baseline and a clear owner. Ambitious clinical pilots generate more excitement and far fewer production deployments.

Sources

  1. Ministry of Health Singapore, AIHGle 2.0 deployers' toolkit, on internal governance and selecting AI solutions. go.gov.sg/aihgle

Designing a Pilot That Decides Something

We work with teams on what to test and what to measure before the pilot starts.

Start a Conversation →