AI Pilot to Production Scaling: Taking a Working Pilot Across the Business

Scaling an AI pilot to production means taking a workflow that already works for one team, site, or shift and making it the normal way that work gets done everywhere it happens, at full volume, run by people who didn't build it. The result usually shrinks on the way, because the pilot ran with the keenest team, the cleanest inputs, and its champion close at hand. The reliable route is to check that the pilot can run without that help, map how the work differs at each other site, test the second site against the same bar as the first, and only then roll out in waves.

Most of the attention on AI goes to the step before this one, which is getting a pilot into production at all. Our guide to why AI projects fail covers that broader picture, and our page on AI pilot purgatory covers the pilot that never gets a verdict. Scaling starts where those end, with a pilot that passed and an owner who now wants it everywhere.

Key Takeaways

• Scaling an AI pilot means getting the same result at every place the work happens, at full volume, with people who didn't build it. A pilot that works for one team has only proved it can work under that team's conditions.
• Economists call the drop in results at scale the "voltage effect": the tendency for a program's efficacy to change, usually to shrink, when it's scaled up (Al-Ubaydli, Lai and List, NBER, 2023).
• Sites inside one business can be run very differently. In a Census survey of 32,000 U.S. manufacturing plants, 40% of the variation in management practices was between plants of the same firm (Bloom et al., NBER, 2017).
• AI can learn the site instead of the task. A 2018 pneumonia-screening model scored 0.931 on X-rays from the hospitals it trained on and 0.815 at an outside hospital, and it could tell which hospital an image came from with over 99.9% accuracy.
• Before scaling, check six things: the result holds without the champion, exceptions have a path, it runs on live systems, it has an owner, you know the running cost at full volume, and you've mapped how each other site does the work.
• Treat the second site as a new test with the same bar, then roll out in waves with a written stop rule. Switch the old way off site by site, not all at once.

Table of Contents

• What does scaling an AI pilot to production mean?
• Why results shrink when a pilot scales
• Two cases where working AI didn't travel
• What changes between one team and the whole business
• Is your pilot ready to scale?
• How to roll out an AI workflow site by site
• Who owns an AI workflow once it's everywhere
• A 140-person injection molder scales its NCR drafts
• When not to scale a working pilot
• Where Avolis fits
• Frequently Asked Questions
• Continue Learning

What Does Scaling an AI Pilot to Production Mean?

Scaling an AI pilot to production means the workflow that passed its test with one group becomes the standard way the whole business does that work. Every team, branch, shift, and job type that handles the work uses it, the old way is switched off everywhere, and it keeps hitting its number without the person who set it up watching over it.

That is a bigger claim than "the pilot worked." A pilot that graduated has shown the AI can do the job under one set of conditions: one team's habits, one site's paperwork, one manager's attention, and usually a few months of the volume the full business produces. Scaling asks whether the result holds when every one of those conditions changes at once.

For a 10-to-200-person business, scaling rarely means thousands of users. It usually means a second branch, a night shift, an acquired location that still runs its own way, or the job types the pilot skipped because they were messier. Those are small numbers, and they still break a surprising share of AI workflows, because each new site brings conditions the pilot never saw.

Why Results Shrink When a Pilot Scales

Results shrink at scale for reasons economists have studied well outside AI. John List and his co-authors call it the "voltage effect," which they define as "the tendency for a program's efficacy to change when it is scaled up," and they note that in most cases the effect gets smaller (Al-Ubaydli, Lai and List, "A Simple Rational Expectations Model of the Voltage Effect", NBER Working Paper 30850, 2023). Their earlier paper sorts the causes into three groups, which map closely onto an AI pilot (Al-Ubaydli, List and Suskind, "The Science of Using Science", NBER Working Paper 25848, 2019).

Was the result real in the first place? A pilot judged on one team over a few weeks rests on a small sample. A good month, an unusually easy batch of jobs, or a team that tried harder because it was being watched can all make a result look stronger than it is. The fix is to check that the pilot's number held over a long enough stretch of ordinary work before anyone talks about rolling it out.

Was the pilot team typical? Pilots go to the team most likely to make them work, which is a sensible way to run a pilot and a poor way to predict a rollout. The keenest project manager, the tidiest branch, and the shift lead who asked for the tool are all different from the people who'll get it next, and the gap usually shows up as lower use and more workarounds.

Did the situation travel with it? Some of what made the pilot work never gets written down. The champion answered questions within minutes, fixed odd outputs the same day, and quietly cleaned up inputs before they reached the AI. None of these ingredients scale on their own, and in an AI pilot the champion's attention is usually the biggest one.

The second group, differences in the population, matters inside one business too. A study run with the U.S. Census Bureau surveyed "structured" management practices (how targets, monitoring, and incentives are run) at 32,000 U.S. manufacturing plants and found "an enormous dispersion" of practices, with 40% of that variation across plants within the same firm (Bloom, Brynjolfsson, Foster, Jarmin, Patnaik, Saporta-Eksten and Van Reenen, "What Drives Differences in Management?", NBER Working Paper 23300, 2017). Two branches with the same logo can be managed in noticeably different ways, with different targets, checks, and habits, and an AI workflow tuned to one branch meets all of those differences when it moves to the next.

Two Cases Where Working AI Didn't Travel

Two well-documented cases from healthcare show the problem clearly, because hospitals and clinics keep the kind of records that make a drop in results measurable.

A model that learned the hospital. In 2018, researchers trained image-reading models to spot pneumonia on chest X-rays drawn from three hospital systems, 158,323 images in all. The model trained on two of the systems combined scored an AUC of 0.931 on held-out X-rays from the hospitals it was trained on and 0.815 at an outside hospital. AUC is a score of how well a model separates positive cases from negative ones, where 1.0 is perfect and 0.5 is a coin flip. Performance was significantly lower on outside X-rays in 3 of 5 comparisons. The same kind of model could tell which hospital system an X-ray came from with 99.95% accuracy for one system and 99.98% for another, and pneumonia was far more common at one hospital (34.2%) than at the others (about 1%). The models had partly learned to recognize the hospital, which didn't help them at a hospital they hadn't seen (Zech et al., "Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs", PLOS Medicine, 2018).

A system that met the clinic. Google researchers spent eight months making regular visits to 11 clinics in Thailand where nurses used an AI system to screen patients for diabetic eye disease. They found tension between the model's thresholds for image quality and the images an imperfect, busy clinic actually produces. Lighting varied from clinic to clinic, some images came out blurred or dark, and the system conservatively called some of them "ungradable." The team changed the process so eye specialists reviewed those images alongside the patient's records instead of referring every such patient automatically (Beede et al., CHI 2020; Google, "Healthcare AI systems that put people at the center", April 2020).

Neither case is about a 60-person contractor, and the AI involved was built differently from the drafting and sorting tools most small businesses use. The lesson still carries over directly. In both cases the AI worked where it was built and tested, and what changed at the next site was the inputs (a different hospital's X-rays, different lighting) and the people around the system (different patients, different routines). A second branch with its own paperwork, codes, and habits is the same kind of change on a smaller scale.

What Changes Between One Team and the Whole Business

What changes at scale is almost everything around the AI, while the AI itself often stays the same. The table below sets the pilot's conditions next to what the full rollout has to handle.

The last row matters more than it looks. If the rollout is judged on a company-wide average, a strong original team can hide a site where the workflow barely works, and that site's people will drift back to the old way without anyone noticing. Our guide to measuring AI ROI explains how to build the number from records, and at scale you need it per site.

Cost changes too. Many AI tools are priced per use, per document, or per seat, so a workflow that cost little on one team's volume can cost several times as much on the whole business's. Our page on why AI projects are failing now covers why usage pricing catches teams out, and the fix here is simple: price the full volume before the rollout starts, not after the first invoice.

Is Your Pilot Ready to Scale?

A pilot is ready to scale when its result has held without special help and the work at the other sites has been mapped. These six checks take a few days for one workflow, and each one answers a question the rollout will otherwise answer the hard way.

If the first five pass and only the last is missing, you're close, and the mapping is the next job. If the result only holds while the champion is involved, the pilot hasn't really graduated yet, and the right move is to fix that on the original team before adding anyone else. Our page on why an AI consulting project fails to reach production covers the integration and exception gaps that most often explain it.

How to Roll Out an AI Workflow Site by Site

Roll out an AI workflow by testing it at one more site against the same bar the pilot passed, fixing what that site reveals, and then moving through the rest in waves with a written rule for when to stop. The sequence below works whether "site" means a branch, a plant, a shift, or a job type the pilot didn't cover.

Step 1: Map how each other site does the work. Spend an hour with the person who runs the work at each site and walk through the last few real jobs, not the official process. Write down the forms, codes, systems, file formats, and who does which step. Most of the differences that break a rollout show up here: a branch that still uses paper sheets, a shift with no reviewer on hand, an acquired location with its own job codes.

Step 2: Decide what to standardize and what to adapt. For each difference, pick one of two answers. Standardize it when the difference is an accident of history, such as two code lists that describe the same defects with different names. Adapt the workflow when the difference is real, such as a customer that requires its own report format. Standardizing is often worth doing for its own sake, and it's usually cheaper than teaching the AI every local variation.

Step 3: Pick a typical second site, not the keenest. The second site should look like most of the remaining sites, because its result is the one that predicts the rollout. Choosing the most enthusiastic manager again would repeat the pilot's selection problem and tell you little you don't already know.

Step 4: Test the second site against the same bar. Use the measure and the level the pilot passed, measured from that site's own records, over four to six weeks of normal work. Give the site a named local contact and a short training, and keep the champion's involvement to what every later site will get. If the site misses the bar, go back to Step 2 with what you learned instead of pushing on.

Step 5: Roll out the rest in waves, with a stop rule. Group the remaining sites into two or three waves and write down before the first wave what would pause the rollout, for example a site falling below the bar for two weeks running or exceptions rising above a set count. A written rule means a weak site gets attention before the next wave starts.

Step 6: Switch the old way off at each site once it passes. Retire the old template, spreadsheet, or process for that site when it has met the bar, and not before. Doing this site by site keeps a fallback where the workflow is still settling in, and doing it at all stops the old way from quietly becoming the default again.

Who Owns an AI Workflow Once It's Everywhere

Once an AI workflow runs everywhere, it needs one owner for the whole workflow and one named person at each site. The owner is usually the manager responsible for the work across the business, such as the operations manager or the quality manager, and their job is the number: they review it for every site each month, decide what changes, and approve the running cost.

The person at each site handles that site's exceptions, answers questions from the people using the workflow, and raises problems early. Often this is a lead or senior hand who already knows the work, and the role takes a small, regular slice of their week rather than a new job.

The champion who built the pilot usually becomes a resource for the owner rather than the owner. That's a change worth saying out loud, because champions often feel the workflow is theirs, and they're usually the right person to train the site contacts and help with the hardest exceptions. What they can't do is support every site personally, so the rollout plan shouldn't depend on them doing it.

A 140-Person Injection Molder Scales Its NCR Drafts

Consider a 140-person custom injection molder with two plants about 40 miles apart, the second acquired three years earlier and still run largely its own way (an illustrative composite). When a part fails inspection, a quality technician writes a nonconformance report, or NCR, that records the defect, the affected lot, how the parts were contained, and what happens to them next. NCRs go to customers and drive internal fixes, and slow ones hold up shipments.

The pilot ran on Plant 1's day shift. The AI drafted each NCR from the technician's notes, photos, and the inspection record in the quality system, and the technician edited and issued it. After ten weeks it graduated against a two-part bar: at least 70% of drafts issued with under ten minutes of editing, and a median of two business days or less from defect found to NCR issued. Plant 1's day shift hit 80% and one business day, against three days before the pilot, and the numbers held for six weeks while the champion went back to normal work. That shift produced about 60 of the company's roughly 190 NCRs a month.

The first attempt to scale treated the rollout as a copy. The quality manager turned the workflow on for Plant 1's night shift and for Plant 2 in the same week.

The night shift mostly worked. Drafts were nearly as good as on days, at 78% issued with light editing. The problem was review: the night lead technician hadn't been trained on the workflow and left drafts for the day shift, so the median issue time on nights stayed near three days. A one-hour training and a rule that the night lead issues routine NCRs and flags only the unusual ones brought nights to the bar within three weeks, and it held there through a six-week check.

Plant 2 missed badly. Only 35% of drafts went out with light editing. The pilot had held without its champion, so the gap was the site map, and mapping the plant, which should have happened first, showed why. Plant 2 used its own list of 31 defect codes against Plant 1's 22, and several shared a name but meant different things, so the AI's drafts used the wrong code for that plant. Its inspectors recorded results on paper sheets that technicians photographed, which gave the AI blurry, partial inputs. And two of Plant 2's largest customers required NCRs on their own forms.

The fixes were mostly standardization, not AI work. The quality manager and the two plants' lead technicians agreed on one shared list of 26 defect codes in about two weeks, which both plants had wanted for years. Plant 2's inspectors moved to the same tablet inspection form Plant 1 already used, which took three weeks. The two customer forms became templates the AI fills in when the order belongs to those customers. Six weeks into the retest, Plant 2 reached 74% of drafts issued with light editing and a median of one and a half business days, against four days before.

The running cost rose with volume, from about $350 a month during the pilot to about $1,050 at all 190 NCRs, which the owner approved before Plant 2's retest started. The quality manager now owns the workflow across both plants and reviews each plant's two numbers monthly, and each plant's lead technician handles that plant's exceptions. The paper NCR templates were retired at Plant 1 after the night shift's six-week check and at Plant 2 after it passed.

When Not to Scale a Working Pilot

Some pilots should stay where they are, and the test is whether the other sites have enough of the same work, done in a similar enough way, to pay for the rollout. A workflow that passed on one team is worth keeping there even if it never spreads.

Keeping it narrow is the right call when the other sites do the work too differently to standardize without a bigger change than the AI is worth. The same goes when their volume is too small to cover the running cost and the support, or when the pilot's result depended on one person's judgment that the other sites don't have. It's also right when the other sites are about to change systems, in which case you map them again after the change rather than building for a system that's on its way out.

A narrow workflow that works is a good result. The mistake is forcing it onto sites where it will run below the bar, because a site where the AI barely works teaches its people that the AI doesn't work, and that makes the next workflow harder to introduce there. Our guide to recovering a failing AI project covers what to do when a rollout has already gone that way.

Citation-ready summary: Scaling an AI pilot to production means making a workflow that worked for one team the standard way the whole business does that work, at full volume, without the pilot's champion. Results often shrink at scale, a pattern economists call the voltage effect, because pilots run with the keenest team and the cleanest inputs, and because sites within one business differ: in a Census survey of 32,000 U.S. plants, 40% of the variation in management practices was between plants of the same firm. Check that the result holds without the champion, map how each site does the work, test a typical second site against the same bar, then roll out in waves with a written stop rule.

Where Avolis Fits

Every engagement starts with our diagnostic. For a business with more than one branch, plant, or shift, we work through the workflow with the people who run it for each location, measure how it runs at each one (volumes, time per task, handoffs), and map where the sites differ before anything is built. You keep the ranked result and the site map whatever you decide next.

When we build, we plan the rollout from the start: the six readiness checks, the second-site test, the waves, and the stop rule are part of the plan, and each site gets its own number. When we stay on after launch, keeping the workflow at its bar at every site, and fixing the site that slips, is part of our job. Our guide to AI readiness assessment services explains what a good diagnostic should produce, and our guide to which AI consulting company to choose covers the questions that show whether a partner plans past the first site.

Frequently Asked Questions

What does it mean to scale an AI pilot to production?

It means making a workflow that worked for one team, site, or shift the standard way the whole business does that work. Every location uses it at full volume, the old process is switched off everywhere, and it keeps hitting its number without the person who set it up. A pilot that graduated has only shown it works under one set of conditions.

Why do AI pilots perform worse when they scale?

Pilots usually run with the most willing team, the cleanest inputs, and a champion who fixes problems quickly, and none of that comes along to the next site. Sites also differ more than owners expect, in forms, codes, systems, and habits. Economists call the drop in results at scale the voltage effect.

How do you scale an AI pilot across multiple locations?

Check that the result holds without the champion, map how each other location does the work, and decide which differences to standardize and which to adapt. Then test one typical location against the same bar the pilot passed, fix what it reveals, and roll out the rest in waves with a written stop rule, retiring the old process site by site.

How long does it take to scale an AI workflow?

For a business of 10 to 200 people, the second-site test usually takes four to six weeks of normal work, plus however long it takes to fix the differences it uncovers. Standardizing codes or forms can take a few weeks on its own. The remaining sites often follow in two or three waves over the next few months.

Who should own an AI workflow after it scales?

One manager should own the workflow and its number across the whole business, usually the person already responsible for that work, such as the operations or quality manager. Each site also needs a named person who handles its exceptions and questions. The pilot's champion is usually better used as a trainer and resource than as the owner.

Continue Learning

A pilot that works for one team has answered the first question, whether AI can do this job. Scaling answers a different one: whether the result survives other people, other paperwork, and full volume without the help the pilot had. Mapping the sites first and testing the second one against the same bar gives you that answer before the whole business depends on it.

If you have a graduated pilot and a second site in mind, spend an hour this week walking through that site's last few jobs and write down every way its version of the work differs.

Why projects fail, in detail:

• Why do AI projects fail? Common reasons and statistics
• AI pilot purgatory: why pilots never end, and how to get out
• Why is my AI consulting project failing to reach production?
• How to recover a failing AI project
• Why are AI projects failing?

Starting the right way:

• AI readiness assessment services
• Which AI consulting company should I choose?
• How to measure AI ROI

---

Sources

• Omar Al-Ubaydli, Chien-Yu Lai and John A. List, "A Simple Rational Expectations Model of the Voltage Effect," NBER Working Paper 30850, January 2023, abstract retrieved 2026-11-02: https://www.nber.org/papers/w30850
• Omar Al-Ubaydli, John A. List and Dana Suskind, "The Science of Using Science: Towards an Understanding of the Threats to Scaling Experiments," NBER Working Paper 25848, May 2019, abstract retrieved 2026-11-02: https://www.nber.org/papers/w25848
• Nicholas Bloom, Erik Brynjolfsson, Lucia Foster, Ron S. Jarmin, Megha Patnaik, Itay Saporta-Eksten and John Van Reenen, "What Drives Differences in Management?," NBER Working Paper 23300, March 2017, abstract retrieved 2026-11-02: https://www.nber.org/papers/w23300
• John R. Zech et al., "Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study," PLOS Medicine 15(11), November 2018, retrieved 2026-11-02: https://pmc.ncbi.nlm.nih.gov/articles/PMC6219764/
• Emma Beede et al., "A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy," Proceedings of CHI 2020, abstract retrieved 2026-11-02: https://research.google/pubs/a-human-centered-evaluation-of-a-deep-learning-system-deployed-in-clinics-for-the-detection-of-diabetic-retinopathy/
• Google, "Healthcare AI systems that put people at the center," 25 April 2020, retrieved 2026-11-02: https://blog.google/technology/health/healthcare-ai-systems-put-people-center/

On the scaling papers. Both NBER papers study the scaling of programs and policies in general, not AI. The three groups of causes (whether the evidence is actionable, properties of the population, properties of the situation) are from the 2019 abstract, and the voltage-effect definition is quoted from the 2023 abstract.

On the management survey. The Census Management and Organizational Practices Survey measures "structured" management practices such as monitoring, targets, and incentives, not how specific tasks are done. The 40% figure is the share of variation in those practices that sits across plants within the same firm.

On the pneumonia study. The 0.931 and 0.815 AUC figures compare the jointly trained model on held-out data from its own two hospital systems with its performance at Indiana University. Prevalence was 34.2% at Mount Sinai against 1.2% at NIH and 1.0% at Indiana University.

On the injection molder. It's an illustrative composite, not a client. Its figures (140 people, two plants, 40 miles, acquired three years earlier, 60 and 190 NCRs a month, ten weeks, the 70% and two-day bar, 80%, 78%, 35%, and 74%, three, four, one, and one and a half business days, six-week checks and retest, a one-hour training, 31, 22, and 26 defect codes, two customer forms, two and three weeks, $350 and $1,050 a month) are representative examples, not measurements or benchmarks.

---

About Avolis Research Group

Avolis Research Group is Avolis's in-house research practice, focused on how operations-heavy small and mid-sized businesses actually adopt AI. It synthesizes primary economic research, government survey data, and results from real implementations into practical, vendor-neutral guidance.

More about Avolis and how we work · Get in touch