The Production Distance Model
In one sentence: the gap between a working proof of concept and a production service is five separate distances, and teams only ever measure one of them.
The problem
An organisation runs a proof of concept. It works. Everyone is pleased. Then nothing happens for nine months.
The usual explanation is that the team lacked engineering capacity. That is occasionally true and usually wrong. What has actually happened is that the proof of concept demonstrated technical feasibility and nothing else, and every other kind of readiness is still at zero. Nobody measured those, so nobody planned for them, so the gap appears as an unexplained delay.
The model
Production distance is the sum of five gaps, not one.
Technical distance. Can it run reliably on supported infrastructure, with monitoring, failure handling and a deployment path? A notebook on a laptop is maximum distance even when the output is perfect.
Control distance. How many control functions must review this, what evidence does each need, and does that evidence exist? In a regulated organisation this is usually the largest of the five and the least visible during the pilot.
Operational distance. Who runs this on a Tuesday when it breaks? Who is paged, what is the runbook, what is the service expectation, and has that team agreed?
Consumption distance. Does anything downstream actually consume the output in a real process? A model producing predictions nobody acts on is not in production in any sense that matters.
Funding distance. Is there a committed owner paying for it beyond the pilot, with the ongoing cost understood rather than assumed to be near zero?
The insight the model exists to deliver: these are largely independent. A technically excellent pilot can sit at zero on the other four. Teams optimise the dimension they can see, which is almost always the technical one, and are then surprised by a delay caused entirely by the four they did not measure.
Scoring
Score each dimension nought to five.
- 0: not started, and nobody has been asked
- 1: someone has raised it
- 2: the requirement is understood but unmet
- 3: work is underway with a named owner
- 4: substantially met, with a known gap
- 5: met and evidenced
Total out of 25. The number is less useful than the shape. A profile of 5-1-0-2-1 tells you exactly where the next quarter should go, and it is not engineering.
When to use it
At the start of a proof of concept, not the end, and with the control functions in the room rather than on their behalf.
Also useful in three other moments: when a piece of work has stalled and nobody can say why; when a leadership group asks why something "nearly finished" is not live; and when deciding whether to run a pilot at all.
The rule that gives it value
The pilot is not finished when the technical dimension reaches five. It is finished when you know the score on all five and have a named owner for each gap.
Two consequences follow. Control distance work can start during the pilot, in parallel, because the requirements are knowable in advance. And a proof of concept with high consumption and funding distance should often not be run at all, however interesting it is.
Limitations
This model measures distance. It does not tell you whether the destination is worth reaching, and it will happily help you move a bad idea into production efficiently. Value assessment is a separate exercise and must come first.
It assumes control requirements are knowable in advance. In genuinely novel territory, and generative AI was exactly that for a period, they are not, and the scoring becomes a guess. There the model is still useful as a conversation structure and useless as a measurement.
The scores are judgement, not measurement. Two people will score the same work differently, and the value is in the conversation that disagreement produces rather than in the number.
It is weakest in small organisations, where control distance is near zero and the whole thing collapses into ordinary engineering planning.
An example
A team has a classification model that performs well and has been demonstrated to stakeholders twice. Scored honestly: technical four, control one, operational zero, consumption two, funding one.
The instinct is to spend the next quarter on engineering. The model says the opposite. Technical is already the strongest dimension. The quarter is better spent establishing who will own the service, which control reviews apply and what they need, and whether the business process downstream is prepared to change.
The failure mode this prevents is the most common one in enterprise machine learning: a beautifully engineered service that arrives at a governance review having never spoken to it, and waits.
References
- D. Sculley and others, "Hidden Technical Debt in Machine Learning Systems". NeurIPS, 2015. The paper that put a figure on it: model code is a small fraction of a production machine learning system, and the rest is the part nobody scopes
- Federal Reserve and OCC, "SR 11-7: Guidance on Model Risk Management". 2011. United States guidance, from the Federal Reserve and the OCC. It is cited here because it is the clearest public statement of the expectation that a model is validated, documented, owned and monitored, which is most of what control distance turns out to mean in a bank. New Zealand has no direct equivalent for model risk. What binds locally is the Privacy Act 2020 for personal information, alongside the Reserve Bank of New Zealand's prudential requirements, and for Australian-owned banking groups the APRA prudential standards