Use a reference class before you trust your own forecast

By Meet Patel · 2026-10-03 · 6 min read

Daniel Kahneman once worked on a team writing a curriculum for a new school subject in Israel. Everyone on the team wrote down how many months the project would take, and the estimates ranged from 18 to 30. Then a colleague asked the team’s curriculum expert to recall similar projects and say how long they had taken. The expert answered, with some discomfort, that about 40 percent of comparable teams never finished. Of those that did, none took less than seven years and none took more than ten. The team carried on anyway and finished eight years later, and the curriculum was rarely used (Flyvbjerg, “From Nobel Prize to Project Management,” 2006, retelling Lovallo and Kahneman).

The same expert had produced an estimate somewhere between 18 and 30 months and a fact pattern of seven to ten years, and he held both in his head without noticing the conflict. That conflict is the subject of this post, and the method that resolves it is called reference class forecasting.

Two views of the same project

Kahneman and Tversky’s work from 1979 found that people tend to underweight distributional information, which means the record of how similar efforts turned out. Flyvbjerg describes the result as an inside view: attention on the particulars of the project at hand, with scenarios built forward from its objectives, resources and obstacles. The outside view ignores those particulars and asks how a class of similar projects fared (Flyvbjerg, 2006).

Founders draft plans from the inside view. You know your team, your product and your pipeline, so you build the forecast from the components. The method feels rigorous because it is detailed. The weakness is that the detail contains only the events you thought of, and the delays that actually arrive tend to be the ones you did not.

What the base rates look like

The evidence for the outside view is large. In 2002, Bent Flyvbjerg, Mette Skamris Holm and Søren Buhl analyzed 258 transportation infrastructure projects worth $90 billion. Costs were underestimated in almost 9 out of 10 of them, and actual costs ran on average 28 percent above the estimate made at the decision to build. By type, rail averaged 44.7 percent over, fixed links such as bridges and tunnels 33.8 percent, and roads 20.4 percent. They also tested whether forecasters had improved and found no effect of time: underestimation was in the same range as it had been 10, 30 and 70 years earlier (Flyvbjerg, Holm and Buhl, “Underestimating Costs in Public Works Projects,” 2002).

Software shows a similar shape. Flyvbjerg and Budzier studied 1,471 IT projects and found an average cost overrun of 27 percent, with about one in six projects overrunning by 200 percent on average and slipping its schedule by almost 70 percent (Flyvbjerg and Budzier, Harvard Business Review, 2011). The average hides the tail, which is where companies get hurt.

For a founder, the outermost base rate is survival. Among U.S. private-sector establishments born in March 2013, 79.6 percent were still operating a year later, 50.6 percent after five years and 34.7 percent after ten (U.S. Bureau of Labor Statistics). That class is very broad, since it counts every kind of establishment, including many with no connection to venture-backed software, and an establishment closing is sometimes something other than a failure. As a prior it still says something useful: a plan that assumes the company is operating in year five assumes an outcome that about half of comparable establishments reached.

The method in three steps

Flyvbjerg sets out three steps for reference class forecasting (Flyvbjerg, 2006):

1. Identify a reference class of past, similar projects. It must be broad enough to be statistically meaningful and narrow enough to be truly comparable.
2. Establish the distribution of outcomes for that class, using credible data from enough projects.
3. Place your project in the distribution to find its most likely outcome.

In statistical terms, as Flyvbjerg puts it, you regress your best guess toward the average of the class and widen your credible interval toward the interval for the class. You stop forecasting the specific events that will hit your project and start asking where it sits among projects of its kind.

A worked example with a team’s own history

Founders often lack an external dataset, but they usually own a good reference class already: their own past projects. Take a hypothetical product team whose last six projects were estimated and delivered like this, in weeks.

• Estimated 4, delivered 7 (ratio 1.75)
• Estimated 6, delivered 9 (ratio 1.50)
• Estimated 3, delivered 6 (ratio 2.00)
• Estimated 8, delivered 10 (ratio 1.25)
• Estimated 5, delivered 9 (ratio 1.80)
• Estimated 10, delivered 14 (ratio 1.40)

Sorted, the ratios run 1.25, 1.40, 1.50, 1.75, 1.80, 2.00, and the median is 1.625. The team now plans a new feature and its inside-view estimate is 8 weeks. The outside view gives a median forecast of 8 times 1.625, which is 13 weeks, with a plausible range of 10 weeks (8 times 1.25) to 16 weeks (8 times 2.00). The commitment to the sales team should be the 16-week end of that range, or the team should shrink the scope until the 8-week estimate stops being a stretch.

The same arithmetic works for revenue plans. If a company hit 3 of its last 8 quarterly targets, the base rate for hitting the next one is 3 in 8, or 37.5 percent, and a plan that treats the target as a 90 percent event has to name what changed. The figures here are illustrations, and the point is the procedure.

Choosing a class without fooling yourself

The method has failure modes, and most of them live in step one.

• A class that is too small. Two past projects form a story, and ten or more start to form a distribution.
• A class made of survivors. Studying the companies that succeeded tells you what success looked like, and says nothing about how many tried the same thing and failed.
• Adjusting too early. In Kahneman’s story the expert judged his team slightly below average and the team still proceeded. Adjust away from the class mean only for features you can name and evidence, such as a measured difference in team experience.
• Using it for the wrong project. Flyvbjerg notes that the outside view may fail to predict extreme outcomes outside all historical precedent, and that choosing a class is harder where precedents are not easily found (Flyvbjerg, 2006). A genuinely new category needs the closest analogues you can find and a wider interval.

The inside view still has a job after the base rate is set. It tells you where in the distribution you have reasons to sit, and it names the specific risks to manage. The order matters, and I would anchor on the class first and let the details move the estimate second.

A checklist before you commit to a plan

1. Write the inside-view estimate and the date you made it.
2. Name the reference class in one sentence, and count the cases in it.
3. Compute the ratio of actual to estimate for each case, then the median and the range.
4. Apply the median to your estimate, and quote the range as the commitment window.
5. List the differences between this project and the class that you can support with evidence, and move the estimate only for those.

This pairs well with a pre-mortem, which looks for the specific causes of failure while the base rate sizes the odds. The sibling post on how to run a pre-mortem covers that half. The base rate also helps with the hold-or-fold question in the pivot trap, and with the early-stage uncertainty described in the zero-to-one problem, where precedents are thin and a wide interval is the honest answer.

The principle I would keep is that your plan is one draw from a distribution that other people have already sampled. Finding that distribution takes an afternoon of work with data most teams already hold, and it should happen before anyone commits to a date.

Written by Meet Patel — startup operator and growth strategist in Dubai.

Read on themeetpatel.com