Posted by How to Make Your Point Estimate Look Like a Cost-Risk Analysis
Downloadable Files:
Abstract
Until recently, sponsors and managers of space-system and other high-technology systems under consideration expected cost estimators to provide “point” estimates of costs of various architectures and/or designs at each program decision point, from the initial trade-study stage to source selection and right through to the completion of the project. Unfortunately, the “point estimate” was never precisely defined – the term never meant the same thing to everybody. For example, was it the “most likely” cost, the 50% confidence cost, the “average” cost, or what? Uncertainty and outright confusion regarding what useful information about system cost this “point” estimate was conveying made it virtually useless as a figure of merit for making decisions regarding comparing competing options, as well as for planning system budgets.
Until well into the 1990s, estimating was done using the “accounting” approach, whereby each WBS element’s estimated cost is considered to be its “most likely” cost, and then the most likely costs of all WBS elements are summed (“rolled up”) to yield an estimate of total program cost. While this particular estimate was often advertised as the “most likely” program cost, it was in fact not and, in addition, the confidence that the program could be delivered at that cost was typically somewhere between 20% and 30%. While generally thought to be a new phenomenon in the 1990s, the difficulties associated with this particular method were recognized in France as early as 1952 (R. Giguet and G. Morlat, “The Causes of Systematic Error in the Cost Estimates of Public Works,” Annals of Bridges and Roads, No. 5, September-October 1952, Paris, France; Translated into English from the French by W.W. Taylor, U.S. Air Force Project RAND, Santa Monica CA, March 1958).
The solution to the problem lies in treating the entire cost-estimating process statistically, a course of action that has come to be known as “cost-risk analysis.” (In fact, even the use of the term “most likely” a fortiori implies that other estimates are “less likely” and is therefore an admission that we are facing a statistical situation.) Probability distributions must be established to model the cost of each WBS element, correlations among these distributions estimated, and the distributions summed statistically, typically by Monte Carlo sampling. The result will be a probability distribution of total system cost, from which meaningful estimates of the median (50% confidence), 70th percentile (70% confidence), and other relevant quantities can be obtained. Without these confidence levels associated with each possible dollar value of cost, it is difficult to use cost estimates in the decisonmaking process.
Nevertheless, it is still quite common for cost estimates in the defense and aerospace sector to be based on the procedure of summing most likely costs. For those who don’t want to do a cost-risk analysis, but who want the benefits of having done so, such as levels of confidence associated with a range of estimates, this report offers a method of doing so.
Author(s)
Stephen A. Book