Posted by Constructing Bounds for S-Curves
Risk Track
Downloadable Files:
Abstract:
After years of developing point estimates analysts knew were uncertain, the estimating and acquisition community has embraced the concept of viewing a cost estimate as a potential distribution of cost represented by a Cumulative Distribution Function (CDF), commonly called the S‐curve. Unfortunately the very thing the S‐curve was intended to counter—the implication of preciseness inherent in a point estimate—has become a preciseness about percentile values and levels of risk. Decision makers, for example, have come to view the 50th percentile as an absolute and wonder why the number budgeted at 50% is shown as 30% when the estimate is updated. While there are many reasons numbers change, part of the error is inherent in the way the S‐curve is developed. Ideally all input distributions in a cost estimate would be derived from reliable data and would have known shape and parameter values. In reality many inputs to an estimate are based on expert opinion and data of unknown relevance, making the distributions for these inputs uncertain. Understanding this we can treat the output CDF as an Empirical Distribution Function (EDF) and quantify this type two or epistemic uncertainty. Utilizing Kolmogorov‐Smirnov and non‐parametric quantile bounds, a “P‐Box” is developed from which an analyst can now define a range of costs associated with specific levels of risk or ranges in risk associated with specific costs.
Author(s):
Christopher Mehl
Omnitec Solutions, Inc.
Chris is a senior mathematician for Omnitec Solutions, Inc., supporting the NAVAIR 4.2 Cost Department. He has more than 11 years experience performing strategic research, technical and quantitative analysis in the academic, commercial, and government sectors, and has applied a wide variety of mathematical techniques to diverse problems in the intelligence and defense communities. He holds a Doctor of Philosophy in Applied Mathematics from the University of Colorado at Denver, and maintains an active program of research in risk, uncertainty analysis, and computational statistics techniques.
Michael Popp
NAVAIR 4.2 Cost Department
Mike is the subject matter expert for software cost estimation, avionic estimating and cost risk analysis techniques for the NAVAIR 4.2 Cost Department. During his 28 year career he has developed numerous cost estimates for a broad spectrum of platforms and sensors and has been leading research into improving software cost estimation for the past several years. He has a degree in Industrial Engineering from Virginia Tech, is level III certified in Cost Estimating and Program Management, is a certified cost estimator through the Society of Cost Estimating and Analysis (SCEA) and is the SCEA 2007 National Cost Estimator of the Year for technical achievement.