Posted by Build Your Own Distribution Finder
Risk Track
Downloadable Files:
Abstract:
A basic step in a cost uncertainty analysis is to define the distribution of every uncertain element in the cost model. Identifying and then defending these distributions is a fundamental challenge. If data is available, the preference is always to perform a statistical analysis to arrive at an objective assessment. Several commercial tools are available for finding the distribution that best describes the shape and dispersion of the sample data set, but they are not in agreement.. Are they all correct? Have these different methods been subjected to an independent validation and verification (IV&V)? Is there a “best method” to derive the “best fit”?
In order to easily analyze hundreds of data sets in a consistent manner and present the results in a tailored form, a prototype utility was built in Excel to derive the parameters for the lognormal, normal, triangular and beta distribution that best fit a sample dataset. A variation on Excel’s PercentRank function is introduced and forms a key building block of the utility. Excel’s solver is used in the prototype and the motivation to search for an alternative is presented. The Chi squared statistic is used by at least one commercial tool as the metric for optimizing the distribution parameters. We examine it and several others as metric to optimize such as sum of squared error (SSE) and standard percent error (SPE). The pros and cons of each are presented and the rationale for the one selected is provided. The applicability of other “goodness of fit” tests are discussed.
We present the fit results in a compact and thorough format. Fitted distribution parameters compare favorably to commercial tools, and the math is provided for validation. The Chi squared test is used to assess the significance of the fitted distribution. The associated assumptions, math, and weaknesses of this “goodness of fit” test are discussed.
Author:
Alfred Smith
Tecolote Research, Inc.
Alfred earned a Bachelor Mechanical Engineering degree from the Canadian Royal Military College and a Master of Science with Distinction in naval architecture from the University College, London, England. He spent 21 years in the Canadian Navy driving submarines
(Navigator, Operations Officer) and ten years as a naval architect. He has over 20 years experience leading, executing or contributing to life cycle cost model development and cost uncertainty analysis for a wide variety of military, Coast Guard, NASA and foreign projects. He has been with Tecolote since 1995 and since 2000 has been the General Manager for Tecolote’s Software Products/Services Group, responsible for the development, distribution and support of a variety of web and desktop products including ACEIT. Alfred has delivered numerous papers on cost risk analysis topics and was the lead writer of the AFCAA Cost Risk and Uncertainty Handbook. He is a SCEA Certified Cost Estimator/Analyst (CCEA).