Posted by Fitting Absolute Distributions to Limited Data
Risk Track
RSK03_Presentation_FittingAbsoluteDisttoLimitedData_Boswell
Abstract:
The choice of probability distributions is a critical component for cost risk and uncertainty modeling. When data is available, distribution fitting techniques, such as Goodness of Fit (GoF) tests and Information Criteria (IC), can be applied to determine distributions that accurately describe potential cost realizations; however, with limited data GoF tests and IC based methods provide little or no insight into the best distribution choice. Therefore, when data is limited it is standard practice in cost risk and uncertainty modeling to solicit expert opinion in the construction of triangular distributions with vertices representing the best case, typical, and worst case scenarios. While this practice is relatively easy to conduct, the finite nature of the triangular distribution (particularly of the right tail) and expert bias can lead to model inaccuracies.
This paper will investigate the use of a new approach to distribution fitting, called Decision on Belief (DoB), to guide the choice of distributions in cost risk and uncertainty models when there is limited data. DoB, developed by Eshragh and Modarres, employs Bayesian statistics and elements of stochastic dynamic programming in the formulation of distribution fitting as a special case of Optimal Stopping problems. The objective of DoB is to select the best fit distribution from a set of candidate distributions with the maximum possible confidence. The method was developed to fit distributions to applications incurring high data collection costs such as pharmaceutical testing, and prototyping industrial products. This study will adapt DoB to cost risk and uncertainty analysis through the incorporation of standard cost analysis distribution fitting practices such as expert opinion, expert opinion bias adjustment, and observations from analogous systems. The study will conclude with an example that assesses the accuracy of distributions chosen by DoB against Triangular fits given the same limited information.
Author:
Blake Boswell
Booz Allen Hamilton
Blake Boswell is an Operations Research Analyst for Booz Allen Hamilton’s Decision Analytics Team. He has three years of experience developing tools and processes in support of analytical projects for a variety of clients in the Defense, Space, and Health industries. In 2010, Blake was recognized for Technical Achievement by the Washington D.C. Chapter of the Society of Cost Estimation and Analysis (SCEA) for his efforts in the application of numerical methods to cost risk simulation. In 2011, he was named National Estimator of the Year by SCEA for Technical Achievement in recognition of his research efforts in the development of Booz Allens RealTime Analytics service offering. Blakes research interests include applied probability, computational mathematics, and modeling & simulation. He is a frequent presenter in the Risk Track at SCEA conferences, and has published original research in a variety of journals. Blake has a B.S. in mathematics from Auburn University Montgomery and a Masters degree in applied economics from Johns Hopkins University.