Posted by DON’T LET THE FINANCIAL CRISIS HAPPEN TO YOU: WHY ESTIMATES USING POWER CERS ARE LIKELY TO EXPERIENCE COST GROWTH
Risk Track
Downloadable Files:
Abstract:
Kurtosis risk is the risk embodied in wider (fat-tailed) distributions. This is well known in the finance industry since the failure to account for it led to the collapse of the Long-Term Capital Management hedge fund in the late 1990s.
Kurtosis risk occurs when the probability of extreme values is underestimated in a risk model. The cost analysis community has recently begun to acknowledge this risk; fat-tailed distributions for cost risk analysis have been proposed in various papers presented over the last few years. This paper will focus on two cost estimating errors, related to kurtosis risk, that have likely contributed to cost growth on every program where non-linear relationships are used to estimate cost. The paper will begin with a demonstration that, when point estimates based on non-linear relationships are summed together to develop a total program level cost estimate, that estimate is invariably at a confidence level lower than the 50th percentile.
The paper will then demonstrate that, using the most common CER development technique, ordinary least squares regression in log space, and then assuming a lognormal distribution in unit space, causes a large underestimation of all percentiles, as well as the mean itself, to occur. This logic will be expanded to demonstrate how this estimating error, when compounded on multiple cost elements or programs (for portfolio management purposes) will lead to a mis- estimation of costs at all percentiles and thus will reduce a program, or portfolio’s chance of success.
The paper will conclude with the recommendations to address both of these problems including the integration of cost estimating and risk analysis efforts and the use of fat-tailed uncertainty distributions when OLS regression techniques are used to estimate costs.
Author(s):
Eric Druker
Eric R. Druker CCE/A graduated from the College of William and Mary with a B.S. in Applied Mathematics in 2005 concentrating in both Operations Research and Probability & Statistics with a minor in Economics. He is employed by Booz Allen Hamilton as a Senior Consultant and currently serves on the St. Louis SCEA Chapter’s Board of Directors. Mr. Druker performs cost and risk analysis on several programs within the Intelligence and DoD communities and NASA. He was a recipient of the 2005 Northrop Grumman IT Sector’s President’s award and the 2008 TASC President’s award for his work on Independent Cost Evaluations (ICEs) during which he developed the risk process currently used by Northrop Grumman’s ICE teams. In addition to multiple SCEA conferences, Eric has also presented papers at the Naval Postgraduate School’s Acquisition Research Symposium, DoDCAS and the NASA PM Challenge. He has also performed decision tree analysis for Northrop Grumman Corporate law, built schedule and cost growth models for Hurricane Katrina Impact Studies and served as lead author of the Regression and Cost/Schedule Risk modules for the 2008 CostPROF update.
Richard Coleman
Richard L. Coleman is a Naval Academy graduate, received an M. S. with Distinction in Operations Research from the U. S. Naval Postgraduate School and retired from active duty as a Captain, USN, in 1993. His service included tours as Commanding Officer of USS Dewey (DDG 45), and as Director, Naval Center for Cost Analysis. At Northrop Grumman, he has worked extensively in cost, CAIV, and risk for the Missile Defence Agency (MDA), Navy ARO, the intelligence community, NAVAIR, and the DD(X) Design Agent team. He has supported numerous ship programs including the DDG 1000 class, DDG 51 class, Deepwater, LHD 8 and LHA 6, LPD 17 class, Virginia class SSNs, CVN 77, and CVN 78. He is the Director of Independent Cost Estimation for Northrop Grumman Information Systems and conducts Independent Cost Evaluations on Northrop Grumman programs. He has more than 65 professional papers to his credit, including five ISPA/SCEA and SCEA Best Paper Awards and two ADoDCAS Outstanding Contributed Papers. He was a senior reviewer for CostPROF and CEBoK and lead author of the Risk Module. He has served as Regional and National Vice President of SCEA and is currently a Board Member. He received the SCEA Lifetime Achievement Award in 2008.
Peter Braxton
Peter J. Braxton holds an AB in Mathematics from Princeton University and an M.S. in Applied Science (Operations Research) from the College of William and Mary. He currently works as a cost and risk analyst as part of the Independent Cost Estimation
(ICE) function within the new Northrop Grumman Information Systems sector. He has worked to advance the state of knowledge of cost estimating and risk analysis, Cost As an Independent Variable (CAIV), and Target Costing on behalf of the Navy Acquisition Reform Office (ARO), the DD(X) development program, and other ship and intelligence community programs. He has co-authored several professional papers, including SCEA/ISPA International Conference award-winners in CAIV (1999) and Management (2005). He served as managing editor for both the original development of the acclaimed Cost Programmed Review Of Fundamentals (CostPROF) body of knowledge and training course materials and its successor, the Cost Estimating Body of Knowledge (CEBoK). He serves as SCEA’s Training Chair and as a Northrop Grumman Technical Fellow. He was named SCEA’s 2007 Estimator of the Year for contributions in Education, and received both a TASC President’s Award and Northrop Grumman Corporate Contracts, Pricing, and Supply Chain Award in 2008.