Posted by An Analytical Framework for Cost Estimating Relationship Development
Models Track
Downloadable Files:
Abstract:
Cost estimating relationship (CER) development, particularly for spacecraft hardware, has traditionally been method-centric. Much attention has been paid to parameter calculation and goodness-of-fit statistics for particular methods. However, parametric modeling involves other critical factors that are often overlooked, including model validation. These factors are discussed, and a complete analytical framework for developing CERs is presented. Three popular methods for spacecraft hardware CER development are compared in the context of this framework, namely Minimum Unbiased Percent Error (MUPE), Minimum Percent Error with Zero Percent Bias Constraint (MPE- ZPB), and log-transformed Ordinary Least Squares (LOLS). Optimality considerations, goodness-of-fit statistics, and bias, which have been thoroughly studied for each method, are summarized. Model validation and selection involves analysis to ensure that a model is consistent with the underlying data. Issues are raised regarding the conditional variance of the estimates produced by each method when applied to NASA data. In particular, NASA cost data are not symmetrically distributed, and the applicability of each technique for modeling skewed data is discussed. The relationships between all the methods and the method of maximum likelihood are pointed out, as this is important for understanding the validity of these techniques for skewed data. Robustness and consistency of each method for spacecraft data are addressed, and the importance of developing robust CERs is highlighted. Parametric modeling techniques are not unique to cost analysis of spacecraft and weapons systems, but are widely used in other industries as well. Comparisons of the three methods with current, modern techniques for parametrically modeling skewed data in other industries is presented, including insurance, health care, and labor economics.
Author(s):
Christian Smart
Dr. Christian Smart is currently employed as a senior parametric cost analyst and program manager with Science Applications International Corporation. An experienced estimator and analyst, he is responsible for risk analysis and cost integration for NASA’s Ares launch vehicles. Dr. Smart is in charge of overseeing the development of the NASA/Air Force Cost Model and has developed numerous cost models and techniques that are used by Goddard Space Flight Center, Marshall Space Flight Center, and NASA HQ. Dr. Smart is a past president of the Greater Alabama Chapter of SCEA, and has served as a reviewer for The Journal of Cost Analysis and Parametrics. He has given numerous presentations on cost modeling and risk analysis both in the U.S. and abroad. Dr. Smart was cited as 2006 Professional of the Year for the Greater Alabama Chapter of SCEA and was awarded best of conference paper at the 2008 Annual Joint ISPA-SCEA conference in Noordwijk for “The Fractal Geometry of Cost Risk”. He is a SCEA certified cost estimator/analyst (CCEA), a member of the Society for Cost Estimating and Analysis (SCEA) and the International Society of Parametric Analysts (ISPA). Dr. Smart earned bachelors degrees in Economics and Mathematics from Jacksonville State University, and a Ph.D. in Applied Mathematics from the University of Alabama in Huntsville.
George Culver
Mr. George A. Culver is currently employed as a senior parametric cost analyst and manager with Science Applications International Corporation. His recent contributions to the cost community consist of a number of updates to the latest version of the NASA Air Force Cost Model including updating cost estimating relationships, research on scheduling trends, and research into structures technology maturity. Mr. Culver has also led the development of three system-of-systems modeling frameworks applying multi-disciplinary optimization techniques to assess lunar and Mars architectures and missions for NASA HQ, Langley Research Center, and Johnson Space Center. Prior to his current job with SAIC, Mr. Culver served on active duty for the U.S. Air Force. He is a member of the Society for Cost Estimating and Analysis and the Alabama Modeling and Simulation Coalition.
Mr. Culver earned a bachelors degree in Space Operations from the U.S. Air Force Academy, a Master of Business Administration degree from Auburn University of Montgomery, and a Masters degree in Modeling & Simulation from the University of Alabama in Huntsville.