Posted by Significant Reasons to Eschew Log-Log OLS Regression when Deriving Estimating Relationships
Methods and Models II Track
MMT210_Presentation_SignificantReasonstoEschewLog-LogOLSRegressionwhenDerivingEstRelationships_Book
Abstract:
Log-Log Ordinary Least Squares (LLOLS) regression, considered in the 18th and early 19th Centuries as the best (and, in fact, the only) method for fitting nonlinear algebraic relationships of the form y = axb to data sets of (x,y) pairs, has a number of serious defects that make it far from adequate for CER development in the 21st. No other option was available 200 years ago, but the advances in computing power and techniques of statistical optimization available to us today leave no reason to stick with an obsolete method. In the 21st Century, we insist that 21st Century engineering technologies be applied, so why would we continue to develop CERs to estimate them using 18th Century statistical methods?
Continuing to derive CERs via LLOLS imposes a number of unfortunate burdens on the analyst that require several special adjustments to counteract them. Among these are the following: (1) LLOLS CERs do not minimize the error of estimating cost; (2) they are almost always biased low; (3) when a bias “correction” is made to them, quality metrics such as standard error and R2 must be recalculated; and (4) the logarithmic space standard error that LLOLS reports is not related in any simple way to the CER’s actual standard error.
Furthermore, restricting one’s CER-derivation techniques to OLS and LLOLS involves one in a web of contradictions, among them: (1) Nonlinear CERs whose coefficients are derived by LLOLS must have fixed cost = zero, while linear CERs whose coefficients are derived by OLS are permitted to have nonzero fixed-cost terms; and (2) Nonlinear CERs derived by LLOLS must have standard errors expressible as a percentage of the estimate, while linear CERs derived by OLS must have standard errors expressed as plus/minus dollar values. Finally, in what may be the most significant issue that makes use of LLOLS impractical, there are only a very few nonlinear algebraic forms that can be treated using LLOLS, namely those that are amenable to the algebraic properties of logarithms.
The objective of this presentation is to encourage cost analysts to wean themselves off the 18th Century LLOLS technique and move on to 21st Century methods that have optimal estimating and statistical properties and a wider range of applicability.
Author:
Stephen A. Book
Dr. Stephen A. Book vacated the position of Chief Technical Officer of MCR, LLC in 2010 (after serving in that position for almost a decade) to concentrate on research, training, and subject-matter-expert customer support. In his former capacity, he was responsible for ensuring technical excellence of MCR products, services, and processes by encouraging process improvement, maintaining quality control, and training employees and customers in cost and schedule analysis and associated program-control disciplines. Earlier, at The Aerospace Corporation, he was a principal contributor to several Air Force cost studies of national significance, including the DSP/FEWS/BSTS/AWS/Brilliant Eyes Sensor Integration Study (1992) and the ALS/Spacelifter/EELV Launch Options Study (1993). He served on national panels as an independent reviewer of NASA programs, for example the 2005 Senior External Review Team on cost-estimating methods for the Exploration Systems Mission Directorate, the 1997-98 Cost Assessment and Validation Task Force on the International Space Station (Chabrow Committee), and the 1998-99 National Research Council Committee on Space Shuttle Upgrades. Dr. Book joined MCR in January 2001 after 21 years with Aerospace, where he held the title Distinguished Engineer during 1996-2000 and served as Director, Resource and Requirements Analysis Department, during 1989-1995. Dr. Book was co-editor of the ISPA/SCEA technical journal, The Journal of Cost Analysis and Parametrics. He received the 2010 SCEA Lifetime Achievement Award, the 2009 NASA Cost Contractor of the Year award, the 2005 ISPA Freiman Award for Lifetime Achievement, and the 1982 Aerospace Corporation Presidents Award for Analytic Achievement. Dr. Book earned his Ph.D. in mathematics, with concentration in probability and statistics, at the University of Oregon.