Posted by MAKING STATISTICAL ANALYSIS ACCESSIBLE: THE RAMS TOOL FOR PERFORMING REGRESSION AND RISK ANALYSIS
Risk Track
Downloadable Files:
Abstract:
The tools for evaluating the uncertainty around cost and schedule estimating relationships (CERs) using statistical techniques remain under utilized by the cost estimating profession today. Three primary estimating techniques, the standard error of the estimate (SEE), SME judgment, and various “industry guidelines” are frequently used to assess uncertainty. These techniques can introduce bias into the estimate and lead to uncertainty being misestimated by, in some cases, orders of magnitude. The false security of high-confidence level budgets developed using these techniques can have a dramatic effect on a program’s chance of success.
The authors believe that statistical techniques are underutilized by the industry due to the difficult and time consuming nature of the analysis needed to perform statistical techniques outside of standard Ordinary Least Squares (OLS) regression. Until these analytical methods can be simplified to the point that it is reasonable to expect that all analysts be able to perform them, they will not be widely adopted.
In an effort to make advanced statistical techniques more accessible to cost analysts of all skill levels in their every day work, the authors have developed the Risk & Regression Analysis Methodology Streamliner (RAMS), an Excel based tool that automates OLS and Generalized Least Squares (including MPE, MUPE/IRLS and MPE-ZPB techniques) regressions and the prediction intervals around their associated estimates. To allow the results to be used in risk analysis, RAMS converts the prediction intervals into uncertainty distributions in a way that interfaces seamlessly with most COTS risk tools. The paper will cover the motivations for the creation of RAMS, along with lessons learned from the tool’s development and a demonstration of the tool. The authors will also discuss areas of risk analysis they would like RAMS to tackle next.
Author(s):
Matt Lytton
Matthew Lytton CCE/A graduated from Virginia Tech with a B.S. in Industrial and Systems Engineering in 2005. He is employed by Booz Allen Hamilton as a Senior Consultant and has three years of cost estimating experience within the Intelligence Community. His cost experience includes model development at NGA, NRO, and DIA. Mr. Lytton’s professional experience also includes one year as a web developer for the Air Force Life Cycle Management Process website. He has experience developing in four languages including Visual Basic for Applications, Java, and .Net.
Eric Druker
Eric R. Druker CCE/A graduated from the College of William and Mary with a B.S. in Applied Mathematics in 2005 concentrating in both Operations Research and Probability & Statistics with a minor in Economics. He is employed by Booz Allen Hamilton as a Senior Consultant and currently serves on the St. Louis SCEA Chapter’s Board of Directors. Mr. Druker performs cost and risk analysis on several programs within the Intelligence and DoD communities and NASA. He was a recipient of the 2005 Northrop Grumman IT Sector’s President’s award and the 2008 TASC President’s award for his work on Independent Cost Evaluations (ICEs) during which he developed the risk process currently used by Northrop Grumman’s ICE teams. In addition to multiple SCEA conferences, Eric has also presented papers at the Naval Postgraduate School’s Acquisition Research Symposium, DoDCAS and the NASA PM Challenge. He has also performed decision tree analysis for Northrop Grumman Corporate law, built schedule and cost growth models for Hurricane Katrina Impact Studies and served as lead author of the Regression and Cost/Schedule Risk modules for the 2008 CostPROF update.
Gregory Hogan
Mr. Hogan graduated from Penn State University in 1990, receiving a BS degree in Industrial Engineering. After college he became a charter member of the Engineering and Scientist Development Program at the Naval Air Systems Command (NAVAIR). Throughout his career at NAVAIR he collected numerous awards & commendations during assignments throughout cost analysis, contracts, and engineering groups. In the evenings, he returned to school, earning an MS degree in Systems Engineering from Virginia Tech. He is a recognized instructor of cost estimating & analysis to audiences from the Department of Defense (DoD), Intelligence Community (IC), UK Ministry of Defense, National Aeronautics and Space Administration (NASA), and Industry.
He has recently contributed to research papers (Class A / Class C Spacecraft, Commercial Space Practices, New Challenges to Cost Estimating) presented at national conferences. In 1997, he published a master’s thesis (Evaluation of Military Base Closure Alternatives) that re-defined the cost-benefit process used to evaluate Base Realignment and Closure (BRAC) scenarios. The Commander of NAVAIR awarded him the prestigious Meritorious Civilian Service award for contributions and dedication to the BRAC cause. He is an author of benchmark cost processes used by NAVAIR that are the foundation for much of the Society of Cost Estimation and Analysis (SCEA) courseware.
Recently, Mr. Hogan joined the Economic and Business Analysis team at Booz Allen Hamilton (BAH). Through his involvement with SCEA, Space Systems Cost Analysis Group (SSCAG), Aerospace Industries Association (AIA), and the Institute of Electrical & Electronics Engineers (IEEE), he has moved Booz Allen Hamilton to the forefront of understanding and estimating emerging technologies