Posted by PERFORMING STATISTICAL ANALYSIS ON EARNED VALUE DATA
EVM/Schedules Track
Downloadable Files:
Abstract:
Some Earned Value Methods, particularly those described in the equations on the DAU Gold Card, suffer from the shortcomings that they are backwards looking (they do not make a prediction of the final CPI) and do not allow for inferential or descriptive statistics. This leads to the propensity for these estimates to tail-chase as the CPI changes over time, meaning that the EAC for an over running program will systematically lag in predicting the overrun, and the EAC for an under running program will systematically lag in predicting the under run. Additionally, the Estimates at Completion (EAC) they yield cannot be evaluated for bias or uncertainty, nor can statistical significance tests be applied to them. Lastly, without quantified uncertainty measures, there is no method by which to perform risk analysis on these estimates without relying on subjective methods.
The purpose of this paper is to present a method by which statistical analysis techniques can be applied to Earned Value data to better predict the final cost of in-progress programs. EACs developed using statistical methods make a prediction of the final CPI and thus do not tail-chase. Also, since they rely on historical data, they are testable, that is to say that they can be subjected to statistical significance tests, and are thus defensible. Estimates produced using statistical analysis techniques will be unbiased and the descriptive statistics developed as a byproduct of the method allow uncertainty to be quantified for risk analysis purposes.
For demonstration, the method will be applied to a set of representative data. The paper will continue with an example of the paradigm shift this type of analysis caused when it was implemented across a production facility. The conclusion will discuss the recommended process and types of data needed to implement this type of analysis.
Author(s):
Eric Druker
Eric R. Druker CCE/A graduated from the College of William and Mary with a B.S. in Applied Mathematics in 2005 concentrating in both Operations Research and Probability & Statistics with a minor in Economics. He is employed by Booz Allen Hamilton as a Senior Consultant and currently serves on the St. Louis SCEA Chapter’s Board of Directors. Mr. Druker performs cost and risk analysis on several programs within the Intelligence and DoD communities and NASA. He was a recipient of the 2005 Northrop Grumman IT Sector’s President’s award and the 2008 TASC President’s award for his work on Independent Cost Evaluations (ICEs) during which he developed the risk process currently used by Northrop Grumman’s ICE teams. In addition to multiple SCEA conferences, Eric has also presented papers at the Naval Postgraduate School’s Acquisition Research Symposium, DoDCAS and the NASA PM Challenge. He has also performed decision tree analysis for Northrop Grumman Corporate law, built schedule and cost growth models for Hurricane Katrina Impact Studies and served as lead author of the Regression and Cost/Schedule Risk modules for the 2008 CostPROF update.
Dan Demangos
Mr. Demangos has over 16 years of professional experience in Earned Value Management (EVM) setup and implementation, planning & scheduling, schedule/cost integration, and Project Management training development and delivery. Mr. Demangos’ clients include FBI where he manages the implementation of an ANSI-748 compliant EVMS; DHS where he led a team that supports budget, acquisition and EVM efforts for the Office of the CIO; and the US Marine Corps. where he led the effort to implement EVM throughout the Budget Department of the Marine Corps. Headquarters. Mr. Demangos has also lead a team to rewrite Booz Allen’s Management Control System Description (MCSD) and has developed a phased training program that is offered to those in the Firm who practice EVM. Mr. Demangos was also a key contributor to the industry recognized AACE Earned Value Professional (EVP) certification and serves on the Certification Board. Prior to coming to Booz Allen, Mr. Demangos led the Project Controls effort for a telecommunications provider, implemented schedule/cost integration efforts for a Project Management software vendor and helped to manage remediation sites for the Department of Energy.
Richard Coleman
Richard L. Coleman is a Naval Academy graduate, received an M. S. with Distinction in Operations Research from the U. S. Naval Postgraduate School and retired from active duty as a Captain, USN, in 1993. His service included tours as Commanding Officer of USS Dewey (DDG 45), and as Director, Naval Center for Cost Analysis. At Northrop Grumman, he has worked extensively in cost, CAIV, and risk for the Missile Defence Agency (MDA), Navy ARO, the intelligence community, NAVAIR, and the DD(X) Design Agent team. He has supported numerous ship programs including the DDG 1000 class, DDG 51 class, Deepwater, LHD 8 and LHA 6, LPD 17 class, Virginia class SSNs, CVN 77, and CVN 78. He is the Director of Independent Cost Estimation for Northrop Grumman Information Systems and conducts Independent Cost Evaluations on Northrop Grumman programs. He has more than 65 professional papers to his credit, including five ISPA/SCEA and SCEA Best Paper Awards and two ADoDCAS Outstanding Contributed Papers. He was a senior reviewer for CostPROF and CEBoK and lead author of the Risk Module. He has served as Regional and National Vice President of SCEA and is currently a Board Member. He received the SCEA Lifetime Achievement Award in 2008.