Posted by Determining the Cost of the Certification and Accreditation Process using Expert Opinion and Monte Carlo Simulation
Estimating Track
Downloadable Files:
Abstract:
The Certification and Accreditation (C&A) paper will address an empirical approach to determining the total cost and schedule of completing the C&A process for a given network or system. The requirement for and policy guidance on how to conduct C&A is covered by Director of Central Intelligence Directive 6/3 (DCID 6/3) along with other DoD and Federal guidance. DCID 6/3 requires agencies to complete a rigorous process of reviews to ensure that the networks and systems are secure. The customer felt that C&A was consuming extraordinary amounts of time and effort but wanted to document and understand the real impact before pursuing changes to the process.
The customer had a published schedule for the C&A process that estimated the duration of each step within the C&A process. There is a sense within the organization responsible for obtaining the C&A that the process is flawed, taking longer than defined and consuming too many resources. Since there was no data to validate their concerns, the cost estimating team was asked to develop a model that estimates the cost and schedule of a given C&A effort. The model would aid in planning and budgeting and help with justification to identify and correct problems with the process if necessary.
Although a database for managing the C&A process does exist at the customer, it did not provide sufficient detail to determine the full schedule and cost of engaging in the C&A process. Thus, data was generated using C&A Subject Matter Experts (SMEs) for initial development of the model with a plan to perform validation once data could be collected.
The team conducted interviews with the SMEs using elements from the Delphi technique, where interviews were conducted individually and then the results of all the interviewees’ inputs were distributed anonymously to each interviewee to review and adjust. Each interviewee was asked to provide schedule and effort data for each step of the C&A process for the notionally hardest system and the notionally easiest system in order to bound the data. The effort data was measured in hours and converted to dollars based on the labor categories of those expending the effort to complete each step of the C&A process. The schedule data was measured in work days to determine the amount of time needed to complete each step of the C&A process. The inputs from the interviewees were then put into a model using a Monte Carlo simulation of SMEs inputs to determine the cost of the process.
During the SME interview process, a Program Manager (PM) was able to provide the team with primary data on the C&A process. This data was normalized, put into the model, and used to cross check the data obtained from the SMEs.
The focus of this paper will be the approach for gathering data, modeling the cost of this process, and the analysis of both the PM Actual data and Simulated SME data.
Author(s):
Amanda Flynn
Northrop Grumman, TASC
Amanda J. Flynn is an Operations Research Analyst in the Advisory Services Division of TASC, Inc. Ms. Flynn has worked as a risk analyst and currently provides cost estimation and budgeting analysis support to the Intelligence Community. In addition, Ms. Flynn has provided cost and risk analysis support for a DDG51 class ship Independent Cost Estimate (ICE) for Northrop Grumman’s Ship Building Sector. In 2009, Ms. Flynn became a Certified Associate in Project Management. Ms. Flynn graduated from Carnegie Mellon University in 2008 with a B.S degree as a double major in Decision Sciences and Hispanic Studies. She received both College and University Honors.
Belinda Nethery
Northrop Grumman, TASC
Belinda J. Nethery is a Principal at TASC with over twenty‐five years experience performing cost estimation and analysis. Ms. Nethery spent ten years in the Air Force where her focus was on software cost and C4I systems. She has been with TASC for over fifteen years and has spent the majority of that time working all facets of Automated Information Systems. Her current focus is on information technology services. Ms. Nethery holds a B.B.A in Accountancy from the University of Notre Dame and a M.S. in Cost Estimation and Analysis from the Air Force Institute of Technology.
Kyle Thomas
Northrop Grumman, TASC
Kyle D. Thomas is an Operations Research Analyst in the Advisory Services Division of TASC, Inc. Mr. Thomas currently provides cost estimation and budgeting analysis support to the Intelligence Community. Prior to joining TASC, Mr. Thomas received a B.S. in Energy Business and Finance, and minor in Economics from The Pennsylvania State University.
Amanda Gerstner
Northrop Grumman, TASC
Amanda E. Gerstner is an Operations Research Analyst at Northrop Grumman Information Systems – Advisory Services Division (TASC). Ms. Gerstner currently provides cost and budgeting analysis support to the Intelligence Community. Ms. Gerstner also supports Independent Cost Evaluation (ICE) teams providing internal risk assessments for Northrop Grumman. Ms. Gerstner received her B.S. in Industrial Engineering from the Pennsylvania State University in 2007.
Brett Dickey
SCITOR
Brett Dickey received his B.S. and M.S in Systems Engineering from the University of Virginia in 2006 and 2008, respectively. After graduating, Mr. Dickey worked as a cost analyst at TASC, providing Independent Life‐Cycle Cost Estimates (I‐LCCEs), TASC‐internal cost estimates, and proposal support. Mr. Dickey currently works at Scitor Corporation where he provides support to the Office of the Director of National Intelligence (ODNI) Systems and Resources Analysis (SRA) Cost Analysis (CA) group, primarily developing Independent Cost Estimates (ICEs) for the Intelligence Community (IC) and conducting research to improve upon current costing methods and tools.
Christina Kanick
Christina M. Kanick is an Operations Research Analysis Analyst in the Advisory Services Division of TASC, Inc. Past experience includes conducting Independent Cost Evaluations (ICEs) for both Northrop Grumman Information Systems’ (IS) Sector and Corporate Level, as required by IS, Shipbuilding, and Technical Services. In addition to her ICE work, Ms. Kanick provides cost, risk, and budgeting analysis support to the Intelligence Community. Prior to joining Northrop Grumman in 2007, Ms. Kanick obtained a B.S. in Industrial Engineering from the University of Pittsburgh. Currently, she is pursuing a M.A. in Applied Economics from Johns Hopkins University.
Peter Braxton
TASC, Inc.
Peter J. Braxton holds an AB in Mathematics from Princeton University and an M.S. in Applied Science (Operations Research) from the College of William and Mary. He currently serves as a senior cost and risk analyst for TASC, Inc. He has worked to advance the state of knowledge of cost estimating and risk analysis, Cost As an Independent Variable (CAIV), and Target Costing on behalf of the Navy Acquisition Reform Office (ARO), the DD(X) development program, and other ship and intelligence community programs. He has reviewed several high-profile proposals as part of the Independent Cost Estimation (ICE) function for Northrop Grumman. He has over a dozen professional papers to his credit, including SCEA/ISPA International Conference award-winners in CAIV (1999) and Management (2005). He served as managing editor for both the original development of the acclaimed Cost Programmed Review Of Fundamentals (CostPROF) body of knowledge and training course materials and its successor, the Cost Estimating Body of Knowledge (CEBoK). He serves as SCEA’s Training Chair, was elected to the Board of Directors for the 2009-2011 term, and was named SCEA’s 2007 Estimator of the Year for contributions in Education. He serves as a TASC Technical Fellow, and received both a TASC President’s Award and Northrop Grumman Corporate Contracts, Pricing, and Supply Chain Award in 2008.