2017 Workshop Training Program Co-Chairs: Remmie Arnold, Derreck Ross
Cost Estimating Basics Training Track
Track Chairs: Omar Akbik, Danielle Mrla, Cynthia Prince
Cost Estimating Basics (CEB01)
Cost Estimating Basics Track (CEBoK Module 1)
Elisa Blagoev
Christina Snyder
The Cost Estimating Overview session introduces the basics of cost estimating and analysis and the reasons for doing cost estimates. Additionally, the motivation for cost estimating, why and how it is used, and an overview of the challenges and issues will be presented. Cost estimating products will be reviewed and a general overview of the process and methods used to develop cost estimates will be discussed, providing the background information and fundamental knowledge needed for the other CEB sessions.
Cost Estimating Techniques (CEB02)
Cost Estimating Basics Track (CEBoK Module 2)
Walter Bednarski
Jean Nicole Sullivan
This module introduces four essential cost estimating techniques that practitioners most often use to develop credible and realistic estimates. They are analogy, parametric, build-up, and extrapolation from actuals. We explain how to use these techniques, compare and contrast them to provide flexibility when estimating. Moreover, each technique has strengths and weaknesses, varying degrees of applicability at different times during a program’s life cycle and different levels of fidelity required for an estimate. Last, we discuss related topics such as schedule and operations & support estimating.
Data Collection (CEB03)
Cost Estimating Basics Track (CEBoK Module 4, PEH Chapter 2)
Danny Polidi
Omar Akbik
This session covers the Core Knowledge section of Module 04 Data Collection of CEBoK. All estimating techniques and cost estimating models require credible data before they can be used effectively. In this module we will discuss the various types of data, processes needed to collect and analyze the data used in parametric applications, as well as data types, sources, and adjustment techniques.
Inflation (CEB04)
Cost Estimating Basics Track (CEBoK Module 5)
Steven D. Smith
This session covers the Core Knowledge section of Module 05 Inflation and Index Numbers of CEBoK. Proper inflation analysis is essential to the success of any cost estimate or economic analysis. Calculating inflation correctly and understanding the fundamental concepts will enable you to produce cost estimates that are timely, accurate, and credible to support your program’s lifecycle needs. It will also empower you to communicate with key stakeholders on the need to adjust your financial estimates based on changes in the economy.
Data Analysis (CEB05)
Cost Estimating Basics Track (CEBoK Module 6)
Eric Plumer
Brian Octeau
This session discusses the first analytical steps to take after obtaining a set of cost data, with a particular emphasis on techniques for displaying and analyzing data graphically. It addresses univariate, bivariate, and multivariate data sets, with a focus on statistical analysis of univariate data sets and graphical analysis of bivariate data sets via scatter plots. It introduces measures of central tendency, measures of variability, and measures of uncertainty, such as confidence and prediction intervals.
Learning Curve Analysis (CEB06)
Cost Estimating Basics Track (CEBoK Module 7)
Denise Nelson
Sriram Krishnan
This is a training track presentation of the CEBoK Module 7 (Learning Curves). The presentation will cover the key ideas, analytical constructs and applications of the learning curve module. The target audience are those preparing to take the ICEAA certification exam. Beyond the theoretical information, we will present the study questions for Module 7 with steps required to solve the problems using only a calculator as is required on the certification exam.
Risk Analysis Basic (CEB07)
Cost Estimating Basics Track (CEBoK Module 9)
Christian Smart
This session will provide motivation for the need for risk analysis and introduce the basic types and uses of risk. It will focus on the practical execution of the general risk analysis process: (1) Develop a point estimate; (2) Identify the risk areas in the point estimate; (3) Determine uncertainty around the point estimate; (4) Apply correlation between uncertainty distributions; (5) Run the Monte Carlo simulation; (6) Assess the reasonableness of results; (7) Calculate, allocate, and phase risk dollars; and (8) Show the results.
Prob/Stat Basic (CEB08)
Cost Estimating Basics Track (CEBoK Module 10)
Christian Smart
Denise Nelson
This will be a review of statistical and probability concepts relevant to cost estimation and uncertainty. Topics will include statistics; measures of central tendency; measures of dispersion; and important probability distributions and their linkage to cost estimates. We also introduce the concept of a random variable; Monte Carlo simulation; the Inverse CDF method; and thoughts on the differences between the normal and lognormal distributions. Finally, we discuss hypothesis testing and its relationship to cost estimating.
Software Cost Estimating (CEB09)
Cost Estimating Basics Track (CEBoK Module 12)
William Barfield
This session will convey the principles of this critical area of specialized cost estimating. It addresses the basics of software development and some of the things to look out for in software estimating, and includes specific examples of applying software estimating techniques. The presentation will walk through examples of how to approach a software cost estimate using several estimating relationships and ideas, commercial models that can be used to construct estimates, and various methods of software development.
Economic Analysis Basic (CEB10)
Cost Estimating Basics Track (CEBoK Module 13)
Neil Chakrabarti
Blaze Smallwood
This session covers the Core Knowledge section of Module 13 Economic Analysis of CEBoK. It will be of particular interest to anyone studying for the ICEAA certification exam. The session provides a practitioner’s perspective for conducting an economic analysis (EA) by reviewing EA concepts, terminology, variables and measures-of-merit. By accounting for monetized costs, monetized benefits, opportunity costs and time-value-of-money (“discounting”), an EA enables one to calculate economic measures-of-merit.
Contract Pricing (CEB11)
Cost Estimating Basics Track (CEBoK Module 14)
Patrick McCarthy
Ramzi Shuhaibar
This session explores the basics of contract pricing and the roles of cost estimating in the process. In this session, we explore the various contract types, and the factors and considerations related to choosing a contract type. We also explore how fee is calculated, and how risk is shared between the government and the vendor. We will also look at the Uniform Contract Structure and its sections as well as the cost-price proposal preparation and evaluation efforts.
EVM Basic (CEB12)
Cost Estimating Basics Track (CEBoK Module 15)
Patrick Malone
Jeffrey Locke II
This session will provide an introduction to the basic concepts of earned value management (EVM), with a focus on implementation, governance, and practical application in support of a project or program. Specific topics will include basic EVM components and data elements, as well as standard earned value analysis techniques. We will use practice problems throughout the presentation to demonstrate and reinforce the basic principles of EVM.
Parametric Training Track
Track Chairs: Chris Hall, Angela Fields
Parametric Estimating and Cost Estimating Relationships (PAR01)
Parametric Training Track (CEBoK Module 3, PEH Chapters 1 & 3)
Kyle Thomas
James Williamson
This module serves as an introduction to parametric cost estimating and cost estimating relationships. Parametric estimating involves developing relationships between technical, programmatic, and cost characteristics in order to project future costs. Key concepts include cost drivers, parametric models, and the development of cost estimating relationships.
Regression Linear (PAR02)
Parametric Training Track (CEBoK Module 8, PEH Chapter B)
Herve Joumier
Jennifer Kirchhoffer
This course introduces the basic concepts of regression and provides a demonstration of a simple linear ordinary least squares model. This session focuses on the basics required to build and evaluate a simple linear model. Key concepts include correlation, minimizing error, homoscedasticity, statistical significance, and uncertainty. The better you understand this simplest (although by no means simple) case, the more you will be able to understand and use more complicated regression techniques.
Regression Nonlinear/Multivariate (PAR03)
Parametric Training Track (CEBoK Module 8)
Marc Greenberg
Herve Joumier
A continuation of Linear Regression Analysis, this training session highlights specific techniques used when conducting regression analysis. Coverage includes non-linear models (logarithmic, exponential and power models), multivariate regression, selecting the best model, and related and advanced regression analysis topics. Prerequisite: Trainees have attended (or have equivalent background to) Linear Regression Analysis.
Regression Multiplicative (PAR04)
Parametric Training Track
Marc Greenberg
Christian Smart
This tutorial focuses on the development and use of CERs that are nonlinear and have multiplicative errors. Three methods for developing CERs are presented: log-transformed ordinary least squares, minimum unbiased percentage error, and zero percent bias minimum percent error. The optimality of each method is discussed.
CER Risk and S-Curves (PAR05)
Parametric Training Track (CEBoK Module 9)
Diane Butler
This session focuses on how to assess three sources of risk for CERs derived from regression: input uncertainty, model uncertainty and external factors. The session introduces formulae of the standard error of the estimate (SEE), confidence interval (CI), and prediction interval (PI), then illustrates error bands and S-curves generated from these formulae. SEE, CI and PI are compared to include how and why SEE-based techniques (while commonly used) almost always underestimate risk and uncertainty.
Complex Models – Company Developed/Hardware (PAR06)
Parametric Training Track (PEH Chapter 4)
David Bloom
Greg Kiviat
This session covers Company Developed and Commercial Complex Cost Models, which are multiple input models typically developed for a specific platform or product, including a proven ten-step approach for developing a multi-CER cost prediction model from historic data. This session provides a short history of complex model parametric estimating along with an overview, examples and recommendations to develop proposal Basis of Estimates, Business Case Cost Estimates, Independent Cost Estimates (ICE) and Estimates at Completion (EACs).
Complex Models Software (PAR07)
Parametric Training Track (CEBoK Module 12, PEH Chapter 6)
Arlene Minkiewicz
This training session provides an overview of complex software estimating models and describes the recommended processes for using them. Because software spending in DoD and NASA is increasingly significant, it is critical for those involved in software acquisition to understand the factors that drive software development and maintenance costs. This training session discusses software estimating methodologies, emphasizing those models used in industry and government and highlights software improvement activities that are relevant to software estimating.
Advanced Cost Topics Training Track
Track Chairs: Ramzi Shuhaibar, Kathleen Hudgins, Derreck Ross
GAO Cost Assessment Guide (ACT01)
Advanced Cost Topics Track
Brian Bothwell
In 2009, the U.S. Government Accountability Office (GAO) published the GAO Cost Estimating and Assessment Guide (GAO-09-3SP) to establish a consistent methodology based on best practices for use across the federal government in developing, managing, and evaluating program cost estimates. This training will introduce the twelve-step process, describe the characteristics and contents of a high-quality estimate, illustrate application of the guide to GAO assessments, and briefly introduce other GAO acquisition guides.
Technical Baselines (ACT02)
Advanced Cost Topics Track
Chris Hall
Peter Braxton
Understanding the technical and programmatic issues of the program/product you are estimating is key to developing a good cost estimate. This session discusses what information you need to baseline the technical and programmatic aspects of the program, the questions that need to be asked to ensure a clear understanding and how these issues translate into a cost perspective.
Work Breakdown Structure (WBS) (ACT03)
Advanced Cost Topics Track
Eric Plumer
Charles D. Hunt
The Work Breakdown Structure (WBS) is a critical part of managing and planning any project. The WBS is the best tool for ensuring consistency and focus on the project. This presentation will discuss the key attributes of a good WBS in scoping and planning of projects. The presentation will provide the generally accepted concepts of the use and application of the WBS as well as why it is often misunderstood and not used as it should for maximum effectiveness.
Monte Carlo Simulations (ACT04)
Advanced Cost Topics Track
Christopher Jarvis
Monte Carlo simulations are the most commonly used tool for performing uncertainty quantification and risk analysis. This training will provide an overview of Monte Carlo Methods, their utility and applications. The bulk of the training will be an interactive demonstration and analysis of results. Additional topics including random number generation, sampling schemes and correlation will be discussed.
Advanced Probability and Statistics (ACT05)
Advanced Cost Topics Track
Neil Chakrabarti
Diane Butler
This session will provide the analyst with an overview of advanced probability and statistical concepts and methods that are applied in cost analysis. Session includes an overview of probability distributions, relationships between distributions, hypothesis testing, confidence intervals and sample size, cost growth factor and percentiles, and method of moments.
Manufacturing Cost Estimating (ACT06)
Advanced Cost Topics Track (CEBoK Module 11)
Cynthia Prince
The goal of the Manufacturing Cost Estimating module is to arm the student with a set of techniques used to address issues unique to estimating in the manufacturing environment. It will be our objective in this module to raise a few of the most common general issues, considerations and concerns the estimator must be aware of in a typical major manufacturing environment and to provide techniques for addressing them. Depending on time and interest of attendees, example problems can be worked as exam preparation.
Economic Analysis Advanced (ACT08)
Advanced Cost Topics Track (CEBoK Module 13)
Brian Flynn
The session revisits the foundations of economic analysis. Taking advantage of recent work by the Nobel laureate Kenneth Arrow and others, it examines alternative conceptual frameworks for discounting. Ethical issues quickly arise, such as putting a price on the future of our children. As a teaser, OMB’s model for discounting, while acceptable, has serious flaws. Leading economists prefer Her Majesty’s Treasury approach – although controversial in its own right!
Schedule Estimating and Analysis (ACT09)
Advanced Cost Topics Track
Justin Hornback
“Do not squander time for that is the stuff life is made of.” Benjamin Franklin. This workshop will cover scheduling basics, desired attributes of a valid project schedules, schedule estimating, and a schedule analysis framework important to schedule and cost estimator analysts.
Schedule Risk Analysis (ACT10)
Advanced Cost Topics Track
David Hulett
Project scheduling is only the start of understanding when your project will finish. This workshop shows how simulating uncertain durations allow the user to determine how much overrun there might be at different target levels of certainty. We introduce the Risk Driver method of driving the Monte Carlo simulations using the risks usually found starting from the Risk Register. We then prioritize the risks for effective risk mitigation.
Joint Cost and Schedule Risk (ACT11)
Advanced Cost Topics Track
David Hulett
Recognizing the need to ensure results from cost and schedule risk analyses are compatible, agencies, such as NASA, have recently implemented policy requiring the integration of these two, formerly separate, practices. This training course will provide analysts an overview on how they can integrate their cost estimates, schedules, and risk register into a single cohesive model that can then be analyzed to predict future cost and schedule growth and identify their sources so that mitigative actions can be taken.
Better Buying Power and Target Costing (ACT12)
Advanced Cost Topics Track
John Deem
Tom Dauber
This session introduces cost management and better buying power on an integrated product team supporting trade studies that drive designs to a target cost. Discussed will be the application of design-to-cost, reduction-of-total-ownership-cost and activity based costing techniques, tools and techniques, and the related discipline of Cost As an Independent Variable (CAIV). Cost management takes a holistic approach, incorporating cost considerations into the overall management approach, and together with other measures of performance. Content is derived from CEBoK© Module 16, Cost Management.
COSMIC Software Estimating (ACT13)
Advanced Cost Topics Track
Alain Abran
This training session will present the key steps to develop estimation models based on COSMIC Function Points, including: An overview of the COSMIC function points for measuring software size, including embedded and real-time software; An overview of the estimation process, and where Function Points are contributing; Economic interpretation of estimation models with Function Points; How to build COSMIC-based estimation models and the insights gained; How to use COSMIC-based estimation models at estimation time. The tutorial will include references to COSMIC estimation models developed in industry.
Software CEBoK Training Track
Solution Based Estimation (SWT01)
Software CEBoK Topics Track
Presenter: Eric van der Vliet
Author: Eric van der Vliet
Estimation methods and metrics often assume a certain level of homogeneity across the system for which the costs are estimated, which is mostly not the situation for more complex solutions. For example, if size based estimation is applied, a productivity is determined based on the technology, size and duration, although the productivity for different elements in a heterogeneous solution can vary. Homogeneity is required to be able to extrapolate and re-use metrics like productivity from previous, comparable solutions. This presentation explains the concepts of Solution Based Estimation that aligns estimation with the solution architecture that is defined. The concept will be presented based on a practical example of a heterogeneous package implementation and shows the benefits for estimators, size experts, metrics experts, cost experts but also architects and project managers.
Estimation Methods (SWT02)
Software CEBoK Topics Track
Presenter: Wilson Rosa
Authors: Wilson Rosa; Jo Ann Lane, SDSU
Intro to Sizing Physical & Functional Methods (SWT03)
Software CEBoK Topics Track
Presenter: Alain Abran
Authors: Dan Galorath, Galorath, Inc.; Frank Vogelezang, Ordina; Tom Cagley, IFPUG; Eric van der Vliet
The size of a applications is an important input for parametric estimation. The size can be provided as Physical Size, for example Lines or Code or as Functional Size. This module will provide an introduction in sizing with an overview of different sizing methods.
Estimation Maturity Model (SWT04)
Software CEBoK Topics Track
Presenter: Brian Glauser, Galorath, Inc.
Authors: Esteban Sanchez, Galorath, Inc.
In today’s environment of massive software cost overruns, the moist finger in the air and trusting of expert judgement are no longer acceptable and companies are slowly awakening to the adoption of formal cost estimation practices; Estimation is thus becoming a key process within the organization. This workshop introduces the concept of Estimation Maturity and presents an example of an Estimation Maturity Model, along with guidelines for implementation and business motivation.
Basis of Estimate (SWT05)
Software CEBoK Topics Track
Presenter: Eric van der Vliet
Authors: Ton Dekkers, Nesma
This module provides an introduction to the Basis of Estimate (73R-13) as applied for the Software Services Industries. The Basis of Estimate (BoE) provides a structure to document the estimate in a transparent and consistent way. With that it’s also a reference for Verification & Validation of an estimate in case of a bidding process and in case of challenges a reference for audits.
Metrics Collection (SWT06)
Software CEBoK Topics Track
Presenter: Arlene F Minkiewicz
Author: Arlene F Minkiewicz
Benchmarking / Cost Drivers (SWT07)
Software CEBoK Topics Track
Presenter: Alain Abran
Author: Harold van Heeringen, METRI
In this module, the main focus is on benchmarking a cost (or effort) estimate. How does an estimate relate to possible peer offers from the industry? This is usually relevant for customer organizations outsourcing their AD projects and for outsourcing supplier organizations wanting to win the bid for these contracts. Goals will be to: to understand the concept of benchmarking in the context of software cost estimation, to learn about the ways historical data can be used to assess the proposals of supplier bids, to understand the importance of estimating software projects in the most accurate ways, as both optimistic and pessimistic estimates will actually result in extra cost!