The table below can be sorted by track or paper title, and is searchable. Click the title or name to jump to a description.
2015 Papers Program Co-Chairs
Nathan Honsowetz
Christina Snyder
Government Processes Track
Track Chairs:
Michael Higdon
Chad Larson
Kammy Mann
Health CERs – Using an interdisciplinary approach to estimating the cost of caring for Canada’s veterans
Erin Barkel
Carleigh Malanik
Government Processes Track
Reports from the Veterans and the National Defence and Canadian Forces Ombudsmen have brought attention to the prevalence of mental illness among veterans. However, there are no published estimates for the cost of treating this growing population. A key obstacle to this research is that the relevant data is dispersed across several government organizations. We fill this research gap by leveraging actuarial and epidemiological studies of the veteran population to create new cost estimating relationships.
NASA Commercial Crew Cost Estimating – A Look at Estimating Processes, Challenges and Lessons Learned
Lance Cole
Rick Battle
Government Processes Track
To support annual PPBE budgets and NASA HQ requests for cost information for commercial crew transportation to the International Space Station (ISS), the NASA ISS ACES team developed system development and per flight cost estimates for the potential providers for each annual PPBE submit from 2009-2014. This paper describes the cost estimating processes used, challenges and lessons learned to develop estimates for this key NASA project that diverted from the traditional procurement approach and used a new way of doing business.
Federal Shared Services: The Future for Administrative Shared Services and Challenges Facing Shared Service Providers (SSPs) and Their Customers
Ruth Dorr
Marilyn Fleming
Sarah Pope
Government Processes Track
This paper will discuss the progress being made in the “Shared-First” approach for sharing of administrative services. Federal Agencies are being asked to execute their missions to ever increasing standards in a resource constrained environment. Administrative shared services have been identified as a way to reduce redundancy and improve efficiency in Federal business solutions. The identification of shared services as a Presidential cross-agency priority along with the designation of several providers for administrative shared services are recent developments in the movement toward shared services within the Federal government.
Lessons learned from applying cost capability curves on an Air Force AoA
Robert Georgi
Government Processes Track
New direction from Air Force leadership established a requirement for the presentation of life cycle cost versus capability tradeoff analysis for all AoA final reports. This presentation examines the approach that was used for a recent AoA that was a pilot program for the Cost Capability Curve process. When appropriately implemented cost capability analysis can provide significant advantages. There are lessons learned from attempting to implement this approach. We will also discuss examples where this approach does not apply.
2015 Best Paper: Government Processes Track
The Navy Modernization Program:
Estimating the Cost of Upgrading AEGIS Guided Missile Cruisers
Jeremy Goucher
Krystian Jenkins
Government Processes Track
The Navy is well into its 20 year, $16B (CY10$) plan to modernize 84 AEGIS warships. This cost estimate covers eleven of the planned ships and accounts for maintenance and upgrades to the ship and the combat systems. The estimate leverages recent contract and shipyard performance data and interviews with engineers, resulting in a detailed study and recommended cost savings initiatives. The methods and data in this estimate will assist ship modernization cost efforts across the fleet for the foreseeable future.
Analysis of Alternatives from a Cost Estimating Perspective
Kirby Hom
Kenny Ragland
Government Processes Track
Subsequent to the Weapon Systems Acquisition Reform Act of 2009, there have been other significant DoD policy changes that require Service sponsors to conduct an Analysis of Alternatives (AoA) prior to Milestone A. “About three-quarters of a program’s total life cycle cost is influenced by decisions made before it is approved to start development“ [GAO-09-665]. From a cost estimating perspective, a successful AoA rests with the study approach, overall costing methodologies, key insights from trades analyses, and lessons learned.
Mining for Cost Estimating Relations from Limited Complex Data
Mark Jacobs
Shawn Hayes
Government Processes Track
NASA’s robotic Earth and space science missions cover a diverse group of projects addressing complex science objectives that include challenging implementation approaches. Progress applying Principle Component Analysis techniques covering project management, systems engineering, mission assurance, integration & test, and spacecraft subsystems is described. Supporting data analysis efforts include a large detailed set of technical and programmatic input candidates that are analyzed to identify the primary spacecraft cost drivers.
Manufacturing Assembly Plan (MAP) Tool: Bridging the Gap between Performance and the Construction Processes
Richard Lee
Haitham Ghannam
Edward Walsh
Government Processes Track
This presentation explains the four module build plan for the VIRGINIA Class Submarine (VCS) program and provides an in-depth look into labor hours by Section 1/2A, Section 2B/5, Section 6/7, Section 8/9, Final Assembly and Test (FAT), and Post Shakedown Availability (PSA). Further understanding these concepts will not only allow cost estimators and program managers to better understand the current construction build strategy, but it will also aid in better aligning cost estimates with reality at the deck plate.
It Ain’t Easy Being Green… Sustainable Manufacturing with an eye on Cost Avoidance and Stewardship
Kammy Mann
Jessica Boatwright
Government Processes Track
The DoD, one of the largest energy consumers in the world, has committed to sustainability. An important element of sustainable weapon systems acquisition currently not under the purview of the DoD’s strategic plan is sustainable manufacturing. This paper will leverage current research and industry best practices to identify the impact of sustainable manufacturing on cost avoidances relating to human health and environmental benefits. Recommendations will also be provided for complying with the FAR sustainability requirements.
Business Case Analysis Wizard
Lauren Nolte
Cassie Capots
Marcie Pfeuffer
Government Processes Track
This paper describes the purpose and design of a “Business Case Analysis (BCA) Wizard” that was created for the Air Mobility Command (AMC) Enterprise Learning Office (ELO). As the value of BCAs within the learning transformation at AMC has grown, ELO identified the need to develop a tool that allows “non-cost analysts” to develop high-quality BCAs. This “BCA Wizard” guides users through a standard process of conducting a BCA via a user-friendly interface.
Masters in Cost Estimating and Analysis
Dan Nussbaum
Greg Mislick
Government Processes Track
This presentation provides an update on the all-Distance Learning Masters Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS).
Methods Used in Pricing and Conveying NASA Assets for Use by Commercial Programs
James Roberts
Torrance Lambing
Government Processes Track
Different agreement types including EUL’s, Use Permits, Land Leases and others are used to convey NASA assets no longer needed by programs such as Space Shuttle and construction of the Space Station. By law, some situations prohibit NASA from pricing facilities in competition with the local market. Costs are estimated either via market-based or actual cost methods, according to circumstances. This paper will explain and clarify how these costs are estimated and pricing is established.
ATC Zero: Estimating the True Impact of the Chicago Control Center Fire
Kellie Scarbrough
Government Processes Track
For seventeen days, the Chicago Air Route Traffic Control Center went offline after a contractor set fire to the intricate communications network that controls some of the busiest airspace in the country. Workers scrambled around the clock for seventeen days to restore functionality to the center. This paper seeks to determine the true economic impact of the event, including the costs of time, operations, and environmental impacts.
Where Does Your Cost Estimate Go?
Emily Stafford
Christopher Metz
Government Processes Track
You have spent several months developing your cost estimate and presenting it to the customer. Now what? This paper will track what happens to your cost estimate after it has been delivered to the customer. It will follow the estimate through the budget formulation process, ultimately becoming a part of the President’s Budget. It will also discuss the cost estimator’s role during the budget process, and the influence of the budget process on future estimates.
Facilitating Predictive Cost Analytics via Modelling V&V
John Swaren
Government Processes Track
We are often asked to “fine tune” estimating software to unique situations. That “calibration” effort is key to Verification & Validation of parametric models. In this presentation, we show an example of using Predictive Analytics to develop a method to predict a Software-estimating model’s inputs as a function of Key Performance Parameters (KPPs). We demonstrate how a calibrated knowledgebase can best be used to V&V an estimating approach reflective of past and future requirement metrics.
Hyperbolic Discounting and Defense Acquisitions
Eric Lofgren
Government Processes Track
Cost estimation has largely been approached as an engineering problem which can be solved using data and techniques. While these are undoubtedly invaluable considerations, the broader production process is an inherently open-ended system subject to human interactions which are ill-defined and non-deterministic. This paper explores how behavior and institutions affect cost outcomes through the avenue of hyperbolic discounting, a time-inconsistent model of personal valuations. It will be argued that evidence for hyperbolic discounting in both government agents and contractor management have negative effects on cost estimates and will proffer some solutions based on commitment schemes.
Methods & Models Track
Track Chairs:
Ann Hawpe
Leone Young
Cost Estimating Challenges in Additive Manufacturing
Joe Bauer
Patrick Malone
Methods & Models Track
Additive manufacturing possibilities are virtually endless: from rapid prototyping, to printing food, to recreating human tissue. This presentation highlights the challenges of estimating 3D printing processes and offers some solutions to the estimator. Challenges include: amortizing non-recurring tooling across multiple programs, capturing software-centric nature of 3D printing, printing speed, and printing media limitations. This presentation highlights the use of existing cost estimating tools and areas for future development.
Long Term Impact of Ship Concepts on Operating and Support Cost Affordability
Eric Buller
Stuart Merrill
Bryan Miller
Methods & Models Track
Achieving affordability can be challenging especially once a program enters full rate production. Developing accurate Operating and Support (O&S) cost estimates that reveal tradeoff impacts and sustainment risks provides decision makers with affordability analysis during key acquisition events. We intend to demonstrate through analysis the impact of legacy and new ship concepts on O&S costs and program affordability. We will examine the cost tradeoffs for each ship concept in each O&S cost element.
The Peculiar World of Sole Source Contracting; Maybe There is a Better Way
Sandy Burney
Shawn Larson
Methods & Models Track
In 2014, 21% of U.S. Government contracts were sole source, accounting for 31% of total contracted value. They typically require negotiation and certification of all cost estimates, which can take up to 18 months. This light-hearted presentation will explore the peculiar world of developing and negotiating sole source estimates, including differences from budgetary and competitive estimating, as well as offer suggestions for achieving fair and reasonable sole source prices for less time and money.
Migration of Microsoft Excel Tools to Next Generation Platforms: “Can You Hear the Footsteps?”
Jeremy Eden
Methods & Models Track
Analysis tools are popular solutions for extended deployments and at the same time an acceleration of platform migrations is occurring to keep up with rapidly changing technology and end user demands. This paper provides specific examples of platform migration challenges, resulting failures, and how they can be best addressed. It will also provide specific best practices for ICEAA member take-away that can be followed to help ensure smoother transitions across platforms.
2015 Best Paper: Methods & Models Track
Generalized Degrees of Freedom (GDF)
Shu-Ping Hu
Methods & Models Track
Minimum-Percentage Error/Zero-Percentage Bias (ZMPE) method is commonly used for multiplicative-error models. But ZMPE users do not adjust degrees of freedom (DF) for constraints included in the regression process. This generates misleading ZMPE CER fit statistics and underestimates the CER error distribution variance. Hence, ZMPE results are incomparable with those derived without constraints. This paper details why DF should be adjusted and proposes a Generalized Degrees of Freedom measure to compute fit statistics for constraint-driven CERs.
Bottom Up Methods of Estimating Depot Level Repairables
Tim Lawless
James Black
Dave Goldberg
Qing Qing Wu
Methods & Models Track
Depot Level Repairables (DLR) costs comprise a significant share of total Operations and Support costs as they capture the scope of repairing and/or replacing failed hardware items. This paper will provide detailed descriptions of bottom up methods to estimate DLR costs using programmatic technical baseline documentation and acquisition costs as well as supply system requisitions and repair price data sources. Emphasis will be placed on producing data-driven analysis to inform cost estimates and estimate customers.
Seven Degrees of Separation: The Importance of High-Quality Contractor Data in Cost Estimating
Crickett Petty
Methods & Models Track
The popular notion that any two people are linked to one another on average via a chain with “six degrees of separation” is based on a relatively small sample. Using massive data sets, researchers have since discovered that the average degrees of separation is closer to seven. This highlights the need for complete and thorough data in any analysis, and cost estimating in particular. Despite its importance the collection of high-quality contractor cost data is challenging. Processes for meeting these challenges are presented.
Interviewing Subject Matter Experts
Melissa Teicher
Methods & Models Track
As estimators, we often have to obtain technical information from Subject Matter Experts (SMEs) in order to provide thorough and accurate analysis. But sometimes, getting the information you need is easier said than done. This paper discusses the steps for interviewing SMEs in order to make the most of their time and your time. It explores the benefits of building a cooperative relationship of mutual respect and the “dos” and “don’ts” of SME consultations.
The Impacts of Design Change on Reliability, Maintainability, and Life Cycle Cost
Andreas Viberg
Oskar Tengo
Methods & Models Track
When making decisions about a design change to a technical system, it is essential to understand how it affects reliability and maintainability, as that directly impacts both overall performance and cost of operations and maintenance throughout the life cycle. LCC analysis based on modeling and simulation of relevant scenarios is an effective way to gain that understanding. This paper presents an approach for such analysis and illustrates its successful application in a recent project.
A Prime Contractor’s View Using JCL Tools and Techniques
Chip Woods
Methods & Models Track
For several years now, Lockheed Martin has been integrating cost and schedule data into a single model to perform integrated analysis. This paper will discuss Lockheed Martin’s JCL process, the tools used to develop JCL curves, and how this analysis can be used to shape and steer the program – including modeling funding constraints.
Class List of Characteristics
Paul Breon
Methods & Models Track
CLOC (Class List of Characteristics) is a coding system wherein each digit of the code indicates one of the cost drivers for the product/activity. CLOC gives the ability to “Call a thing by what it really is” and creating a universal language that can be utilized for any purpose. CLOC does NOT include Bias like quantity, cost, etc. These things are used in conjunction with the code to do any kind of analysis needed.
Achieving Affordability through high fidelity ROM cost estimates
Eric Buller
Methods & Models Track
Achieving affordability can be challenging especially if early decisions are made without proper cost estimates. Developing high fidelity ROM cost estimates and identifying capability tradeoffs during the requirements generation phase can be critical to the success of achieving long term affordability. We intend to demonstrate through analysis that preforming high fidelity ROM cost estimates with capability tradeoffs allow decisions makers to informed choices on mission requirements that achieve affordability for future programs.
Parametrics Track
Track Chairs:
Eric Gabrielson
William Lueker
A Regression Method for Sparsely-Populated Datasets
Daniel Barkmeyer
Parametrics Track
A method is presented for including sparsely-populated independent variables in a regression analysis without introducing subjectivity or statistical bias. The method is shown to reproduce the result of a traditional regression method for a fully populated dataset, and to converges to the same result on a sparsely-populated dataset as that dataset decreases in sparseness. Application of the method to develop a new, empirically-derived model for spacecraft testing schedule duration will also be presented.
2015 Best Paper: Parametrics Track
Improvement Curves:
An Early Production Methodology
Brent Johnstone
Parametrics Track
Learning slope selection is a critical parameter in manufacturing labor estimates. Incorrect ex ante predictions lead to over- or understatements of projected hours. Existing literature provides little guidance on ex ante selection, particularly when some actual cost data exists but the program is well short of maturity. A methodology is offered using engineered labor standards and legacy performance to establish a basic learning curve slope to which early performance asymptotically recovers over time.
The NRO CER Analysis Tool
Donald MacKenzie
Parametrics Track
The NRO Cost and Acquisition Assessment Group (CAAG) uses its CER Analysis Tool, CERAT, to inform CER development. CERAT analyzes CERs for sensitivity to influential data points (IDPs). The impacts on estimated costs for a target data point when data points are removed are the primary outputs. The changes in recorded CER constants when data points are removed also help in assessing overall CER “stability.” CERAT also provides several other aids to CER developers.
A Parameter Driven World
Jeffrey Platten
Parametrics Track
Parametric estimating is gaining acceptance. DFARS requires DOD contractors to maintain an estimating system. Parameters are the characteristics, features, or measurable factors that define a product or program. What makes a good parameter? What are the pros and cons of parametric cost estimating? How do you develop and maintain a parametric estimating database/model? What does the future hold, beyond just estimating? How do you gain parametric control over touch labor, support labor, and parts costs to improve affordability?
Generating a Semi-Automated “Similar To” Basis Of Estimate from a Complex Parametric Hardware Cost Model for Antennas
Danny Polidi
David Bloom
Parametrics Track
This paper discusses the development of a “Similar-To” Basis Of Estimate (BOE) generation tool used in conjunction with a Complex Parametric Antenna Cost Model. Starting with the parametric model based on quantifiable sizing mechanisms which are designed to quickly and accurately calculate the “top-down” cost for all engineering and operations disciplines and functions required for antenna development and test, cost is generated, “similar to” programs are identified, and a BOE can be auto-populated.
Macro-parametrics and the applications of multi-colinearity and Bayesian to enhance early cost modelling
Dale Shermon
Parametrics Track
This paper will consider the spectrum of parametric cost models from cost estimating relationships (CER) to micro and finally macro-parametric models. This will lead to a description of the Family of Advanced Cost Estimating Tools (FACET) parametric suite of models and their top-down capability to estimate costs incorporating the application of multi-colinearity and bayesian techniques.
Using Historical Cost Reports to Derive System Specific Factors
Jocelyn Tague
William Banks
Parametrics Track
Non‐Recurring Engineering (NRE) is often a cost driver for new, complex systems and must be given high attention during Milestone B cost estimating. Our team developed an approach for developing system specific Cost Estimating Relationship (CER) using actual costs for from combat weapon systems with shared capabilities, resulting in a credible defensible, yet flexible approach that can be replicated for a wide range of estimates.
A CER Development Process for Spares Estimating
Cheryl Wilson
Parametrics Track
This presentation shows the direction of a group of Spare Parts Estimators and how we are advancing our statistical skills and CER development processes. It shares the progress in our CER development journey attained through education, research, analysis and teamwork.
Improving Parametric Software Cost Estimation Through Better Data Collection
Bob Koury
Parametrics Track
In 2010 PRICE Systems LLC in conjunction with PEO STRI agreed to work together to improve the data being collected to support software development cost estimation in the command. The paper will compare and contrast the experiences of two other collection efforts with those of original effort. The paper will identify core findings which may be used universally to improve the quality, quantity, and applicability of data being collected.
Building an Integrated Aerospace Electronics Development Parametric Model from the Ground Up
David Bloom
Parametrics Track
This paper presents the results of a multiple year effort to create a business specific set of integrated parametric cost models for complex electronics hardware development in such a way that all disciplines and activities that participate in that development process have a piece of the model and a stake in the data collection that drives the model. This “grounds up” approach provides that added benefit of cost allocation guidance once the development is approved.
Program Management Track
Track Chairs:
William Barfield
Rodney Olin
Essential Views of the Integrated Program Management Reports (IPMRs)
Thomas Coonce
Glen Alleman
Program Management Track
During this session, the authors will describe key views of the IPMRs and other data that will allow a government stakeholder to a) understand the cost, schedule and technical status on applicable contracts, b) investigate the nexus of performance problems, and c) observe likely future problems. The presentation represents a “synthesis” of existing traditional earned value management metrics and other metrics designed to provide an integrative picture to assist in keeping the program “green”.
Meeting Fiscal Constraints: the Evolving Role of Performance Measurement
Steve Green
Kevin Davis
Kurt Heppard
Program Management Track
Performance measurement tools can reflect the strategic intent of management and help them prioritize initiatives to meet strategic objectives. In fiscally austere times, the Balanced Scorecard can help organizations address these tough decisions. Applying concepts from the Balanced Scorecard to cost analysis and budgeting, this paper develops an integrated aligned system of strategic goals, performance metrics, and cost parameters at a university facing pressure to manage costs. There may be lessons learned for the DoD.
The graph is always greener on the other side: examples of how to interpret and fix bad visual representations of data
Stephen Ketcham
Program Management Track
In my role performing independent reviews of estimates, I’ve seen a quality spectrum of different types of graphs, charts, and forecasts. These visuals summarize key points and remain in decision-makers minds, often providing the convincing point to win funding approval for the project. Unfortunately they can also be poorly crafted, employ scare tactics or even frame the data in a misleading way. This presentation provides bad examples of visuals, and discusses possible improvements.
Seeing What We Need to See vs. What We Hope to See: Visualizing Integrated Cost and Schedule Data for Earned Value Analysis
Brian Leach
Program Management Track
When applied properly, data visualization can serve analysis by showing us what is otherwise difficult to see (and not always what we hope to see). With the emergence of the UN/CEFACT XML standard, integrated cost and schedule data can be shown in exciting new ways. This paper will examine and explore innovative new ways to show integrated cost and schedule data which have practical applications in earned value management system analysis.
Different Incentives in Government Contracting
Jennifer Leotta
Program Management Track
This paper shows stock prices are used as the primary incentive for executives of publicly traded government contractors rather than profit sharing contracts. Looking at the reaction of companies’ stock prices to public and private sector news, this paper shows stock prices do not capture “bad news” for both the public and private sector customers compared to overall market indicators; and would make poor contract incentives. Finally, alternate methods of contract incentives will be explored.
Agile Project Management Controls: Issues with EVM and a Viable Alternative
Omar Mahmoud
Blaze Smallwood
Program Management Track
As more government software programs adopt agile practices, program managers are trying to find effective ways to monitor and control cost and schedule. The challenge is that traditional project control techniques, like EVM, are not well suited to deal with the requirements fluidity and new terms/metrics that come with agile projects. This paper will describe the inherent issues with using EVM on agile projects and offer a recommendation for a more effective solution.
2015 Best Paper: Program Management Track
2015 Best Paper Overall
The Psychology of Cost Estimating
Andy Prince
Program Management Track
Research into human psychology has yielded amazing findings into how we process information and how we use information to make decisions. Cost estimators can use psychology and behavioral economics to improve not only our cost estimates but how we communicate and work with our customers. By understanding how our customer’s think, and more importantly, why they think the way they do, we can have more productive relationships and greater influence. The end result will be better decisions by the decision makers.
Cost Management
Georges Teologlou
Program Management Track
Cost Management has a key role in project management and is the management trend due to the actual financial and economic worldwide constraints.
Cost Management solution was built to be in line with the recommendations of the GAO (US Government Accountability Office) that stated that a large number of Defense projects are at risk of experiencing cost overruns and schedule delays. GAO found that the two major causes of project failure were first an inaccurate initial estimate and secondly poor subsequent cost control.
Using Total Lifecycle Cost as the Basis for Proposal Cost Evaluation
Ryan Timm
Program Management Track
This paper discusses benefits and obstacles of using the TLCC of an entire system instead of the bid price for the purpose of proposal cost evaluation. An example acquisition of a space vehicle (SV) ground system explains the impact to TLCC of portions of the ground system developed by the SV manufacturer, SV operations, external software interfaces, government furnished IT infrastructure, and legacy sustainment until transition. Using TLCC allows higher-level requirements and enables innovative architectures.
A Maturity Analysis of the American and European Cost Community
Mark Gilmour
Program Management Track
The Cost Engineering Health Check (CEHC) is a cost capability benchmarking tool used to identify strengths and weaknesses within organizations costing functions. In 2014 the authors of this paper conducted a CEHC at the ICEAA conference in Denver. That workshop used e-voting to anonymously gather the views of the audience in attendance. This paper presents a summary of these results and contrasts them against data gathered from European workshops, providing a snapshot of the international costing capability.
Presenting Estimates
Melissa Teicher
Program Management Track
Decision‐makers rely on cost estimators to guide them in making the most educated judgments on what is best for the program or organization. A thorough and well‐organized presentation can make the difference between an easy decision and a difficult one. This paper discusses how public speaking skills contribute to an effective estimate presentation. It also delivers tips and tricks on how to make an estimate presentation more visually pleasing and easier to understand.
Risk Track
Track Chair:
Jeremy Goucher
Using Stochastic Optimization to Improve Risk Prioritization
Eric Druker
Risk Track
Minimizing the cost of complex programs is critical for government agencies trying to meet their missions in today’s fiscal. Today, identifying cost-savings measures is a manual procedure where an analyst must make an educated guess as to what risks to mitigate. This process is manual and slow. This paper will outline a methodology for using the emerging field of stochastic optimization to automate the identification of cost-savings measures on complex programs.
Why Are Estimates Always Wrong: Estimation Bias and Strategic Misestimation
Dan Galorath
Risk Track
The biggest source of estimation error usually comes from people, either by accident or strategically. This is a disaster because viable estimates are core to project success as well as ROI determination and other decision making. Most people don’t know how to estimate. Those who estimate are often optimistic, full of unintentional bias and sometimes strategically mis-estimating.
This paper discusses the issues of estimation bias and strategic mis-estimation as well as how to mitigate these issues.
A “Common Risk Factor” Method to Estimate Correlations between Distributions
Marc Greenberg
Risk Track
A “common risk factor” method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared leading to an estimated ‘intersection’ of pair of common risk factors. This intersection is then assumed to be a proxy of correlation.
Data-Driven Guidelines for Correlation of Cost and Schedule Growth
Sidi Huang
Mark Pedigo, PhD
Chris Shaw
Risk Track
When running Monte Carlo simulations, one of the most difficult aspects to account for is correlation between the independent variables. Ignoring correlation generally reports overly optimistic analysis results; industry standard suggests that when no additional insight is available, a default value of 0.2 can be used. This study analyzes historical NASA program data to develop data-driven guidelines for correlation. Moving forward, this study can be replicated to other industries and organizations with critical schedules.
Risk Mitigation: Some Good News after the Cost / Schedule Risk Analysis Results
David Hulett
Risk Track
The results of your cost‐schedule risk analysis say you are likely to be overrun on both schedule and budget. Are you are out‐of‐luck? No. Do not forget the risk mitigation phase. Risk mitigation typically proceeds from a prioritized risk list developed from analysis of the risk and uncertainty‐loaded schedule. Complete mitigation of risks is not usually feasible, and uncertainty is usually deemed to be irreducible. Some good news comes from the risk mitigation actions.
A Mathematical Approach for Cost and Schedule Risk Attribution
Fred Kuo
Risk Track
The level of sophistication in cost estimate has progressed substantially in the last decade. This is especially true at NASA where, spearheaded by the Cost Analysis Division (CAD) of NASA Headquarters, cost and schedule risks in terms of confidence level have been codified. In this paper, the author attempts to employ the portfolio approach in calculating risk attribution; first applying to a portfolio of cost risks and extend the same concept to schedule risks analysis.
Triangular Distributions and Correlations
Jennifer Lampe
Risk Track
Triangular distributions are often used in estimating cost risk because the math is relatively simple. The concept of an estimate having a lower limit, an upper limit, and a mode is easy to understand.
We show the math behind triangular distributions and correlations.
What does it mean to say that two risks are 30% correlated?
How do risk analysis programs handle triangular distributions and correlations?
How can you use random numbers in Excel to generate a Monte Carlo simulation for any defined triangular distribution?
The Impact of Selected Assumptions and Core Tenets on Schedule Risk Assessment Results (A Progressive Model Comparison)
James Quilliam
Risk Track
In the quest to ensure the sound representation of Schedule risk assessment (SRA) simulations this case study will provide a progressive model comparison of schedule risk assessment assumptions and core tenets. The elements of this approach will focus on the: methodology and tools; the progressive assumptions and core tenets applied; conclusions and lessons learned for practitioners. This will greatly enhance the understanding and confidence that leadership and project teams have in the schedule risk assessment results.
Wine as a Financial Investment
Michael Shortell
Adiam Woldetensae
Risk Track
The purpose of this presentation is to look into wine as an investment and analyze its characteristics as a financial asset. The goal here is to look into vintage wines through time, analyze its profitability and predict how investing in wine could be a good source of diversification. Our objective is to detect the types of wines that are a good source of investment and their associated risk. We aim to identify a good investment portfolio for a wine collector in order to maximize returns and minimize the associated risk.
2015 Best Paper: Risk Track
Beyond Correlation: Don’t Use the Formula That Killed Wall Street
Christian Smart
Risk Track
Risk models in cost estimating almost exclusively rely on correlation as a measure of dependency. Correlation is only one measure (among many) of stochastic dependency. It ignores the effect of tail dependency, when one extreme event affects others. This leads to potential outcomes that do not make sense such as a program with a large schedule overrun but no cost overrun. This method has been widely blamed for the financial crisis a few years ago. The copula method is introduced as a way to overcome these deficiencies.
Connecting the Dots: Integrating the risk cube with the POE
Christina Snyder
Risk Track
DoD guidance instructs program managers to include risk and uncertainty analysis in their cost estimates. Program managers also maintain a risk cube that tracks known program risks and assigns a likelihood and consequence. Risks, regardless of if they are programmatic, technical, or schedule defined, will have an effect on the overall program cost. This paper investigates the linkage between identified risks and POE cost and shows how to translate the risk cube into dollars.
Methodology for Integrating Risk mitigation Activities into JCL analysis
James Taylor
Justin Hornback
Risk Track
The intent of this whitepaper is to outline a an improved methodology for capturing risk mitigation activities for both cost and schedule into a Joint Confidence Level (JCL) analysis and assessing mitigation effectiveness.
Software Estimating Track
Track Chair:
Daniel French
Michael Shortell
The Continual Pursuit of the One True Software Resource Data Reporting (SRDR) Database
Remmie Arnold
Peter Braxton
Software Estimating Track
This paper will demonstrate the ability for one database to store Software Resource Data Report (SRDR) source data in its original form, as submitted by WBS element and reporting event. This database allows evaluations, interpretations, and annotations of the data, including appropriate pairing of Initial and Final reports; mapping of SLOC to standard categories for the purposes of determining ESLOC; normalization of software activities to a standard set of activities; and storage of previous assessments.
Agile Estimation Using Functional Metrics
Thomas Cagley
Software Estimating Track
Agile methods have matured and are now being integrated into many different approaches to the development of software. Estimation has been problematic for all methods, agile to plan based and therefore it tends to be a lightning rod for experimentation and synthesis such as is being described in this paper. Agile Estimation Using Functional Metrics has presented a path for integrating the discipline found in functional metrics with the collaborative approaches found in agile parametric estimation.
Put some SNAP in your estimating model
David Lambert
Software Estimating Track
An effective estimating model is dependent upon having the capacity to accurately size the problem domain for the solution being developed. Function Point Analysis is a widely accepted industry and ISO standard for sizing functional requirements; however, it does not take into account the sizing of non-functional requirements. Learn how the newly released Software Non-Functional Assessment Process (SNAP) can provide a more complete and accurate assessment of project size.
Forecasting Software Maintenance
Alea DeSantis
Software Estimating Track
Providing a defensible estimate for software maintenance can be complicated. By overlaying historical data with lifecycle events one can derive event driven factors and use trend analysis to forecast future Software Problem Reports. Topics addressed are: common-cause variation, special-cause variation, severity level, complexity level, and decay rate. This method moves away from using a general defect rate calculation to a more complete product-specific calculation.
Effective use of Function Points for Analogous Software Estimation
Daniel French
Software Estimating Track
Analogous estimation is often used to estimate software projects, especially in the early phases, because it is believed to be quicker and easier than other techniques. However, many organizations do not employ this technique effectively and correctly. One pitfall is improperly applying the technique & basing these estimates on Software Lines of Code (SLOC). This presentation will discuss common mistakes made when using analogous estimates as well as how the use of IFPUG function points can greatly improve this estimation technique
NASA Software Cost Estimation Model: An Analogy Based Estimation Method
Jairus Hihn
Software Estimating Track
Software development activities are notorious for their cost growth. While there has been extensive work on improving parametric methods there is very little focus on the use of models based on analogy and clustering algorithms. In this paper we will describe the methodology being used in the development of a NASA Software Cost Model using data mining clustering algorithms and evaluate its performance by comparing it to estimates from COCOMO II, a calibrated COCOMO II, liner regression, and K-nearest neighbor models.
Estimating Agile Software Development
Bob Hunt
Software Estimating Track
The use of Agile Software Development continues to grow in large and small Federal programs. These programs often utilize Story Points as a key size metric. This presentation will provide an update on Agile Cost Estimating techniques and emphasize how to utilize Story Points as a key size metric.
How Much Does Software Maintenance Cost?
Cheryl Jones
James Judy
John McGarry
Software Estimating Track
To accurately estimate, budget, allocate, and justify the resources required to meet evolving mission and service affordability requirements across the system life-cycle we must be able to answer the fundamental question: How much does software maintenance cost? This presentation provides a summary of the findings from an ongoing U.S. Army study that is focused on developing improved software maintenance estimation processes, cost relationships, and associated models.
Exploring DoD Software Growth: A Better Way to Model Future Software Uncertainty
Nicholas Lanham
Software Estimating Track
This presentation highlights trends and estimating relationships derived from the April 2014 SRDR data. This analysis provides the estimating community with several new variables to better model future software growth uncertainty. In addition, the analysis includes growth by contract type, domain, and program type as well as introduces a multivariate model to predict final hours. All of the relationships included within this slide set have been derived from the best 400+ data points, were approved by DCARC, and are DoD-specific.
Considerations in Data Center Migration Life Cycle Cost Estimating: A Joint Information Environment (JIE) Perspective
Dennis McKeon
Software Estimating Track
The potential benefits of consolidating data centers has led to the creation of initiatives such as the DoD Joint Information Environment (JIE). The goal is to reduce IT infrastructure costs by consolidating system support functions, improve hardware utilization, and lower energy and facility costs. This paper is from the perspective of an organization migrating to a consolidated data center. It will describe the migration steps, cost estimating approach, benefits, risks, and organizational challenges from their perspective.
At the Intersection of Technical Debt and Software Maintenance Costs
Arlene Minkiewicz
Software Estimating Track
Ward Cunningham introduced the technical debt metaphor in 1992 to ease business leaders understanding of the cost of making poor decisions during software development. While this author has seen many different definitions of technical debt, it is basically an indication of the amount of ‘should fix’ issues that remain in production code. This paper reports on an on-going research effort to understand technical debt and its implications on software maintenance cost and effort estimation.
2015 Best Paper: Software Estimating Track
Early Phase Software Cost and Schedule Estimation Models
Wilson Rosa
Software Estimating Track
Software cost estimates are more useful at early elaboration phase, when source lines of code and Function Points are not yet available. This study introduces effort estimation models using functional requirements as a size predictor and evaluates development schedule using estimated effort based on data from 40 military and IT programs delivered from 2006 to 2014. Statistical results show that estimated functional requirements and peak staff are significant contributors and that estimated or actual effort is a valid predictor of development duration.
Software size measures and their use in software project estimation
Harold van Heeringen
Software Estimating Track
Using the right size measure is extremely important for software cost engineering purposes. However, the number of software size measures is growing rapidly and there is a lot of confusion in the industry with regard to the methods that should be used. The advantages and disadvantages of the major software size measures are discussed and recommendations are given on what size measures should be used to estimate the different types of software systems.
Software Size Growth
Corinne Wallshein
Software Estimating Track
NCCA will present their findings including cost estimating relationships (CERs) of software size growth using selected subsets. CERs will be compared using initially estimated parameter distributions against CERs using software size. Other parameters include requirements, staff size, hours, and duration. As the 2014 update of the Joint Agency Cost and Schedule Risk and Uncertainty Handbook advises use of standard error for uncertainty bounds, NCCA will compare bounds for recommended CERs, by selected data subsets.