Search past ICEAA Workshop Proceedings in the table below and click the title to access the downloadable files.
2007-2024 Workshop Proceedings are available online. For 2006 and earlier, please email us.
| Title | Author(s) | Summary | Year | Track |
|---|---|---|---|---|
| Data-Driven Lifecycle Analysis to Optimize Cost, Risk, and Sustainability | George Bayer | Many government infrastructure investments adhere to a standard lifecyle to estimate program cost, plan replacement timing, and compare business cases to one another in a cost-benefit analysis. What if those lifecycle replacement timelines are inconsistent with system sustainability and are not cost-effective? Some infrastructure systems which are replaced according to an end-of-life schedule can be sustained more cost-effectively for longer periods of time via preventative maintenance. Our team examined multiple infrastructure program replacement timelines and analyzed operational effectiveness, cost/risk trade-offs, system redundancy, and sustainability, and we recommended lifecycle adjustments based on those considerations. We reduced overall program cost by extending replacement timelines, eliminating system redundancy without compromising sustainability, and reprioritizing maintenance portfolios on critical backlogs. We document a comprehensive process to customize program lifecycles to optimize cost, risk, and sustainability. | 2024 | Analytical Methods |
| Triage the Sub-Projects: Calculating and Applying Portfolio Contingency | Stephen Koellner | Risk-adjusted cost estimates are needed to understand the potential range of actual costs through execution. Cost risk analysis produces uncertainty distributions which can be used to calculate an expected cost as well as contingency, which can be thought of as the difference between expected cost and a higher confidence level chosen for planning purposes. In a portfolio of projects, allocating uncertainty at the portfolio level will result in a different risk-adjusted cost than applying the same allocation at the project level, and so it is unclear whether a portfolio should allocate and manage risk-informed contingency at the portfolio or project level. This topic will explore methods for calculating portfolio contingency, using a tangible example to demonstrate. | 2024 | Analytical Methods |
| Things Forgotten Since College - Foundational Statistics | Jordan Harlacher | Statistical analysis is one of the foundations of cost estimating, but fundamentals are easy to overlook. This presentation will help ensure that is not the case for your next estimate as we will discuss how the data collection and organization processes can form the basis for your estimate. Once the relevant data has been collected and organized, the real fun begins, as the central tendencies and variability of the data can now be examined. The central tendencies and variability can be used to determine the most applicable distribution and assess the probability of different events occurring. We will examine the best ways to visualize different data sets, using charts and graphs to convey the information clearly to stakeholders, as visualizing the data can help inform relationships between variables. Finally, we will touch on key statistics to look for in your regression analysis to ensure a meaningful relationship is defined. | 2024 | Analytical Methods |
| Stretching Purchasing Power through Improved Escalation Methods | Amanda Schwark | Escalation methods ensure cost estimates adapt to economic changes and facilitate accuracy and reliability. The NNSA chartered the Programmatic Recapitalization Working Group (PRWG) to track mission-critical equipment directly supporting weapons activities across the NSE. The PRWG maintains a comprehensive database of equipment above the NNSA capital acquisition threshold of $500,000. The previous escalation methodology for equipment purchase price was limited to using a single equipment inflation index. Additional fidelity in price projections can be achieved by leveraging empirical price data and published indices to derive escalation rates specific to various equipment categories. This paper explores our approach to improving upon the previous escalation methodology to better inform planning and programming decisions. This approach can be leveraged when one broad escalation index is used to predict costs for many significantly differing data elements. | 2024 | Analytical Methods |
| Spacecraft Design to a Cost Target: From CAIV to Cosmos | Ryan Sukley | Perfect performance of every system is critical for space missions. Identifying capable designs is a challenging process, and one that often comes at the expense of exceeding cost targets. The Cost as an Independent Variable (CAIV) approach helps mitigate this issue by treating cost as a primary consideration in the design or procurement of systems. Establishing a fixed cost target sets a ceiling for the cost versus performance trade-off and, in the case of NASA's in-house spacecraft, enables more cost-conscious decision making. This paper examines the application of CAIV to identify upper bounds for parameters (mass, power, quantity, etc.) early in the process of designing a spacecraft that satisfies mission requirements. It describes the process of developing, maintaining, and explaining the limitations of this capability, and addresses potential applications of the approach to other commodities. | 2024 | Analytical Methods |
| Early-Stage Cost Growth CER Development | Gabriel Sandler | Capital acquisition projects at the National Nuclear Security Administration (NNSA) have experienced significant early-stage cost estimate growth, driven in part by early optimism and missed scope. To account for these potential scope changes, NNSA's Office of Programming, Analysis, and Evaluation (PA&E) developed a cost estimating relationship (CER) for construction projects which relates the actual total project cost (TPC) to its early-stage scope estimate. This methodology differs from usual CERs which model actual cost as a function of actual scope, but reflects the scope uncertainty NNSA projects have at early stages. Three cost drivers (gross square footage, hazard category, and equipment complexity) were selected as the variables to solve for the TPC. The results of the CER were compared to another PA&E CER built with actual scope and actual costs so that early-stage cost estimate growth at the NNSA for various types of capital acquisition projects could be quantified. | 2024 | Analytical Methods |
| Market Dimensional Expansion, Collapse, Costs, and Viability | Douglas K. Howarth | Most government programs set out with cost caps and minimum force requirements. Commercial projects usually begin with a budget, sales targets, and specifications. All too often, in both cases, producers and customers give little thought to the changing market structures they face. When it comes to Demand, markets self-organize to form up to four boundaries each, including 1) Upper (price-limited), 2) Outer (saturation-limited), 3) Inner (efficiency-limited), and 4) Lower (margin-limited) Demand Frontiers. When new market segments appear as different product forms with enhanced functionality over existing options, as the new markets grow, the product groupings they replace may contract across one or more Demand Frontiers. This paper examines preparing for these inevitable eventualities in an N-dimensional framework. | 2024 | Analytical Methods |
| Comparison of UMP in the Great Recession and the Covid-19 Recession | Nathan Gallacher | This piece aims to produce a review of the Unconventional Monetary Policy (UMP) used in both the Great Recession 2007-09 and the COVID-19 Recession, then compare the two recessions to show how unconventional monetary policy changed, differences in tools used by the Bank of England and the size of the tools put in place. Notably, tools such as quantitative easing see use in both recessions suggesting similarities in the aims of the Bank of England during both recessions. The main results show a significant increase in the use of unconventional monetary policy from the Great Recession to the COVID-19 Recession. At the same time, inflation outcomes were worse during the COVID-19 Recession. This suggests that the greater reaction by the BoE in the use of UMP towards the COVID-19 Recession may not have been as effective in controlling inflation compared to the Great Recession. | 2024 | Analytical Methods |
| Explosive Analysis: Using Data to Hold Warfare Centers Accountable | Ryan Webster | The Joint Service Explosive Ordnance Procedure Publications program creates and maintains critical documents for the safe handling of ordnances. This effort is managed by Naval Warfare Centers. Historically, senior leadership has funded these efforts without the ability to evaluate reasonableness of annual funding requests. Augur has recently obtained publications system data, resulting in valuable analysis of historical efforts. This data is being leveraged to develop a planning calculator, capable of estimating ranges of labor hours based on ordnance type, country of origin, and other complexity drivers derived through regression analysis and other visualization techniques. This tool and the accompanying insights will enable senior leadership to negotiate with warfare centers and more easily measure performance. | 2024 | Data Science & Machine Learning |
| Maximizing Analysis of Minimalized Datasets | Taylor Fountain | Many techniques exist to determine parametric relationships within large datasets. While cost estimation relies heavily on identifying such relationships, a data-scarce environment, driven by factors such as vendor proprietary restrictions, security concerns, and the uncertainty of emergent technologies, is a common barrier in implementing these techniques. This topic will evaluate common methods to analyze minimalized datasets for developing defendable cost estimates, such as complexity factors and 3-point distribution fitting, and demonstrate the statistical impacts of their underlying assumptions. | 2024 | Data Science & Machine Learning |
| Labor Mapping in Parametric Estimates | David Ferland | Contractors and Original Equipment Manufacturers (OEM) alike often struggle applying specific resources or labor categories to their parametric estimates. Many parametric modeling techniques produce hours by generic resources that still need to be translated into labor resources that have rates and other attributes before they can be useful for analysis. I will outline a tool development framework that fills this gap and allows the cost estimates to stay in-sync with downstream tools like ProPricer that may compile the final estimate. This case study uses TruePlanning® as an input to the pipeline but can be applicable to most parametric sources. In cases where Basis-of-Estimates (BOEs; as opposed to Realistic Cost Estimates or RCEs) using proposed resource hours are still being required to justify parametric estimates, the traceability and justification of these pipelines is also an important consideration. | 2024 | Data Science & Machine Learning |
| Data Cleaning in Python for Beginners | Alexis Somers | As cost estimators, we collect large amounts of data from many sources, and it's often messy. Cleaning and organizing the data often requires time-consuming manual effort before proper analysis can begin. Using Python to clean and manipulate data is one of the easiest ways to save time and maximize efficiency when working on cost or other data analyses. As a free, beginner-friendly, and versatile tool, Python is an excellent choice for processing and analyzing data. This session will cover how to get started using Python to create simple scripts that produce clean, organized data. We will use the pandas and NumPy libraries to clean datasets by correcting errors, reformatting data, handling missing values, adjusting for outliers, and more. The ability to create simple Python scripts can improve the quality of your cost estimates and other deliverables by improving accuracy, efficiency, and saving time. | 2024 | Data Science & Machine Learning |
| Going Beyond Count-Based Methodologies with Semantic Vector Embeddings | Trevor Lax | Machine Learning (ML) is a topic of persistent interest and a frequent buzz word because of the astounding capabilities it has shown across disparate fields. However, the complexity of ML combined with the overwhelming number of options can lead to decision fatigue and reduced understanding in new users. While much attention is duly focused on the data and machine, occasionally the basic components of ML, such as input data type, are not properly interrogated. Indeed, a frequently used Natural Language Processing method, Term Frequency - Inverse Document Frequency (TF-IDF), simply uses counts, which cannot encode syntactic or semantic information. An alternative to TF-IDF, Word-2-Vector, creates vector embeddings of the words in a corpus, instead of relying on sentence-level counts, and attempts to encode Semantic information. Word-2-Vector has its own limitations, such as the need for a large corpus, however, it can allow for better performance and greatly improved flexibility. | 2024 | Data Science & Machine Learning |
| Automation and Process Improvement in Cost Estimating | Anil Divvela | 2024 | Data Science & Machine Learning | |
| AI and Machine Learning/Data Science Tools for Cost Analysis | Daniel Harper | AI and Machine Learning/Data Science Tools such as Chat GPT have taken on an expanded presence in Cost Analysis. E.g., NLP is used to automate functional software sizing in commercial models. Large Language Models (LLM) may even have applications for cost and acquisition professionals. We will present an overview of modern usages of data science, to include Machine Learning, AI and data visualization. We will also provide several use cases for applying these tools in cost estimation. | 2024 | Data Science & Machine Learning |
| Costing Web App Development for Operations Research | Kyle Ferris | Commercial-off-the-shelf (COTS) web application development platforms empower analysts to leverage low-code environments to build comprehensive business tools. Therefore, understanding the lifecycle cost requirements to design, develop, deploy and maintain low-code web applications as both analytical and decision support tools for stakeholders is of interest to the cost community. We define web application lifecycle requirements as analogous to an overarching Data Operations Stack. The Data Operations Stack is a conceptual framework that describes data operations as a set of hierarchical requirements, from base-level IT infrastructure and tools to high-level business products. With this framework in mind, we describe web application lifecycle requirements through successive levels of the Data Operations Stack, elucidating the required personnel, tools, and capabilities integrated into each level. Finally, we discuss how an understanding of interconnected dependencies across the Data Operations Stack can be used to develop defensible cost estimates and manage resources for web application lifecycle requirements. | 2024 | Data Science & Machine Learning |
| From a Man-Month to an AI-Minute, Myth or Reality? | Colin Hammond | In this session I will share some of our discoveries of using AI over the last five years that can help software cost estimators and our thoughts on how AI will be changing software development costs in the coming years. Back in 1975 Fred Brooks discussed observations of software engineering, many of which are counter-intuitive in a book entitled The Mythical Man Month, we pay homage to his book title in this talk as we share some observations and quantifications of how AI is helping to improve early software estimation. I will also share our predictions on areas where AI will help accelerate software development and impact on software costs over the next few years. | 2024 | Data Science & Machine Learning |
| Implications of Generative AI (Artificial Intelligence) in Software Engineering | Arlene F. Minkiewicz | Generative AI is positioned to revolutionize software development, with potential far reaching implications for productivity. Generative AI applications leverage Large Language Models to understand language, imagery and code, then use what they learned to generate content; answering questions, organizing multimodal information, and writing text and code snippets. A McKinsey report from 2023 reports that the software development landscape is quickly changing as Generative AI applications such as ChatGPT and Github Copilot have the potential to enable software engineers to complete development tasks; achieving as much as 2x productivity over traditional development practices. Activities such as inception and planning, system design, coding, testing, and maintenance can all be aided through applications of Generative AI. This paper will include an introduction to Generative AI in the software engineering context. Following will be a discussion of productivity impacts and guidance for incorporating them into a software estimates. | 2024 | Data Science & Machine Learning |
| Distribution Free Uncertainty for CERs | William King | For this presentation we intend to introduce and demonstrate the application of conformal prediction as a tool to specify prediction intervals for any machine learning algorithm. Conformal prediction intervals offer rigorous statistical coverage guarantees without distributional assumptions and only require the exchangeability of data (a weaker assumption than independence). Moreover, generating these prediction intervals is an easy consequence of retaining the sub-models trained during k-fold cross-validation. Specifically, we intend to summarize the "CV+ for K-fold cross-validation" method (and its locally weighted variant) from Predictive Inference with the Jackknife+ (Barber, Candes, Ramdas, Tibshirani, 2021, The Annuals of Statistics), and show how conformal prediction enables distribution free uncertainty for CERs. Additionally, we plan to discuss how this technique can be applied to direct human-in-the-loop intervention when applying machine learning models. | 2024 | Data Science & Machine Learning |
| Industry Leaders' Insights: Enhance Efficiency and Simplify Your Work Using AI | Karen Richey Mislick | The modern workplace is increasingly influenced by leaders who recognize the transformative power of data analytics and AI. This presentation delves into the practical experiences and insights gleaned from industry frontrunners effectively utilizing these technologies. These leaders have not only achieved significant operational efficiencies but have also mastered the art of simplification in complex business processes. Their lessons underline the importance of strategic integration, the value of data-driven decision-making, and the transformative potential of AI-driven automation. Attendees will gain a comprehensive understanding of how top enterprises are reducing costs, streamlining operations, and fostering innovation. Drawing from real-world case studies, this presentation aims to encourage cost analysts to tap into the immense potential of data analytics and AI, turning insights into actionable strategies for enhanced work efficiency. | 2024 | Data Science & Machine Learning |
| Generative AI for Government | Conner Lawston | ChatGPT' has been making massive waves across the world in the last year! This presentation gives an introduction to several 'Generative AI' models, and how they can create new images, code, data, and text, seemingly out of thin air. We will look at the process of how to build these models, including their training dataset sizes and costs. Examples will be shown of how to use ChatGPT to generate python code for you, as well as R, and PowerBI. After the general overview, specific examples of applications to Government will be shown (including acqbot- an AI tool for generating proposals). There will also be a demo of the 'GURU' bot, which was trained on the Federal Acquisitions Regulation (FAR) pdf, and can answer questions about PPBE, EVM, and Acquisition questions. We will summarize the pros, cons, and potential risks of Generative AI, as well as the future outlook to come. | 2024 | Data Science & Machine Learning |
| The Cost-Risk Uncertainty Determination (CRED) Model – A New Approach | Cheryl L. Jones | The objective of this model is to improve the credibility of and trust in a cost estimate by: 1) Identifying, characterizing, and accounting for different cost performance factors that may be sources of risk/uncertainty that can result in creating material impacts on a software sustainment and maintenance cost estimate. 2) This approach makes visible the “knowledge gap” (if any) between "what should be known" and "what is known" about the system under assessment - this "gap" is an input used to assess a range of uncertainty associated with the estimate. 3) It also fully documents the key program issues and related performance factors that may influence the cost estimate and why. While this presentation focuses on the software domain, it is easily adaptable to other domains. | 2024 | Management, EVM & Risk |
| Schedule Risk at Early Acquisition | Gabriella Magasic | It can be difficult to construct a realistic schedule early in the acquisition lifecycle due to the limited certainty of requirements, design decisions, and other key elements of program planning. Understanding risk and uncertainty in a schedule is essential, and the GAO Scheduling Guide includes "Conducting a Schedule Risk Analysis" as one of the 10 Best Practices. A Schedule Risk Analysis (SRA) can provide quantitative insight into potential areas of delay along with associated cost impacts. However, a well-formed SRA requires clear input and structured analysis of risk events and uncertainty. In this presentation, we will discuss how to address schedule risk in low maturity projects by investigating different risk modeling techniques, reviewing existing guidance on schedule risk, and analyzing how uncertainty analysis must be interpreted and applied early in the project lifecycle. | 2024 | Management, EVM & Risk |
| Cost Estimation for Project Control | Ed Spriggs | Project control in software development is a critical responsibility of program managers and contracting officers. And although the job is a difficult one for most analysts, the inability to measure and control what is being created and tested can result in loss of stakeholder confidence and, in the worst case, a cancelled project/program. What got us here? You guessed it - agile development. The adoption of agile means less defined up-front scope and little to no requirements documentation. While that flexibility allows for more development freedom it creates more variability in the features and functionality of the delivered product. This paper will describe the best new and existing practices for forecasting capabilities (features) that can be delivered within a certain timeframe given the fixed parameters of cost, schedule, and development team size. We will explore innovative techniques to measure software features, even in the absence of requirements, using function points and story points among others. | 2024 | Management, EVM & Risk |
| Advancing EVM with a Modernized Framework | Aaron Everly | DoD's FY24 procurement budget is the largest in history. The cornerstone of this budget is the procurement of complex, technologically advanced systems. DoD programs require new technologies to meet end-user requirements; however, the challenges inherent in new technology often translate to significant cost growth. PMs utilize EVM analysis to make informed decisions and mitigate contract cost growth. The IPMDAR exemplifies DoD's recognition of the need for meaningful data by requiring a modernized data schema (machine-readable format providing near real-time cost performance). Likewise, Technomics implements a modern approach to EVM using data analytics software and BI tools applied through a framework that incorporates a comprehensive view of EVM. This paper describes Technomics' EVM Framework (EVMS Surveillance, Contract Startup, Data Aggregation, EV Analysis, and Program Management), which implements modern tools to not only reduce monthly reporting tasks but also perform powerful EV analysis that enables programmatic decisions. | 2024 | Management, EVM & Risk |
| EVM Reviews – Surveillance Reviews vs. IBRs | Sam Kitchin | Successful Earned Value Management (EVM) implementation requires an effective Earned Value Management System (EVMS) and a well-planned performance measurement baseline. Meaningful insight into project performance can only be achieved with this combination of a compliant system with the active planning and management of project execution. A critical method to evaluate adherence to EVM best practices is to conduct reviews. Compliance reviews and surveillance reviews are used to evaluate the sufficiency of the EVMS, while integrated baseline reviews are used to assess the reasonableness of a project baseline. This presentation will compare and contrast these two types of review, demonstrating how and why they differ. Key terminology, stakeholders, artifacts, timeline, and intended results will be discussed. Real life examples may be used. | 2024 | Management, EVM & Risk |
| Advanced EVM Analysis using Time Series Forecasts | Anna B. Peters | The recent digitization of contractor EVM data affords cost analysts a newfound ability to execute robust statistical and data science techniques that better predict total project cost and schedule realism. Time series analysis, a well-established method in private sector finance, is one such method. Auto regressive integrated moving average (ARIMA) models may capture the persistence and patterns in EVM data, as measured by CPI, SPI, and schedule execution metrics (SEMs). As a second option, macroeconomic regression models can measure the relationship between contract performance and external considerations, like unemployment and inflation, over time. Both techniques, moreover, may forecast future changes in EVM variables interest, like IEAC. This presentation will discuss how these types of time series models and forecasts are employed on real acquisition programs and their associated IPMDAR data using Python based tools to raise program analysts' alertness to emergent acquisition risks and opportunities. | 2024 | Management, EVM & Risk |
| Deriving Total Project Costs from WBS Elements' Probability Distributions | Rainald Kasprik | Studies on possible cost variances in major acquisition projects are focusing on total project costs in order to come to plausible project budgets with a confidence level of 80%. Different lognormal probability distributions had been worked out representing different states of uncertainty. However, these models cannot be applied when using risk management software for deriving the total project costs based on cost probability distributions for WBS elements. Due to a limited processing capacity, risk management software demands a division of the underlying probability distributions into intervals A simple discretization of the models developed to date is not possible, as these models contain unrealistic cost growth factors. Based on simulation studies, three lognormal probability distributions are presented that meet these challenges. Finally, some practical hints are given on the minimum number of intervals which still represents the curvature of a probability distribution and on how to interpret the joint CDF's not-defined areas. | 2024 | Management, EVM & Risk |
| Cascading Effects - Performance Impacts of Fragile Tasks | Tommie (Troy) Miller | The growing popularity of Joint Cost & Schedule Analysis has highlighted the need for quality Schedule Risk Assessments (SRA). Modeling schedule risk and uncertainty requires an understanding of schedule networks. Network Analytics (NA) has been furthered in recent years due to research in fields such as social networks, IT networks, and transportation networks. Key aspects of these advancements can be used in SRAs to improve our understanding of schedule risk and mature our modeling techniques. For example, epidemiologists study the propagation of diseases through a community. The techniques used to model this phenomenon can be applied to SRAs to model the propagation of task slips through schedules. This presentation integrates classical concerns in schedule analytics, principally Merge Bias, with NA processes, such as node centrality measures and edge properties, to uniquely identify fragile tasks and illustrate how delays in these tasks cascade through a schedule and ultimately affect program execution. | 2024 | Modeling |
| Data-Driven Constellation Architecture Design Using Integrated Models | W. Allen Wautlet | The modern space mission landscape requires consideration of numerous trade variables to deliver optimal mission performance at low cost. Academic methods exist to address such challenges, however, practical deployment of these methods to constellation mission design remains uncommon. This paper presents a practical space mission constellation architecture approach that employs proven statistical, data science, and machine learning techniques on the products of an integrated cost and engineering modeling framework. When deployed at the early stages of constellation development, this integrated modeling framework and analysis approach provides stakeholders insight into key design parameters that drive mission performance and cost sensitivity. Furthermore, it enables the uncovering of promising design regions in large trade spaces that can be further examined and refined by technical subject matter experts. This approach leads to better decision making earlier in the acquisition timeline and increases the efficiency of design cycles. | 2024 | Modeling |
| Mission Operations Cost Estimation Tool (MOCET) 2024 Status | Marc Hayhurst | The Mission Operations Cost Estimation Tool (MOCET) is a model developed by The Aerospace Corporation in partnership with NASA's Science Office for Mission Assessments (SOMA). MOCET provides the capability to generate cost estimates for the operational, or Phase E, portion of full NASA space science missions. MOCET is a widely accepted model in the NASA community used in full mission Announcement of Opportunity competitions since 2015. MOCET was awarded NASA Cost and Schedule Team award in 2017 and honorable mention in the 2021 NASA Software of the Year competition. The cost estimating relationships and documentation have been implemented as a standalone Excel tool that is available within NASA and publicly through software.nasa.gov. Extended mission and Level 2 work breakdown structure costing capabilities are continuing to be developed and a status will be presented. | 2024 | Modeling |
| A CASE for Estimate Analytics at the Enterprise Level | Josh Angeo | Are our estimates improving over time? What did this cost 2 years ago? When was the last time we reviewed this estimate? These questions, amongst many others, are why SSC FMC developed the Cost Analytics for SSC Estimates (CASE) tool. CASE includes over 175 cost estimates, 60 programs, and goes back as far as 2017. The tool creates comprehensive dashboards capable of analyzing programs individually and in aggregate. CASE utilizes various data sources and performs extensive data pre-processing to ready the data for Power Bi. Data pre-processing steps utilize python, DAX, and Power Query. Estimate data comes from a combination of POST reports, PDFs, and spreadsheets. Custom meta data tables were developed to enable parsing and other functions. Lastly, data sources comprising of program actuals have recently been integrated. All of this results in a new found capability to evaluate estimates using analytics. | 2024 | Modeling |
| Modeling Electronic/IT System Deployment Projects | F. Gurney Thompson III | This presentation will discuss the development and application of cost models for electronic and IT system deployment projects. The deployment projects include various technical studies and preparation activities, site survey visits, and comprehensive installation efforts across many sites. The models consider size drivers such as the amount of hardware and software systems to be installed, number of sites, scope of activities, and number of different system configurations. Project complexity can be adjusted for many system/technology intricacies and site conditions. The models have been applied successfully, with validation against actuals, in estimating deployment costs for communication network deployment projects such as data centers, air traffic control towers, and military vehicle/ship/aircraft communication systems. Additionally, these models have been applied to weapon system and train signaling equipment deployments, with model validation relying on expert judgment. This presentation outlines the model's development, stru | 2024 | Modeling |
| Recipe: Homemade Pizza (or Facility Estimate) | Kristen Marquette | Have you ever wanted to "wow" your guests with a homemade pizza, but didn't know where to start? This is how we felt when beginning our facilities estimates. This presentation will break down both recipes step by step, leaving everyone satisfied and writing rave reviews. Just as you need delicious dough, sauce, and toppings for great pizza, you need detailed structural, material, and recurring scope requirements for a great facilities estimate. We will take you through our experience with data collection spanning multiple facilities and serve up comprehensive methodologies with high fidelity. If you don't have time to create a homemade pizza or perform your own detailed facilities analysis, you can leverage the tools and methodologies provided (as to-go slices), to build your own facilities estimate based on your specific program requirements. | 2024 | Modeling |
| Well, That Escalated Quickly – A Novel Approach to Forecasting Escalation | Sean Wells | Escalation rates are an important part of estimates and as such the provenance and derivation of indices should be regularly scrutinized, yet are rarely contemplated. This paper will compare a commonly used black-box escalation resource, IHS Global Insight, to a traceable, simplified forecasting method to determine if a purely mathematical model delivers an improved level of forecasting accuracy. Our model relies on a curated set of Bureau of Labor Statistics (BLS) indices to develop a moving average forecast. With access to over 15 years of IHS forecasts dating back to 2006, spanning 800+ indices, this study has the unique opportunity to quantify the accuracy of IHS and moving average forecasts against historical BLS indices. Our paper will establish and explore various measures of forecast accuracy for use in creating defensible estimates. The goal is to provide a quick, transparent, and flexible way to develop tailored escalation projections without sacrificing accuracy. | 2024 | Modeling |
| Comparative Analysis of NASA Cost Estimating Methods | Camille Holly | NASA policy and customer expectations dictate use of various cost estimating tools depending on milestone and program maturity, regardless of level of effort or accuracy of results. This paper presents a case study of the tradeoffs of modeling the cost of an unmanned space mission using different NASA-approved parametric tools. The comparison addresses subsystem and component-level cost estimates, providing invaluable insight into the granularity of cost modeling for complex space missions and differences in results associated with more or less granular estimates. The study offers perspective on the challenges and opportunities associated with parametric cost modeling methodologies due to the varying levels of input detail, and of effort, needed to complete an estimate. It also aims to provide practical insights on the number and types of subjective decisions made when modeling costs using different approaches, and the impacts that these choices have on cost results. | 2024 | Modeling |
| The Nuclear Option: Avoiding Critical Delays with Advanced Constraints Analysis | Hannah Hoag Lee | NNSA construction projects are often subject to funding constraints. The ripple effect of funding shortfalls can be severe; projects are forced into suboptimal execution profiles that produce costly schedule slips with drastic mission implications. This experience is not unique to NNSA construction projects. Funding constraints occur in most government sectors, negatively impacting many types of projects' progression, schedule, and mission. However, since inadequate funding is often unavoidable, it is imperative to use a data-driven methodology to predict schedule deviations and calculate ideal cost phasing to mitigate additional or unanticipated implications on project timeline. This paper demonstrates how a constrained phasing model uses historic project cost and schedule data to estimate a new project timeline based on a constrained funding profile. It also reveals how the model re-phases costs for the remainder of the project duration to generate a viable execution plan. | 2024 | Modeling |
| Costing a Ballistic Schedule | Rob Carlos | Join us to explore an imminent solution addressing recurring concerns in the DoD involving cost overruns and schedule delays resulting from program practices and schedule dynamics. We'll address the power of Integrated Cost & Schedule Risk Analysis (ICRSA) & Joint Confidence Level (JCL) assessment from a DoD program office perspective, emphasizing its practicality. Such outputs yield more reasonable and quantifiable estimates by incorporating cost & schedule risk and uncertainty. We'll present a case study involving a DoD ACAT IB program, discussing the lessons learned during ICSRA implementation and JCL attainment. Our presentation illustrates the impact of ICSRA and JCL, facilitating improved forecasting, early risk identification, trade space analysis, and informed decision-making. The primary objective is to provide real world insight based on lessons learned, quantitative analysis, and creative problem solving on the efficacy, utility, and power of the ICRSA and JCL. | 2024 | Modeling |
| Flavors of Commonality: Learning in a Multiple Variant Environment | Brent M. Johnstone | Commonality – the reuse of parts, designs and tools across multiple aircraft models -- is a popular strategy to reduce program costs in commercial and military applications. But its use poses unique challenges to learning curve practitioners. This paper examines five approaches to estimating multiple variant programs using different learning curve techniques. A notional dataset is created, and the accuracy of each method is measured to highlight the advantages and disadvantages of each. This presentation should be of interest to anyone doing learning curve analysis in their cost estimates. | 2024 | Modeling |
| Installation Cost Analysis | Eric White | Navy IT program managers have been frustrated in recent years by increasing system installation costs. Large amounts of siloed but related installation cost data has previously proven difficult to analyze and identify core problem areas. This paper describes an innovative new solution to this problem, utilizing data visualization tools to combine related data sources and illustrate the trends and relationships in visuals that make it easy for program managers to consume and act upon. By dynamically expanding cost data, this visualization dashboard can express cost across time, product types, location, and more, while also offering the ability to quickly drill into the inherent cost makeups. Not only can this tool quickly identify significant variances, but also offers an explanation to those cost variances. Once the historical cost data is understood it is then used in a cost model that accounts for the time value of money across future years. | 2024 | Modeling |
| Scrutinizing an Organization's Project Planning Performance | Sergey Kozin | Explore the intriguing world of Project Planning within a typical sustainment organization, spanning nearly a decade worth of estimation and execution data for dozens of special projects as PMs, Engineers, and Estimators desperately fight to defend their budgets and keep the system operating. Did we prioritize having thoroughly developed requirements definitions or wait till the 11th hour to establish them? Was schedule and scope realistic or heavily reliant on optimism as a primary methodology? Did we find ourselves broken and send up a signal flare or accept the shackles of a constrained budget? It is accepted that no plan or estimate is perfect, but rarely do we scrutinize and quantify the errors of our ways to encourage improvements within the process. Join this thought-provoking expedition, as we use metrics to judge the performance of planning practices, seeking insights and wisdom for the projects that lie ahead. | 2024 | Processes & Best Practices |
| Mission Class in Unmanned Space Estimating | John Swaren | The cost engineering community needs consistent guidelines in addressing mission assurance processes for a given space vehicle mission risk class (A, B, C, or D) based on programmatic constraints and mission needs. This presentation reviews current considerations and research. Current best practice recommendations typically relate Mission Class to an operational environment specification that conveys quality information based upon requirements. Specific end item maintenance accessibility, reliability, structuring, testing and documentation requirements are typically driven by mission operating environment. More user-definition is needed for factoring in parts quality, test-sampling, orbit-ranges and mission duration. Mission Class estimating needs to tailor component-level Part Quality as well as affect "informed" higher-level assembly and system charges. Modeling operational environment should reflect specification flow-down, validation and documentation and modification/ integration of subcontracted material items. | 2024 | Processes & Best Practices |
| A Look Back: Ten Years of Cost Estimate Audits | Ben Wilder | Since the Government Accountability Office (GAO) Cost Guide was released as an exposure draft in 2007, GAO has used it as criteria to review and assess agency's cost estimates. This presentation will look at a 10 year period (FY13-FY23) to see (1) if there are any consistent gaps in agency performance of the four characteristics of a reliable cost estimate and (2) if there has been any improvement in scores over the course of the 10 year period. | 2024 | Processes & Best Practices |
| GAO Agile Assessment Guide: Best Practices in an Agile Environment | Jennifer Leotta | In 2020, the Government Accountability Office (GAO) released the Agile Assessment Guide as an exposure draft. After an open comment period, vetting comments, updating text, and applying best practices during audits, we have recently issued the updated Agile Guide. This presentation will provide an overview of the best practices discussed in the Guide and then take a deeper dive into Chapter 7; using program controls such as using a WBS in an Agile environment and what we have found in recent audits for programs using cost estimating, scheduling, and EVM best practices in an Agile environment. | 2024 | Processes & Best Practices |
| A Series of Unfortunate Slides | Shannon Cardoza | Embark on a journey through the realm of impactful presentations, where we unravel the secrets to captivating briefings. Picture this: a vivid showcase of real-life blunders that often muddy the waters of comprehension and engagement—slides lacking labels, drowning in excessive words, or confounded by chaotic transitions. Join us as we delve into the essence of strategic naming and purposeful design to craft presentations that captivate and inform. Witness the transformation with us as we reveal the magic of visuals and charts, drawn from successful briefings to Cost Chiefs, PEOs, and beyond. You'll discover how to master the art of avoiding tricky questions by leveraging compelling visuals and enhancing your soundtrack for seamless narrative flow. Moreover, we'll shed light on how these skills not only save valuable time and resources but become a cornerstone for professional growth — empowering you to conquer larger audiences with clarity and confidence. | 2024 | Processes & Best Practices |
| The ABC's of Contract Proposal Pricing Evaluation & Cost Analysis | Christopher Svehlak | Almost every nook and cranny of the Government relies on contracts for services, support and, well, "stuff." As a cost estimator (especially a certified one), you may not know that you probably already have the requisite base of knowledge, skills, abilities and Excel-spreadsheet-jockey talent to learn and do pricing evaluation and cost analysis of contract proposals. This presentation offers you the "what-for," the "why," and the "how-to-perform" to potentially add this tool to your arsenal. It will distinguish between price evaluation/analysis and cost analysis, their purposes, when each is needed, and explain cost realism and reasonableness. Then comes the nitty-gritty -- how to perform a pricing evaluation and cost analysis on contract proposals. The goal: you leave with a better understanding & appreciation of the process ... and perhaps even consider offering your services to the contracting department. | 2024 | Processes & Best Practices |
| The Complex Database Design Tool Belt | Jamie Bouwens | The process of designing a Dimensional Database (DDB) for complex and evolving data types can be difficult for those who have never made one before. A case study is used to demonstrate how to turn an unsustainable method of data management into a DDB using two Six Sigma methodologies, Define-Measure-Analyze-Improve-Control (DMAIC) and Define-Measure-Analyze-Design-Verify (DMADV). DMADV is a preventative method used to create a process from scratch. While DMAIC is a reactive method that is used to improve an existing process. We walk through answering questions such as: What is a fact in a varied, complex, and evolving data set? How do you visualize these fact tables and dimensions? How do you track time phased data? We illustrate that these techniques are cost saving, because they reduce rework, and, most importantly, enable individuals without extensive prior experience to successfully implement an operable DDB. | 2024 | Processes & Best Practices |
| Context-Responsive Cost Evaluation: Dynamic Approach to Cost Estimate Reviews | Brittany Holmes | Annually, MDA's Cost Estimating Directorate self-audits program cost estimates and provides each program a score ranging from 0.0 to 4.0 based on questions extracted from GAO's Cost Estimating and Assessment Guide. The questionnaire does not consider details about the program being estimated, so it assumes that an immature program can achieve the same score as an established program. This project suggests a new approach to assess estimates that takes into account the life cycle and known details of the program. This presentation compares the two scoring mechanisms, details the variables used to define optimal scores, and defines how to implement the new scoring mechanism. | 2024 | Processes & Best Practices |
| Space Fence: A Cost Analysis Success Story | Rick Garcia | In 2007, the AFCAA was tasked with providing a Non-Advocate Cost Assessment (NACA) for the Space Fence program, among others. As a central cost estimator on that team, this analysis will describe the research and actions taken by the team to develop cost and schedule estimates that represented an unbiased view of the program's most likely cost under normal conditions. This analysis will also describe the partners used for the critical independent technical assessment, as well as the successful interactions with the industry leading companies that were competing for the contract. Lastly, this analysis will describe how our team developed a Ground Radar specific expenditure phasing model based explicitly on historical ground radar programs. | 2024 | Processes & Best Practices |
| Convincing Leaders of the Value of COTS Tools for Quick Assessments | Karen Mourikas | Multiple COTS tools and industry databases exist in our profession. But many organizations prefer to develop their own tools based on their own historical data, which then better represents their own environment. However, often the effort to develop these tools can be time-consuming. What happens when decision makers need answers immediately and there isn't enough time to collect and analyze their own data? One approach employs COTS tools and their underlying industry data. But cost analysts often need to convince decision makers of the validity of using COTS tools. This presentation describes several use cases in which program decision makers needed information right away, issues facing the decision makers, how the cost analysis team convinced program leaders of the validity of using COTS tools, including their pros & cons, as well as surprising insights that emerged, ultimately enabling decision makers to determine feasible paths forward. | 2024 | Soft Skills & Trending Topics |
| Priceless Culture: Crafting a Culture for the Future of Work | Cassidy Shevlin | Priceless Culture: Crafting a Culture for the Future of Work delves into the intricate web of elements that constitute a thriving organizational culture. At its foundation lies effective leadership, setting the tone for a space where core values are not just stated but lived out daily. A unified purpose drives every team member, fostering genuine accountability across all levels. Essential to this mosaic is effective communication, ensuring that everyone is not only heard but also understood. Furthermore, the culture is enriched when leadership embraces vulnerability, showing authenticity and encouraging openness. Intertwined with all these is the spirit of gratitude, acknowledging every contribution, big or small. In an era where workplaces are rapidly evolving, crafting such a priceless culture is not merely beneficial—it's imperative for the future of work. | 2024 | Soft Skills & Trending Topics |
| Equity and Environmental Justice in Early-Stage NNSA Planning | Haley Harrison | Recent executive orders (EO13985, EO13990, EO14008) directed federal agencies to prioritize environmental justice and reduce systemic barriers affecting minority and underserved groups. As an organization specializing in decision support for early-stage planning, the Office of Programming, Analysis, and Evaluation has developed a framework for incorporating quantifiable factors as proxies for equity and environmental justice-related factors in analyses of alternatives and early-stage planning studies. This framework will be used to inform decision-makers about potential project impacts from an equity and environmental justice-focused lens. An equity and environmental justice-informed approach to planning within the NNSA can minimize the incidence of negative environmental and health outcomes, maximize the number of opportunities available to historically marginalized groups, and contribute to greater trust of the NNSA mission within minoritized communities improving equity. | 2024 | Soft Skills & Trending Topics |
| Advancing the Art of Cyber Cost Estimating | Austin MacDougall | The growth in quantity and intensity of cybersecurity threats has led to new cyber best practices, such as Zero Trust and Secure by Design. These practices present challenges when developing cost estimates for the development and maintenance of information systems. This paper examines how these topics and other new cyber trends influence costs. It evaluates the cost implications in both the design (incorporating cyber requirements into new system development) and sustainment (cyber support for existing systems) phases. This research also examines existing cyber frameworks and relates them into a cost element structure to drive data collection and methodology development. Finally, this paper translates cyber cost estimating lessons learned into recommended content improvements to the technical baseline documentation upon which cost estimators rely. Standard treatment of cyber in technical baselines should facilitate much needed consistency in the composition of cyber cost estimates. | 2024 | Soft Skills & Trending Topics |
| Mind the Gap: Bridging the Generational Divide | Jennifer Aguirre | Do you ever feel you're speaking a different language than your peers? Ever struggle relating to your IPT as they talk about recent college experiences or upcoming retirement plans? Join us as we explore various ways each generation sees the world, whether through their own eyes or through a high-res smartphone camera. Let's bridge that gap to reap the full benefits of working in a multi-generational environment enabling effective connections between cost and IPT members. With each wave of people comes new ideas, perspectives, communication styles, and workplace preferences. This diversity can be challenging to navigate and when not properly managed can cause miscommunication, feelings of exclusion, disconnected goals, and failed tech baselines. When harnessed properly, it can be the superpower enabling success within team cohesion, gathering cost inputs, and delivering estimate packages. Whatever stage of life you're in, come with us on a journey of self-discovery in the workplace! | 2024 | Soft Skills & Trending Topics |
| ChatGPT: Friend or Foe - Meet Your New EN SME | Patrick Casey | ChatGPT Friend or Foe is an insightful exploration into the capabilities and nuances of ChatGPT. Delving deep into the genesis of this AI model the presentation tracks its evolution from inception by OpenAI to its fourth iteration. Patrick Casey, a Senior Cost Analyst at Quantech Services, candidly shares his experiences with the tool, highlighting its transformative power in various use cases for cost analysts ranging from WBS considerations to innovative recipe creations. While celebrating its prowess the presentation does not shy away from addressing its limitations and security concerns, urging a cautious approach. As a grand finale, attendees are treated to an entirely AI-generated TV commercial. This engaging journey demystifies ChatGPT offering both appreciation and critical insight into this modern marvel. Beyond mere technology the presentation invites audiences to consider the impact of AI in our lives challenging us to harness its potential responsibly. | 2024 | Soft Skills & Trending Topics |
| Economics of Digital Twins in Aerospace and Defense | Patrick K. Malone | Defense and Aerospace systems engineering is transforming from a document to a digital model framework, leveraging low-cost multidisciplinary modeling, analysis and simulation tools. With these methods, engineers can specify, analyze, design and verify systems. Digital Twins enable the approach, they are digital or virtual replications of physical products and processes allowing increased speed to market and performance evaluation at reduced costs. Not straightforward is return on investment evaluation when generating cost to develop digital twins. This paper looks at development of DT architectures, capabilities and resulting life cycle cost estimates. Factors impacting DT development costs are model fidelity, design features, analytical tools, integration difficulty, scalability, and programming languages. Concepts are grouped providing practitioners tools and methods to apply digital twin concepts to recommended solutions that maintain positive ROIs and identify cost drivers. | 2024 | Soft Skills & Trending Topics |
| From "Plan and Pray" to "Sense and Respond": War Gaming Defense Acquisition | Alex Wekluk | "The most dangerous phrase in the language is, 'We've always done it this way.'" - Rear Admiral Grace Hopper. The need for flexible and rapid solutions in the face of emerging threats warrants a radical reset in defense acquisition. NATO's canonical post-World War II plan-acquire-pray acquisition processes lack the agility to meet a generational change in what military historian John Keegan calls the face of battle. A new paradigm is urgently needed to meet the exigencies of modern warfare with the adaptability of the best business firms: innovating and reacting at the speed of competition. This paper provides an innovative risk-driven framework for an Acquisition War Game that laser-focuses on key metrics such as scalability, logistical footprint, time-to-contract, and fungibility – to support today's battles and near-peer competition with our enemies. This new Acquisition War Game strategy senses and responds rather than plans and prays, meeting reality head-on in an ever-changing battlespace. | 2024 | Strategy |
| FP&A: Can We Disrupt Traditional Government Cost Estimating? | Christopher Metz | There is tremendous value potential in the cost estimates built today across Government under the guidance of GAO's best practices, but perhaps with varying realization. "Cost Teams" and "Cost Estimators" are sometimes viewed as simple calculators of FTEs times labor rate in the minds of those who do not understand where a cost estimate goes and how its value can increase the chance of mission success. At our relatively new DoD Agency, we set out to find the industry equivalent to "Cost Estimating" and found "Financial Planning & Analysis (FP&A)." As we stand-up this competency we have been gathering and integrating best practices from industry's "FP&A" and Government's "Cost Estimating" along with novel ideas and contracting vehicles to disrupt the cost estimating field to better operationalize our cost estimates, steward taxpayers' dollars, and meet the mission. | 2024 | Strategy |
| Portfolio Analysis Made Effective and Simple | Brandon Schwark | Effective portfolio analysis strategies rely on robust recognition of resource constraints, competing priorities, interdependencies, and executability. They transform complexity into simplicity. Our strategy details a flexible, efficient, and analytically rigorous evaluative framework that integrates complex sets of interconnected analyses to assist leadership with data-driven resource allocation. The framework offers solutions in data cleaning, optimization algorithms, and visualization tools that enable stakeholders to effectively navigate complicated portfolio landscapes. Applicability of the framework is demonstrated through a use case that details a facility construction portfolio expected to grow aggressively in the coming years. This paper addresses the complex and often conflicting portfolio objectives mentioned above and outlines their corresponding solutions. | 2024 | Strategy |
| Parametric Construction Estimating: Consistency through Turbulence | Cortney Collins | Not all estimates are created equal, but all are necessary. How is construction estimating different from DoD estimates? They both predict costs based on agreed-upon requirements; they both use historical information to develop parametrics; and they both exist as living documents, updated as new information becomes available, and delivered to the customer to assist with budgeting and purchasing. So – how are they different? This paper will highlight some of the major disparities – from how inflation is handled, to validity of pricing, to how the current economy factors into the estimate. We will also explore how materials could be affected by pandemics and natural disasters (hurricanes, earthquakes, etc.). All of these events can impact the prices of lumber and steel – which in turn, can have estimators scrambling to update the models. | 2024 | Strategy |
| Leveraging Cost Estimating Techniques in Price to Win Analysis | Darren Kreitler | Leveraging cost estimating techniques is pivotal in "Price to Win" (PTW) analysis for competitive bidding. This session delves into various techniques, from analogy-based to parametric and bottom-up estimating. By integrating these methods with PTW analysis, organizations can strategize optimally, balancing profitability with competitive pricing. Real-world applications underscore the benefits of this synergy, emphasizing the role of accurate cost prediction in securing contracts and ensuring sustainability in today's dynamic markets. | 2024 | Strategy |
| Cloud Estimating in the 21st Century – Okay, well in 2023! | Chris Price | Cloud deployments represent a fast-paced technology. The ability to produce quality cost estimates for Cloud Deployments is challenging. In the current state, cost estimates must be able to address Kubernetes Orchestraters, Containers, IaaS and PaaS. Cybersecurity is also key to cloud deployments and modern development processes include working in a DevSecOps environment using Agile software development approaches. This presentation will discuss all these challenges and describe ways to perform quality cost estimates for cloud deployments. | 2024 | Software |
| Simplified Software Estimation (SiSE) – Updated on Advancements and Trends | Curtis Chase | In 2019, representatives from the DHS Cost Analysis Division (CAD) presented early research findings for their Simplified Software Estimating (SiSE) approach at the ICEAA annual professional development workshop. Since then, further advancements ensued facilitated by the IFPUG Simple Function Point Method (SFP), revisions and expansion of the DHS CAD verb keyword lexicon, the addition of requirements risk and uncertainty considerations, and a full guidebook supporting the method. The addition of uncertainty reduces the risk associated with requirements and verb keyword ambiguities. It also gives the estimator the flexibility to create min/max/most likely estimates for requirements that are simply vague at this early requirements stage. As such, the sizing results take into account uncertainties related to different document writers, style, and verb interpretations. This presentation outlines some of the key findings, ongoing research, and (re-)introduces the SiSE approach, offering a more streamlined and accessible process. | 2024 | Software |
| Agile Software Development Cost Estimating | Jim Golden | This presentation will discuss agile software development cost estimating in the multi-year planning cycle. Agile software development programs focus on near term workload and activities with only limited future planning cycles identified. Future cycles are only activated as their start date nears. Any model-based cost estimate or predictive analysis for the budget needs to be flexible, responsive, and adaptive to the daily dynamics of Agile software development program planning and execution. As a cost estimator, integrating with the IPT for a particular program or project has always been a critical factor in understanding requirements, gathering data, and producing a quality estimate. With agile processes being adopted more frequently across software development organizations, cost estimators and program offices are challenged even further to work closely with developers to continuously update cost estimates. Agile sprint results reveal progress of development, and subsequently could affect the cost estimate and budget requests. | 2024 | Software |
| How to Choose a Database Storage Model | Tristan Judd | To design and implement a database solution, teams must conceptually understand how data is formatted in storage. We compare traditional ways of storing data in Excel or CSV formats with that of a scalable SQL format. Within a SQL database, data is typically stored in either a relational or dimensional format and we will explain these formats for novices with examples. Relational may be easier to implement but is less powerful than a dimensional format. We take you through the process of analyzing the types of data used in a team, and how that would be reflected in a dimensional format. The ability to query efficiently, linkage to popular business intelligence techniques, and scalable structure make dimensional databases the preferred option for structured data storage. | 2024 | Software |
| Measuring Software Development Efficiency in an Agile Environment | Benjamin Truskin | Agile software development practices, while designed to deliver value sooner and accommodate changing requirements, are not intended to mitigate cost growth. Nevertheless, Program Managers must navigate this paradigm and control risk while ensuring stakeholder requirements are fully met. Traditional metrics used to measure growth (e.g., SLOC counts, productivity factors, requirements sell-off) are likely unavailable in Agile projects and while recent DoD policy recognizes the need for metrics, agile metrics are not standardized and using them for independent estimation is uncommon. This paper discusses real-world experience balancing leadership's goals for independent analysis with the realities of an Agile environment. It will show the value of utilizing program-specific metrics and calculating useful measures such as Change Traffic and Feature (in)efficiency for producing defensible estimates, enabling better program outcomes, and providing insights for others to use themselves. | 2024 | Software |
| A Software Sizing Toolkit – Choosing the Right Approach(es) for Your P | Carol Dekkers | You've probably heard of source lines of code (SLOC) and function points as choices for software size, but what about RICEFW, t-shirt sizing, story points and Simple Function Points? Like the old adage "If all you have is a hammer, everything looks like a nail" – the most appropriate software sizing approach for your cost estimate may include multiple sizing methods. This presentation outlines the various units of measure available and outlines how and when each approach is most suitable. It's a primer for cost estimators new to software intensive systems and who need to understand what are the options available when estimating software projects. | 2024 | Software |
| Unlocking Untapped Software Metrics Potential with Jira's RESTful API | Blaze Smallwood | Many software projects manage their efforts in Application Lifecycle Management (ALM) tools, like Jira, and these tools can capture a rich set of data, which can be a treasure trove for a cost or project analyst. However, many analysts limit themselves by simply exporting flat lists of records from the tool's User Interface (UI), which ignores valuable data captured in the system that can further enhance various analyses. This paper will focus on Jira and explain how an analyst can access several interesting additional data sets from its RESTful Application Programming Interface (API) with appropriately structured Uniform Resource Identifiers (URI). This paper will also cover how an analyst can use Java or Python programming to parse the JSON data returned from the API and produce simple but powerful data formats that can inform metrics dashboards or cost analyses. | 2024 | Software |
| Risky Business: Navigating the World of Software Productivity | Dave Brown | Size and productivity are commonly cited as the two major software development cost drivers. Logic dictates that the two are related and inversely correlated. But what is the probabilistic range of uncertainty for productivity, given a software size? What is meant by “an 80% confidence level for productivity”? Cost analysts often quantify uncertainty with an S-Curve; why can't this be done for productivity directly? We use International Software Benchmarking Standards Group (ISBSG) data to estimate the distribution of productivity directly and provide closed-form formulas for the fitted distribution(s). We find that productivity (and, with certain assumptions, cost) can be estimated with an S-Curve directly, using built-in Excel formulas, with no need for Monte Carlo simulation. This result has significant implications for almost any software development cost estimate, and is particularly relevant to agile development efforts where time-boxed effort is generally fixed. | 2024 | Software |
| Sizing Agile Software Development Programs | Bob Hunt | Size is a critical element in software cost estimation. As Agile has become more prevalent, the use of lines of code as a software size metric for software estimation has become less accepted. This presentation will discus and compare sizing alternatives including "tee shirt" sizing and functional size alternatives on large Federal Programs. The presentation will provide some emerging metrics for assessing size.. Since many automated models convert functional size to physical size, the presentation will address techniques to accomplish "backfiring". The presentation will address the use of Natural Language Processing and models such as Cadence and ScopeMaster. And, the presentation will discuss models such as COCOMO III that directly convert functional size to hours. | 2024 | Software |
| Why Care About CEBoK-S if we Don’t Build Software? | Carol Dekkers | Given the increase in software-intensive programs today, it should come as no surprise to experienced cost estimators that even minor software development can render a program overbudget and behind schedule. This presentation outlines the key differences in cost estimating approaches from traditional industries (hardware, facilities, systems) versus software development, and why CEBoK-S knowledge is critical for today’s cost estimators. Given that close to 60% of software projects are deemed failures (overbudget and/or late), with little improvement despite modern technologies, understanding the basics of software cost estimating can provide a competitive advantage for anyone involved in estimating programs for which software development is a component. | 2024 | Software |
| Design to Cost: What it is and Why You Should Care | Karen Mourikas, Henry Apgar, Lisa Colabella | Design to Cost (DTC) has been around since the 1970s, falling in and out of favor. But what exactly is DTC? And will it help control program or product costs? Googling it yields various definitions, sometimes contradictory. Examples fluctuate from large defense programs to mass-produced commercial components. Implementation differs depending upon company, life-cycle phase, and objectives. This presentation is the culmination of an effort by the SoCal ICEAA Chapter to educate our community on DTC. | 2023 | Analytical Methods |
| Accuracy, Precision and Uncertainty in Cost Estimates | Timothy P. Anderson | Cost proposals for large government contracts are necessarily very detailed and typically highly precise, expressed down to the nearest dollar. This presentation explores the differences between precision and accuracy in cost proposals in the context of uncertainty and cost growth. Moreover, the presentation is interactive, with thought-provoking questions peppered throughout, providing numerous opportunities for audience participation for improved knowledge and retention. | 2023 | Analytical Methods |
| Innovative Risk-Driven Contract Pricing Strategy | Brian Flynn, Robert Nehring, Peter Braxton | This paper presents a framework or scoring matrix that quantifies contract risk using a numerical evaluation of the many factors that make or break a program, such as: experience of the contractor, stretch in technology, solidity of requirements, and degree of competitive procurement. The framework, in turn, is made operational by leveraging benchmarks from over 60 U.S. contracts, enabling data-driven selection of contract type, incentives, and share lines for use in evaluating future contract prices. | 2023 | Analytical Methods |
| One Number to Correlate Them All - Efficient Implementation of Correlation | Anh (Anne) Harris, Karen McRitchie, Christian Smart | Correlation is a key consideration in cost and schedule risk analysis, as its exclusion causes significant underestimation of uncertainty. When assigning values in the absence of functional correlation, this can be accomplished by considering every WBS estimate. However, this can be time consuming for a detailed estimate. In this presentation, we discuss an alternative method that uses a single value, which offers significant time savings and discuss its implementation in the SEER model suite. | 2023 | Analytical Methods |
| Incorporating Risk into Analysis of Alternatives Results: A Novel Methodology | Brittany Clayton, Lauren Mayer | While Analysis of Alternatives (AoAs) commonly calculate the expected cost and effectiveness of alternatives, risks to cost overruns and degraded mission effectiveness are analyzed separately. RAND developed and employed a novel methodology for incorporating these risks into cost and effectiveness results. We collaborated with each AoA working group to identify, quantify and map risks to existing cost and effectiveness model variables, allowing for results to be displayed as a comparison of the alternatives’ risk-adjusted cost-effectiveness. | 2023 | Analytical Methods |
| Using Bayes' Theorem to Develop CERs – Extending the Gaussian Model | Christian Smart, David Jo | Bayes' Theorem is a mathematical method to combine prior experience with new data and is extremely important in leveraging limited information, which is often the case in cost estimating. This paper is an extension of a previous ICEAA paper that dealt with the application of Bayes' Theorem to cost estimating. In this update we show how the Bayesian approach for linear models can be extended for different, more realistic assumptions. | 2023 | Analytical Methods |
| Alternative Risk Measures for Determining Program Reserves | Louis Fussell | NASA project managers hold cost and schedule reserves equal to a 50% joint confidence level and the managing directorates hold reserves to a 70% confidence level. These joint confidence levels are quantile risk measures. This paper discusses the drawbacks of quantile risk measures and proposes the use of super quantiles for determining project reserves. Several projects are analyzed and a comparison of risk measures is presented. | 2023 | Analytical Methods |
| The Space Between Us: A Novel Collaborative Spacecraft Estimating Framework | Alex Wekluk, Benjamin Truskin | Government space acquisitions are renowned for their complexity and can suffer from notable cost and schedule overruns. The space cost community is composed of disparate organizations employing different cost estimating methods, even when the same industry partners are building the spacecraft. This paper leverages a Technomics internally-developed space estimating framework, industry-released Cost Estimating Relationships from multiple agencies, and NRO CAAG external collaboration experience to examine and contrast previously incompatible methods, and improve synchronization. | 2023 | Analytical Methods |
| Measuring Schedule Uncertainty | Tommie (Troy) Miller | The growing popularity of Joint Cost & Schedule Analysis has highlighted the need for quality SRAs. SRAs require that uncertainty distributions be assigned to all schedule activities. The most popular technique for making this assignment employs a binning structure that assigns each task to a particular bin. Most binning structures have little theoretical support. This paper attempts to improve on this technique by discussing how uncertainty determination and allocation is conducted within the Sentinel program. | 2023 | Analytical Methods |
| Spread Thin: Managing Coefficient of Variation in Monte-Carlo Based Cost Models | Stephen Koellner | Coefficient of Variation (CV) can be utilized to determine whether sufficient uncertainty is captured in Monte-Carlo based estimates. This topic explores common barriers to capturing program level risk using the interpretation of a WBS as a linear combination of distributions. A WBS CV equation is provided to model perturbations of a baseline case and then randomized WBSs are generated to analyze CV at scale. Estimators can apply these insights to improve program estimate risk calculations. | 2023 | Analytical Methods |
| Follow the Money in Government Estimates | Geoffrey P. Boals | Struggle with finding the correct source for actuals? Contracts for RDTE items are only bundles with no breakouts for the data you need? This session we will assist estimators in how to use government contract, Military Interdepartmental Purchase Requests (MIPRs), and other sources to find the relevant data needs for a strong basis for estimating. The session will detail a proposed data structure to store relevant data from the separate data sources to help power your estimates. | 2023 | Data Science |
| "Cracking the Code": Demystifying Programming Languages for Data Analytics | Kyle Ferris, John Maddrey | In recent years, buzzwords such as "Machine Learning" and "Artificial Intelligence" have made their way into the cost community’s vocabulary. While data science concepts such as these certainly advance the cost estimating industry, their significance is lost when analysts find themselves unfamiliar with fundamental programming principles. This presentation aims to demystify and compare commonly used programming languages such as R and Python, with the purpose of helping analysts who wish to develop coding expertise but don’t know where to begin. | 2023 | Data Science |
| Level up your Cost Analysis with Data Science | Sarah Green | How can developments in the field of data science help us transform many of our analytics capabilities in the Cost Community? AFCAA is leveraging the AF's cloud-based platform VAULT and state-of-the-art toolsets to in order to increase process automation, data curation and predictive models, and therefore significantly increase both quality and efficiency of our data analytics. This presentation will include a live demonstration including a demo of custom cost applications, tools and dashboards to help you in your everyday estimating- come see how the cost community is being revolutionized! | 2023 | Data Science |
| Web Applications for Cost Analysis Use: R Shiny vs Python Plotly Dash | Eric J. Hagee, Christine K. Clay | Web applications are a new tool for the cost community. Here, we compare and contrast two common web application tools, R Shiny and Python Dash. We develop a dashboard in each tool to display various USA government inflation measures and compare/contrast the use of either tool. Our findings are summarized, future thought points identified, and relevant background on the tools, inflation measures, use cases, and tool implementation provided. | 2023 | Data Science |
| Using Machine Learning to Improve Cost Data Normalization | Bryan Mariscal | Cost data normalization is time consuming, but remains critical to building a defensible cost model. SSC's Unmanned Space Vehicle Cost Model (USCM) database contains over 120 normalized datasets that span nearly a 50-year history. Developing machine learning models from the older data that generalize well to newer data would be difficult due to data obsolescence. However, we have identified an example use case with promising initial results that will be examined in detail. | 2023 | Data Science |
| Automating the Data Preparation Process using R Programming | Zachary West | One of the most challenging aspects of the cost estimating process is the collection and consolidation of data prior to carrying out any analysis. Initial data collection and preparation steps are often manual, messy, and time consuming. I plan to present an automated solution using R Programming that efficiently collects and merges structured data reports, organizes and prepares the relevant data, and exports the results to support cost estimating efforts. | 2023 | Data Science |
| Interesting Results from EVAMOSC or "Wow There is a Lot of O&S Data" | Daniel Germony | EVAMOSC is an new OSD-CAPE developed database which ingests and normalizes Operations & Sustainment (O&S) data on over a thousand weapon systems for the DoD. In this presentation, the EVAMOSC team will review interesting results it has found related to O&S data using the hundreds of millions of datapoints it has ingested and normalized. Pareto rule? Try hyper-Pareto rule! How did COVID impact vehicle maintenance? Not how you would have expected. Are field units experiencing inflation/escalation? All this and more. | 2023 | IT& Cloud Computing |
| Applying Simple Function Point Analysis to an 804 Rapid Acquisition Program Cost | Bob Hunt, Rainey Southworth, Chad Lucas | When software is developed under the Middle Tier of Acquisition (Section 804), Adaptive Acquisition Framework (AAF), these programs tend to be actual agile software development programs. These programs focus on delivering capability in a period of 2-5 years with rapid prototypes and rapid fielding with proven technology. Sizing a Software development program remains a major challenge. This paper outlines the advantages and issues associates with using Simple Function Point analyses to size and estimating the cost for a Major Rapid Acquisition Software Program. | 2023 | IT& Cloud Computing |
| Cloud-Based Machine Learning in the DoD Environment | Conner Lawston, Bryan Eckle | Follow up to the 2022 Presentation "Supercharging Machine Learning with Cloud Computing." This project has made huge strides in the past year- having made over 5,000 predictions since February, all 100% autonomously and within the DoD ADVANA environment. There have been numerous improvements (like increasing ML accuracy via PCA, & speeding up calculation time with SPARK) and lessons learned as we expanded into production for the Joint Chiefs, MDA, Navy, Marine Corps, and more! | 2023 | IT& Cloud Computing |
| Addressing CI/CD with Automated Testing in Your Software Estimate | Arlene F. Minkiewicz | Agile development and DevOps/DevSecOps practices are becoming increasingly common in software development projects in commercial, government and government contractor organizations. Three practices arising are Continuous Integration, Continuous Delivery and Continuous Deployment. Automated testing is clearly an investment, but one that promises to lead to productivity and quality improvements. This paper presents the general concept of automated testing focused on practices highlighted above. The cost impacts of employing automated testing practices will be demonstrated and discussed. | 2023 | IT& Cloud Computing |
| Comparing the Cost of Cloud Services: AWS vs Azure | Gunnar Nichols, Patrick Casey, Sergey Kozin | Cloud Services continue to play a growing role in how the internet is used to provide capabilities to users. Amazon Web Services and Azure are the two biggest Cloud Service Providers on the market today. This research investigates cost differences for cloud services comprised of comparable virtual machines for workloads with defined computational requirements. This analysis is utilized to characterize scenarios where cost savings should be considered when supporting a Cloud Service Provider decision. | 2023 | IT& Cloud Computing |
| Shining Rays of Light & Savings on Cloud Portfolios: An Important Advance! | Kenneth Rhodes, Alex Wekluk, RJ Krempasky | Existing cloud cost estimating and pricing tools have at least two significant drawbacks: 1) they require many assumptions, therefore a detailed understanding of current/future architectures and 2) they do not provide insights required to manage costs. Technomics developed a groundbreaking alternative that will revolutionize how cloud costs are estimated and managed. This paper describes our parametric-based toolset, which enables cloud lifecycle costs estimation based on known programmatic design requirements and cost reduction and efficiency analysis. | 2023 | IT& Cloud Computing |
| Budget Execution and Margin Simulation (BEAMS) | Erik Burgess | ODNI works with organizations in the Intelligence Community to submit multibillion-dollar budget requests to Congress, based on cost estimates that are not exact. For an agency portfolio, this study quantifies the impact of budgeting to higher or lower confidence levels. By modeling the interactions among programs and the mechanics of the annual budget closure process, we show that estimates somewhat higher than 50 %tile are the "sweet spot" between minimizing growth and maximizing missions delivered. | 2023 | Modeling & Case Studies |
| Foundational Cost Models | Paul Broaddus Franklin Jr. | This study will unveil more than forty-five previously unpublished simple and learning-augmented cost estimating relationships for common construction materials and equipment. These relationships and other data allow us to investigate the hypothesis that the error distribution of conventional building components follow a skew exponential power distribution rather than the lognormal one usually applied to defense estimates. We also examine whether backcasting to the intercept can isolate approximate labor cost underlying composite expense data. This paper shares those relationships in the hope of expediting cost estimating from the ground-up. | 2023 | Modeling & Case Studies |
| CSI EU (Cost Scene Investigation – European Union) | Doug Howarth | In 2000, several Airbus units from European Union member countries launched the Airbus A380. The program was projected to sell 1250 units with a development cost of €9.5 billion but shut down after selling 251 copies, as costs more than doubled. This paper examines its causes and how the entire debacle might have been prevented, as all the information needed to assess its viability existed before it started. | 2023 | Modeling & Case Studies |
| Trouble With the Curve: Engineering Changes and Manufacturing Learning Curves | Brent M. Johnstone | Engineering changes pose a dilemma for estimators: If learning curves assume cost improvement due to repetitive build, what happens when that repetition is interrupted by a change of task? Design changes are common occurrences, but rarely addressed in learning curve literature. This paper addresses how to analyze an engineering change by breaking it into its pieces and outlines techniques to calculate the reversionary impact on the learning curve to derive the estimated cost of change. | 2023 | Modeling & Case Studies |
| Developing a Schedule Model From a Cost Modeler's Perspective | Benjamin Kwok, Daniel Newkirk | Cost estimators need to utilize and develop estimating methods for project components beyond cost. Schedule is one of those components. The team that brought you USCM is now applying their model building experience to the world of schedules. This journey has yielded some exciting new methods, products, and processes. It has also brought challenges as we could not address schedule exactly as we would with cost. | 2023 | Modeling & Case Studies |
| All in the Hierarchy: Meta-Estimators to Standardize Work Breakdown Structures | Maura Anne Lapoff, Patrick Shelton | The National Nuclear Security Administration (NNSA) has demonstrated how to use Machine Learning and Natural Language Processing to map disparate cost data to a standard, high-level Work Breakdown Structure (WBS). However, mapping data to deeper WBS levels becomes increasingly complex due to the hierarchical relationship between levels, rendering common machine learning models inadequate. Here we demonstrate how to implement a Hierarchical Classification Machine Learning scheme to map multi-level, hierarchical cost data to a common WBS. | 2023 | Modeling & Case Studies |
| The Cost of Bureaucracy | Zachary Matheson, Jenna Vandervort, Daniel Mulrow | The National Nuclear Security Administration (NNSA) has separate management methods for capital construction projects, with enhanced requirements for larger projects. Naively, one might expect that enhanced requirements might increase program management costs and possibly impact schedule. However, as the enhanced management requirements are tied to a dollar threshold, comparing costs is not straightforward. In this paper, we share results of a regression analysis that developed separate cost and schedule regressions for each management method and a comparison of the costs by management type. | 2023 | Modeling & Case Studies |
| Unprecedented – Accurate Estimating in the Hypersonics Era | Eric Sick, Christian Smart | Many cutting-edge programs and emergent domains, such as hypersonics, have few direct historical precedents. This presents challenges in estimating the cost of such programs at the system level. A key insight is that many of the components on these systems have been implemented on numerous other historical programs, so one way to successfully tackle this challenge is by estimating at a component-level Work Breakdown Structure. This paper provides comparative examples of accurate estimates for aerospace advanced technology programs, including commercial space and government hypersonics programs | 2023 | Modeling & Case Studies |
| Systems Engineering Cost Estimation Aligned with MIL-STD-881F | F. Gurney Thompson III, Vivian Tang | This presentation reviews advancements in estimating top-level Systems Engineering efforts. We will propose a new model structure for alignment with MIL-STD-881F definitions, and improved scope control and output usefulness. We will discuss the drivers of effort distribution within systems engineering activities and resources, including system criticality level, safety risks, logistics scenarios, human factors considerations, technology readiness level and more. This presentation also discusses ongoing data analysis efforts to support model validation and prototype demonstration. | 2023 | Modeling & Case Studies |
| Cost Estimating: The Estimator's Guide to a Parametric Universe | Taryn Anne Reilly | Imagining tomorrow's cost estimating using parametric methods. Experience the advantages of parametric estimating throughout the opportunity and program lifecycle. Learn ways to significantly improve your estimating efficiency, accuracy and traceability through historical data calibration. Develop cost early so it influences the design process. Understand the importance of a standard WBS and curating completed program data to build calibrated models. Augment information in the Cost Community through data sharing, improving publicly available sources, methods, models. | 2023 | Modeling & Case Studies |
| Shortfall Analysis – Creative Approaches to Problem Quantification | George Bayer, Brian Carroll | Government acquisitions start with problem identification as a means of solving public sector problems or "shortfalls" and inefficiencies with technical hardware and software solutions. To justify government business cases, analysts are tasked to (1) define the problem, (2) identify impacted stakeholders, and (3) creatively quantify and monetize impacts to operations and services. Using a proven approach, the team demonstrates how to effectively perform shortfall analyses and monetize the largest underlying value to justify business cases. | 2023 | Processes & Best Practices |
| You've Invested, Now Sustain it: Insights into the Sustainment Review Process | Adam Kidwell, John Liss, Alex Bonich | Congress, in NDAA 2017, established Sustainment Reviews to assess the planning and execution of Major Defense Acquisition Program (MDAP) Product Support strategies. Readiness is a critical aspect of Defense Operations, and rising supply chain costs highlight the fact that effective planning, budgeting, and execution of a program’s Sustainment Strategy is crucial to its long-term success. This paper provides an objective, experience-based perspective of Sustainment Reviews and best practices to ensure successful outcomes. | 2023 | Processes & Best Practices |
| Educating Future Cost Engineers in the Space Station Design Workshop (SSDW) | Fabian Eilingsfeld, Nicolaus Millin | The space sector seeks young cost engineers who can work in a concurrent engineering environment. One means for scouting young talent is the annual Space Station Design Workshop (SSDW) at the University of Stuttgart, Germany. Two competing teams (size: 20) each must design a space station within one week. SSDW’s concurrent design facility (CDF) format offers students a unique opportunity to learn in a hands-on, international, and interdisciplinary environment. The use of cost estimating tools for teaching at SSDW is described. Lessons learned from the 2022 post-workshop survey are presented. | 2023 | Processes & Best Practices |
| From Software CONOPS to ROM Estimate in Six Easy Steps | Carol Dekkers, Dan French | Creating software estimates using a Concept of Operations (ConOPS) document is a challenge for software estimators due to the low fidelity of the requirements. This presentation will illustrate how to create a defensible ROM (Rough Order of Magnitude) Estimate using the new IFPUG software functional size measure. Carol and Dan will also explain how to use this size metric to develop a ROM Cost and Schedule by leveraging various software estimating techniques. | 2023 | Processes & Best Practices |
| Interviewing Subject Matter Experts | Melissa Stone | As estimators, we often must obtain technical information from Subject Matter Experts (SMEs) in order to provide thorough and accurate analysis. But sometimes getting the information you need is easier said than done. This presentation discusses the steps for interviewing SMEs in order to make the most of their time and your time. It explores the benefits of building a cooperative relationship of mutual respect and the "dos" and "don'ts" of SME consultations. | 2023 | Processes & Best Practices |
| NATO Transformation Command: Launching a Cost Estimating Capability | Candace Mahala, Ryan Feeks, Michael Thompson | After over 70 years of establishing NATO, the Transformation Command recognized a cost estimating gap in its capabilities. They launched the development of a new cost estimating capability, for the pre-concept and concept stages remaining solution-agnostic at programmatic level. This is the story of how to grow and cultivate cost estimating from program inception, establishing the fundamentals, integrating with other NATO organizations’ cost communities and developing a closed-loop cost track for robust data. | 2023 | Processes & Best Practices |
| Managing Schedule Risk Expectations During Program Execution | Patrick Malone | Program managers use Integrated Master Schedules to complete projects. It is a time phased logical view of the work and activity network. Organizations identify risks using rule-based risk taxonomies; or selecting risk exposures and impacts from tables grading them low to high. These assessments are carried forward but not always implemented in the schedule. We investigate schedule risk methods to provide an approach to more accurate forecasting resulting in a process to implement across programs. | 2023 | Processes & Best Practices |
| Take the Blue Pill! | Javier Provencio, Chris Hutchings, Cassidy Shevlin | Digital ecosystems are distributed technical systems that are scalable, sustainable, and often self-organizing; much like ecosystems found in the natural world. We will offer insight into what a digital ecosystem encompasses, along with relatable examples, and then explicate the virtues and value for our community in areas such as on-demand design to cost, MBSE, in-service support, and simulation. Finally, we will examine emergent relevant and significant use cases | 2023 | Processes & Best Practices |
| Accidental Proof of Calculus for Cost Analysts | Jack Snyder, Brittany Smith | Ever get derailed chasing your tail checking calculations, only to discover the cross check initiating the train wreck was flawed to begin with? Join us on an unflattering journey that began with a quick integral cross check that snowballed into the perfect proof for why calculus is a requirement for cost analysts. MATLAB, calculator, or by hand were the academic tools of the calculus trade, we’ll also share how we use Excel for integral calculus. | 2023 | Processes & Best Practices |
| Methodology for Assessing Reasonableness of Large Scientific Facilities' Costs | Raymond D. Woods, Valerie Rockwell | Aerospace analysts developed a methodology to address the questions and challenges that arise when looking at reasonableness of large scientific facilities' costs. This methodology is an iterative process of developing tailored reasonableness ranges and systematically addressing lower WBS levels to refine estimates. Aerospace will share their experience in using this method, when it is applicable, the benefits of implementing this, and associated products - including a specialized ‘Scorecard' - to provide insights to program management. | 2023 | Processes & Best Practices |
| NLP: A New Approach to SFP Estimation | David H. Brown | The need for data-driven methods for predicting software size has never been greater. This is especially true for agencies that rely on SME opinion or T-shirt based sizing as the primary method for estimating the size and cost of agile-developed software. This paper offers an alternative method — use of Natural Language Processing to automate the estimation of Simple Function Point counts – that offers two important advantages: 1) consistency across multiple counts and 2) speed. | 2023 | Software & Agile |
| The Symbiotic Nature of Requirements Quality and Software Cost Estimation | Colin Hammond | It is widely recognised that high quality software requirements lead to reliable functional size estimates, what is less apparent is the potential contribution of functional sizing to help assess and improve the quality of requirements. This paper looks at the observations from assessments of software user stories using our NLP (Natural Language Processing) requirements analysis tool, and how this symbiotic relationship is revealed and enhanced with automated analysis tooling. | 2023 | Software & Agile |
| A Novel Approach for Early Phase Agile Software Estimating and Sizing | Wilson Rosa, Sara Jardine | This study investigates how well three high-level size measures (Capability Gaps, Capabilities, and Epics) accurately predict total agile development effort and schedule at the earliest acquisition phase. These measures were obtained from early program documents including Mission Needs Statement, Concept of Operations, and Release Roadmap. Analysis of Variance and Goodness-of-Fit were used to introduce six new estimation models based on data from 20 agile projects implemented between 2014 to 2022 in the DHS and DoD. | 2023 | Software & Agile |
| Scheduling Agile-Fall - Best Practices | Sam Kitchin, Ryan Webster | A high-quality schedule can improve a program’s ability to clarify requirements, predict cost, and communicate effectively by creating a common denominator that links complex efforts across a project. Many Agile programs operate using a hybrid waterfall approach making a properly built schedule critical for understanding the relationship between agile development and government requirements. This presentation will demonstrate best practices for schedule construction in an Agile environment and its implications on cost, delivery, and performance. | 2023 | Software & Agile |
| Tales from the Trenches – Challenges of Data Analytics in a DevSecOps Program | Alex Smith | The large databases of information captured in software planning tools within DevSecOps environments contain a wealth of information. However, this data is user-input and not created with data analysis in mind, which results in several challenges when attempting to analyze. This presentation walks through the real-world process that resulted from collecting, cleaning, reviewing, and analyzing over 20,000 observations of agile software data to facilitate DevSecOps planning and road-mapping activities for a program entering the Software Acquisition Pathway. | 2023 | Software & Agile |
| How Efficient are Your Efficiency Levers? | Eric van der Vliet | For application service projects an important aspect is making services more efficient by means of efficiency levers like juniorization, test automation, robotics, etc. The challenge is to combine these efficiency improvements and determine the effort and cost impact during the maintenance period. This presentation presents lessons learned with respect to applying efficiency levers in large application service projects and shows an estimation tool that supports a combination of multiple efficiency levers with different impacts and durations and calculates the monthly impact. | 2023 | Software & Agile |
| Ticking Time Bomb of Telework: Communication and Time Management | Jennifer Aguirre, Shannon Cardoza, Annie Bagay | Today’s evolving work environment has demonstrated the importance of emphasizing soft skills in the workplace. Employers must seek candidates with high interpersonal abilities in conjunction with technical expertise. Hard skills often dominate in job-related training, while critical foundations like the ability to communicate and manage workload effectively are often overlooked. This presentation will focus on expanding a person’s marketability by discussing techniques to open these communication pipelines and increase productivity despite an ever-growing workload. | 2023 | Technical Innovations |
| Data Science's Expanding Role in Cost Analysis | Kevin McKeel, Daniel Harper, Greg Forman | Data Science has taken on an expanded presence in Cost Analysis. E.g., NLP is used to automate functional software sizing in commercial models. Data Science methods and tools such as NLP can be also used to extract data to identify spending trends and agency funding. We will present an overview of modern usages of data science, to include Machine Learning, AI and data visualization. We will also survey cost and budget data sources which are publicly available and ripe for analysis using data science methods. | 2023 | Technical Innovations |
| Software Estimation Using Functional Size Derived From User Stories | Karen McRitchie, Esteban Sanchez, Alton Ng | Natural language processing and ISO standard functional size metrics offer an opportunity for an improved approach to sizing and estimation directly from user stories. This presentation will offer an overview of how functional size generated by ScopeMaster can be used for directly for estimation in SEER-SEM. This approach uses either Cosmic or Simple Function Points and considers the uncertainty around sized, ambiguous and potentially missing requirements. | 2023 | Technical Innovations |
| Software Maintenance and Software Obsolescence: A Tale of Two Cost Drivers | Sanathanan Rajagopal | Software Obsolescence is one of the key issues in the modern warfare technologies especially in the cyber space where vulnerabilities of obsolete software are exploited. Understanding these vulnerabilities, predicting, managing them cost effectively are now key priorities of any key defence contracts. In this original research paper the author looks at the relationship of software maintenance and its key cost drivers with software obsolescence. Author will further explore the appropriate mitigation strategy and resolution approach for managing software obsolescence while providing an insight | 2023 | Technical Innovations |
| Wouldn't It Be Nice to Use All of Your Data Points: An Introduction to NICE | Steven Read | The difficulty of having non-homogeneous data when fitting a CER can be addressed using data imputation. A Non-linear Iterative Constrained Estimator (NICE) is an approach to both data imputation and CER fitting that considers the whole model and uses the WBS relationships to recalibrate systems of equations for a better fit. This presentation will explain what NICE is, how it works, and begin to explore what problems this method can solve. | 2023 | Technical Innovations |
| VEGA: Shining Light on the Battles of Data Collection and Management | Abby Schendt, Omar Akbik, Raymond Vera | Every cost organization battles both the collection and management of cost, schedule, technical, and programmatic data. This paper describes how the Office of Cost Estimating and Program Evaluation (CEPE) at NNSA has built and matured this capability since the office’s inception in 2015. This paper will present seven years of lessons-learned that will pay dividends for any cost organization looking to optimize how it collects and manages mission critical data. | 2023 | Technical Innovations |
| Hilbert's Problems III: The Post-Pandemic Cost Community | Peter Braxton | Upended by the pandemic, the cost community stands at an inflection point. Revolutions in telework and data science open doors to collaboration and innovation. After a decade, we revisit the seminal Hilbert’s Problems paper and reassess Professional Identity, Cost Estimating Techniques, Cost Estimating Implementation, Cost and Schedule Risk Analysis, and Integration with Other Disciplines. We summarize recent progress and issue challenges for future research. We recommend taking advantage of trends in Digital Engineering and Agile and providing meaningful dashboards instead of static briefs. | 2023 | Trending Topics |
| Placing a Value on Employee Well-Being | Kevin Buck, Asma Abuzaakouk | MITRE applies Social Return on Investment (SROI) principles to inform government workplace Diversity, Equity, Inclusion, and Accessibility (DEIA) decisions. An Investment Value Management Framework (IVMF) compares DEIA solutions and identifies optimal solutions. The framework includes process guidelines and an SROI model to estimate cost, benefit, uncertainty, and risk. The model translates metrics to benefits, and an early warning system manages lifecycle DEIA investment performance, comparing projected to actual SROI. | 2023 | Trending Topics |
| Standardising and Governing Cost Estimation and More in the Australian Defence | Kimberley Rowlands | In August 2020 the Australian Defence Department established a central governing branch to standardise costing practices and professionals within Defence. This presentation will walk you through the challenges we faced, our approach to overcoming them and the resulting successes. It will outline what we're focussing on now, understanding the environment of the cost professionals we work with and how we might leverage from each other. As the branch head, I am interested in maturing the profession of cost estimation in the Australian Defence Department. | 2023 | Trending Topics |
| Modeling Battery Manufacturing Complexity | Sara McNeal | Fossil fuels are the primary energy source used to drive our economies, but with limited supplies, the reliance on batteries has increased. Batteries are energy storage devices, and the chemical molecules react and produce voltage differences between positive and negative electrodes. These differences produce current flow in the energy forms to power products. In this study, the emphasis is researching cost and specifications to identify variables that have strong relationships with manufacturing complexity for structure. | 2023 | Trending Topics |
| Sustainment Contracts Cost Growth Analysis | Candice Schultheis | Contract type varies based on a program’s position in the acquisition lifecycle, the type of items/skills procured, and associated risks. Awarded fixed-price contracts are cited as having "locked in" program costs or result in savings claims at initial award, but these claims may not reflect the actual contract cost performance that occurs over time. This analysis quantified sustainment contract cost changes over time and measured cumulative growth. Cost growth comparisons between fixed-price and cost-type contract groups are presented. Root causes are identified, and work-level characteristics are analyzed. | 2023 | Trending Topics |
| Cost Engineering a MVDC Power & Energy Design for Navy Surface Combatants | Richard Shea, Henry Jones, Ann Hawpe, Victor Sorrentino | Since maritime MVDC systems do not exist, a statistically significant cost estimating relationship is not easily determined. The components required for a MVDC system do exist at some scale and technical level of maturity. However, there is nothing in production meeting all the same requirements of a Navy surface combatant ship installation. Therefore, this paper follows an approach to compare the theoretical costs of a MVDC to a MVAC architecture in a US Navy Ship. | 2023 | Trending Topics |
| The Economics of Rocket Reusability | Ryan Timm | Reusability of space launch vehicles promises some obvious recurring cost savings; but we delve deeper to enumerate and evaluate the effects of reusability on non-recurring costs of plant and equipment and rate and learning impacts. We model and explore the interplay of refurbishment and launch cadence and the maximum number of launch vehicle reuses. Finally, we compare hypothetical costs for the launch of expendable, partially reusable, and fully reusable space launch systems. | 2023 | Trending Topics |
| Using TRLs to Predict the Future of Nuclear Weapons | Abby Schendt, M. Michael Metcalf, Raymond Vera | Technology Readiness Levels (TRL) are used to measure and assess technology maturity. This paper presents innovative research demonstrating how historical TRL data can be used to perform credible, data-driven schedule analysis for programs early in development. Originally designed to help schedule analysis for the nuclear weapons stockpile, the resultant methodology combines historical milestone data and statistical methods to generate a Monte Carlo simulation of a risk-adjusted schedule for complex programs. | 2022 | Analytical Methods |
| Software Licenses: A Bill You Can't Pay? | Cheryl L. Jones, Brad Clark | For the past 20 years the DoD has emphasized the purchase and use of commercial software licenses as an approach to cut overall software development costs. Has this approach achieved its desired effect? How can one tell? This presentation will show the challenges and results of analyzing over 4,500 license records collected for the Army Software Maintenance Initiative. Implications to DoD's future maintenance costs as well as implementing its DevSecOps objectives will be discussed. | 2022 | Analytical Methods |
| 8D Cost Trades With Entanglement | Doug Howarth | Many modern goods have elements that have substantial costs or create hazards if unavailable. If a component begins to add more cost than value to a final product or becomes hard to get, the project's viability may be at risk. Here, we study the connections between a subcomponent and an ultimate product across interrelated market pairs. We look at their seven primary dimensions to visualize market mechanisms and add time as an eighth dimension. | 2022 | Analytical Methods |
| Sustainment Analysis Methodology for Cost Models and Business Cases | George Bayer, Austin Lutterbach | Government infrastructure systems often require detailed sustainability analyses - historical parts failures analysis, parts' procurement, end-of-life, and economic analyses - to (1) forecast the optimum date for an acquisition to replace aging infrastructure with a New Investment, (2) conduct a cost-benefit analysis to justify further F&E investment, and (3) provide a standard sustainment cost estimate (corrective and preventative maintenance) for acquisitions. Using parametrics, research, and software algorithms, the team demonstrates an optimum approach for sustainment analysis. | 2022 | Analytical Methods |
| Implementing Quantile Regression for Capital Acquisition Project Estimating | Zachary Matheson, Jeff Beck, Gregory Stamp, Dr. Charles Loelius | The National Nuclear Security Administration (NNSA) is investigating the use of quantile regression on NNSA capital acquisition data to determine the cost drivers of project costs at values above the median, instead of via use of Log-Ordinary Least Squares. Utilizing this methodology will provide a better understanding and quantification of the uncertainty inherent in the NNSA's capital project budgets, which are required to be budgeted at between the 70% and 90% confidence level. | 2022 | Analytical Methods |
| The Progression of Regressions | Jennifer Aguirre, Kyle Davis | Do you lose sleep at night over not being able to find the best regression for your data? Well fear no more, for this brief will alleviate those concerns. We review a case study creating a CER with a limited data set exploring multiple forms of regression (Linear, Nonlinear, Cubic, Polynomial), segmenting data into applicable portions while comparing predicted vs. actual results, and using predictive statistics and guidelines to assess best fit. | 2022 | Analytical Methods |
| Predicting and Minimizing Industrial Base Risks to Emerging Technologies | Kevin Ray | The global pandemic has highlighted the fragility of the U.S. industrial base relied on to develop the next emergent technologies. Programs are routinely impacted by industrial base issues causing schedule delays and cost growth without the analytical means to predict and prepare for them. An analytical framework was developed that leverages cost estimating methodologies to predict these industrial base issues, quantify their impact, establish a new risk adjusted baseline, and reduce risk through targeted investments. | 2022 | Analytical Methods |
| Developing Conceptual Cost Estimates for Good Ideas | Adrian S Mitchell, John Millhouse | This paper outlines the use of a super learner method for generating conceptual estimates of Australian Department of Defence facilities and infrastructure products. Many organisations require quick turnaround cost estimates for good ideas looking for budget funding. Developing estimates in the absence of firm requirements and a full understanding of proposal scope is challenging. A transformative approach to cost and duration estimation is needed to solve this problem. | 2022 | Data Science |
| Supercharging Machine Learning with Cloud Computing | Conner Lawston, Bryan Eckle, Ketan Ganatra | Modern cloud computing environments can speed up machine learning models by up to 100X! In this case study with the DoD Joint Artificial Intelligence Center (JAIC), see how AWS and Databricks can be utilized to improve existing machine learning (ML) models for detecting fraud correcting unmatched transactions. When combined with robotic process automation (RPA), this effort saved 100,000 manual hours in FY20, and identified potential fraud with over 90% accuracy. | 2022 | Data Science |
| Statistical Techniques to Improve Software Effort Estimation Data Quality for Cost Engineers | Tomeka S. Williams | Since the topic of improving data quality has not been addressed for the U.S. defense cost estimating discipline beyond changes in public policy, the goal of the study was to close this gap and provide empirical evidence that supports expanding options to improve software cost estimation data matrices for U.S. defense cost estimators. The purpose of this quantitative study was to test and measure the level of predictive accuracy of missing data theory techniques that were referenced as traditional approaches in the literature, compare each theories' results to a "ground truth" data set. | 2022 | Data Science |
| Integrating Data Science & Cost Analysis | Sarah Green | Data is more abundant than ever before. How can we as cost analysts leverage the full power of the data science world to get the most out of our data and utilize things like automation, machine learning, interactive dashboards and more? The author will demonstrate the innovative ways that AFCAA is utilizing a cloud based platform and various data science tools to transform our entire process from data ingestion/ storage to end products and even end to end cost modeling. | 2022 | Data Science |
| Managing Data Science: A Stacked Approach to Integrating Advanced Data Analytics | Eric J. Hagee, John Maddrey | Recent initiatives in applying data analytics to cost estimating have propelled the industry to evaluate how data science can be integrated into the estimating process. In this presentation we will dispel the operational hurdle of incorporating data science by introducing a stacked approach to data science integration, defining the levels of the data science stack, answering ‘What does success in each level of the stack mean?', and organizing the work force within the stack. | 2022 | Data Science |
| Cost Estimating Maturity and a Vision for the Future | Patrick Malone, Henry Apgar, William King | The science of cost estimating has matured. From the Egyptians to modern methods, cost analyses advances were progressive, recent methods further the science. This paper reflects on rich heritage and looks to the future of cost estimating and analysis using emerging and maturing techniques. We look at postmodern tools and methods like Natural Language Processing and other methods enhance our ability accurately forecast costs of more complex systems. Last, we discuss future research and trends. | 2022 | Data Science |
| Utilizing Artificial Intelligence to Evaluate Requirements | Amar Zabarah , Kevin McKeel | Poor requirements have been a problem across the commercial and federal arenas. Business Analysts face the challenge of parsing through hundreds/thousands of requirements to define and understand system needs. This is a grueling process and despite the most meticulous analysis, mistakes are still an inevitability. AI can serve analysts in the understanding and parsing of requirements and dramatically reducing the time and effort necessary to accurately evaluate requirements. This paper provides an overview of select AI capabilities and shows their applications to requirements analysis. | 2022 | Machine Learning/NLP |
| Estimation of Application Maintenance by Means of Machine Learning | Eric van der Vliet | This presentation is about an estimation model for application maintenance. The model calculates the required effort and costs for maintaining software application based on incidents with different service levels and priorities. The calculation is based on historical data collected of different business domains. To manage the large amount of data, application characteristics and domains, advanced data analytics and machine learning techniques are required. The development approach as well as the prototype results will be shown in this presentation. | 2022 | Machine Learning/NLP |
| Linear Regression: How to Make What's Old New Again | Kimberly Roye, Sara Jardine, Dr. Christian Smart | With the booming popularity of machine learning techniques, data scientists may have you believe there is no place for linear regression. Though many cost estimating applications are nonlinear, when there are linear relationships among features, ordinary least squares regression often performs better than the most powerful machine learning techniques. Scenarios for which linear regression should be chosen over more complicated algorithms are presented, as well as techniques such as regularization, gradient descent, and Bayesian methods. | 2022 | Machine Learning/NLP |
| Adopting a Data Science Paradigm | Kyle Ferris, Zoe Keita | The availability of large unstructured datasets accelerated via autonomous data collection and warehousing, makes the use of advanced analytical skillsets a necessity. The incorporation of data science methodologies can yield meaningful improvements to cost estimating practices through effective data governance, streamlined data collection/normalization, and increased opportunities for exploratory analysis. This presentation will evaluate the software and skillsets required for advanced modeling and analysis, as well as planning considerations for data science training development. | 2022 | Machine Learning/NLP |
| Advanced Natural Language Processing for Work Breakdown Structures | Maura Lapoff, Aaron Vermeersch | The National Nuclear Security Administration (NNSA) collects Work Breakdown Structure (WBS) data for Capital Asset Projects. Implementing the structure across projects grows exponentially in complexity due to varying scope, contextual changes, and vendor requirements. This paper will demonstrate how Natural Language Processing can automate the process of identifying and classifying WBS elements used for Capital Asset Projects. NLP can improve the Cost Analyst's workflow by reducing time-consuming aspects of their work. | 2022 | Machine Learning/NLP |
| Strategic Investment Planning using Machine Learning Techniques | Wendy E. Robello | Early identification of anticipated customer needs is a major component to winning new business in competitive environments. This allows bidders to make strategic investment decisions that will improve core competencies to meet anticipated customer needs. Current approaches to predict customer needs are subjective, costly, and time-consuming. This research will provide bidders an economical and objective approach using supervised machine learning to predict customer needs deduced from prioritized customer selection criteria to strengthen their proposed solution. | 2022 | Machine Learning/NLP |
| A Deep Look into Optimistic Bias based on a NASA's STEM 4th Grade Activity | Steve Sterk | Beyond this 4th, Grade NASA STEM activity found on NASA JPL's web site, this paper is a direct result based on independent research being conducted by Steve A. Sterk who works in the Program Planning and Control (PPC) Branch at NASA Armstrong Flight Research Center. It is a believed optimistic cognitive bias, is the number one problem while developing cost and schedule estimates and thus digging into human behavioral, a heuristic technique in completing the task at hand. This presentation will include; How people think, the start of an Optimistic Cognitive Bias data base for Machine Learning. | 2022 | Machine Learning/NLP |
| Delusions of Success: Overcoming Optimism Bias in Schedule Forecasting | Jeffrey Voth, Maxwell Moseley, Ann Hawpe | Schedule performance of major defense acquisition programs remains a challenge, leading to an average capability delay of more than two years and substantial cost growth, according to the latest U.S. Government Accountability Office assessment. Stakeholders must de-risk schedule estimates and improve performance through data-driven approaches based on realized prior program histories to prevent optimism bias. The authors evaluate the merits of taking an ‘outside view' to mitigate risk through reference class forecasting, utilizing empirical distributional information from 116 programs across six commodity classes to develop more realistic and reliable front-end schedule estimates. | 2022 | Management & Risk |
| Real-World Data Transformation Challenges in the DoD Supply Chain | Michael Ahearn | Merger and acquisitions have driven tremendous growth of the top one hundred government contractors. This growth has also embedded land mines in the historical data of these "compete-mates." The presenter will discuss real-world lessons from the extraction and transformation of programmatic, cost, schedule, and effort data in support of processes from Should Cost to Basis of Estimate for bid and proposal pursuit. | 2022 | Management & Risk |
| Cost and Schedule Risk Analysis of Megaprojects | Michael Trumper, Lev Virine | Cost and schedule risk analysis of megaprojects can be complex, time consuming, and may not give meaningful results. The presentation demonstrates a practical and easy to implement steps how to perform risk analysis of projects of any size. They include schedule diagnostics consolidation. The large megaproject schedule can be separated on subprojects, which can be analyzed individually. The analysis can be performed for any milestone and multiple times during the course of project. | 2022 | Management & Risk |
| Uncertainty of Expert Judgment in Agile Software Sizing | Peter Braxton, Ken Rhodes, Alex Wekluk, David Brown | Agile software estimating and planning often rely on expert judgment to assess the size of the development effort at various levels of granularity and stages of maturity. Previous research by the author quantified the inherent risk and uncertainty of the self-similar scales (e.g., T-shirt sizing) commonly used in these assessments. This paper expands those a priori mathematical results and empirically tests the accuracy of experts in applying those scales. It elucidates the ideal ratio to align with the desired confidence interval, and recommends feedback mechanisms to improve consistency. | 2022 | Management & Risk |
| Alternative EAC Methodologies – Calculating EACs without Standard EVM Data | Richard Lee, Emily Goldhammer, Kate Malcolm | Developing an Estimate at Complete (EAC) typically involves establishing a performance-based estimate founded in the best practices of Earned Value Management (EVM) Gold Card analysis. But what if you are missing EVM data or do not have a performance baseline? This presentation explores alternative approaches to developing an EAC by leveraging burn rate profiles. Estimates are then developed by applying regression analysis and/or linear interpolation techniques to the historical burn rates observed. | 2022 | Management & Risk |
| Decision Support and Operation Design in Mission Critical Applications | Amin Rahimian | This talk covers key concepts of decision making under uncertainty. Building on applications in soldier-robot teaming, we explain how to balance exploration and exploitation tradeoffs using multi-armed bandits and reinforcement learning. The second half of the talk is concerned with challenges of operation design in costly and limited-information environments. Using examples from adversarial information operations, we explain how to balance costs and benefits of data collection and how to mitigate risks to data acquisition resources. | 2022 | Management & Risk |
| Second Source Manufacturing: Lessons from the Second World War | Brent M. Johnstone | Manufacturing defense systems at different sites is increasingly common due to foreign coproduction and international cooperative ventures. These situations challenge estimators, posing questions about the transfer of learning and relative efficiency of multiple production sites. This paper examines cost history from World War II, when U.S. bomber production lines were shared across multiple companies. The conclusions are tested against modern experience and guidance provided to estimators seeking help. | 2022 | Modeling & Analysis |
| Model-based Cost Engineering launches with Space Missions Estimating | Vincent Delisle, Shawn Hayes, Mark Jacobs | NASA continually strives to improve cost estimation for the highly advanced technology flown on planetary as well as earth orbiting space missions. Over the years it has been proven that parametric cost models are a desired way to obtain accurate estimates. Still there is still room for improvement. This paper will discuss two of the latest and best methods for obtaining accurate cost estimates using best-of-breed model-based cost engineering techniques. | 2022 | Modeling & Analysis |
| NRO CAAG Parametric Model for Spacecraft-to-Launch-Vehicle Integration | Daniel Barkmeyer | New, innovative launch service providers and spacecraft designs have broadened the tradespace of launch service options for government customers, placing new importance on launch cost, an often understudied component of space programs' enterprise-level cost. This paper demonstrates a parametric approach to estimating the most mission-dependent component of launch cost, spacecraft-to-launch-vehicle integration engineering. It addresses primary and secondary cost drivers and typical time phasing, and demonstrates the model's performance compared to NRO and NASA historical costs. | 2022 | Modeling & Analysis |
| Integrating Cost into Model-based Systems Engineering Environments | Daniel Kennedy, Karen Mourikas | Model-based Systems Engineering (MBSE) incorporates digital models to represent system-level physical attributes and operational behavior throughout the system life-cycle. To date, many MBSE efforts have focused on technical requirements with little emphasis on cost. Integrating cost models and MBSE enables rapid exploration of design trades and associated cost impacts, traceability between requirements and cost targets, and earlier identification of affordability issues. We present results from several ongoing efforts as a continuation of our 2019 presentation. | 2022 | Modeling & Analysis |
| NASA Instrument Cost Model (NICM 9) Introductory Training | Joseph Mrozinski | The NICM 9 Tool suite allows for cost and schedule estimation for space flight instruments. This introductory training session will examine a case study utilizing both the System and Subsystem tools, moving from inputs and assumptions to the cost and schedule estimates and interpretations. An overview of the NICM model development methodology will be given. Instructions to obtain follow-on training and to download NICM 9 will conclude the talk. | 2022 | Modeling & Analysis |
| Cost and Throughput Analysis for the NASA Ames Arc Jet Modernization Program | Jennifer Scheel, Christian Smart | The facilities of NASA's Arc Jet Complex are used to simulate the aerothermodynamic heating that a spacecraft endures throughout hypersonic atmospheric entry, and to test candidate Thermal Protection System (TPS) materials and systems. Ms. Scheel and Dr. Smart analyze the cost and throughput impact of several different modernization options. Estimating the cost requires independent research into highly specialized subsystems and customizing inflation application. Throughput analysis of the number of possible test runs per year is conducted via a probabilistic simulation. | 2022 | Modeling & Analysis |
| Is Your Organization Ready for Model-Based Cost Engineering? | Michael Ahearn | Model-based design continues to yield unprecedented speeds in technology solution development. Integrated and data-driven Model-Based Cost Engineering ("MBCE") promises to estimate concurrently at the speed of design. Taking advantage of MBCE requires vision and a level of organizational maturity. The presenters will discuss critical attributes regarding people, processes, data and technology that impact the success of model-based costing. | 2022 | Modeling & Analysis |
| Mission Operations Cost Estimation Tool (MOCET) 2022 Research | Marc Hayhurst, Brian Wood, Cindy Daniels, Lissa Jordin | The Mission Operations Cost Estimation Tool (MOCET) is a model developed by The Aerospace Corporation in partnership with NASA's Science Office for Mission Assessments (SOMA). MOCET provides the capability to generate cost estimates for the operational, or Phase E, portion of full NASA space science missions. Research topics being studied in 2022 will be presented, including: Level 2 Work Breakdown Structure (WBS) cost modeling, and extended mission cost modeling. | 2022 | Modeling & Analysis |
| DICEROLLER: Estimating D&D Costs for the NNSA | Zachary Matheson, Charles Loelius | The National Nuclear Security Administration Office of Programming, Analysis, and Evaluation has developed a model for estimating the cost of Decontamination and Disposition (D&D) activities. This effort involved collecting and normalizing cost data from past D&D projects, and then generating a parametric cost estimating relationship to predict future D&D project costs. The resulting model, named DICEROLLER, will be used to make lifecycle cost estimates and one-for-one replacement cost estimates of capital acquisition projects. | 2022 | Modeling & Analysis |
| Applying System Readiness Levels to Cost Estimates - A Case Study | Patrick Malone | Estimating cost, schedules and expected technical performance of large complex systems pre-development is difficult. When programs are executed, they are plagued with cost growth and schedule delays due to less than required maturity of some elements. We investigate the James Webb Space Telescope program actual cost and schedule history using System Readiness Level (SRL) methods to reveal trouble areas early. The resulting approach will support future estimating accuracy through higher fidelity information for early decision-making. | 2022 | Modeling & Analysis |
| Cracking Open the 'Black Box' of Product Technical Support Contracts | Alexander Bonich, Patrick McCarthy, Rhys Bergeron | Product Technical Support contracts represent a significant annual cost to the DoD. While these contracts serve as useful "catch-alls" for various programmatic requirements, estimating these costs for future systems can be challenging. This paper analyzes historical technical support costs for various Army ground vehicle systems and demonstrates how combining contracts, work directives, and cost data sources along with categorizations and tags for "Service Categories" provides a comprehensive understanding and management of technical support cost requirements. | 2022 | Processes & Best Practices |
| Visual Exploration of Data – The Missing Element in CER Development | Benjamin Kwok | Within cost estimating training literature, it is common for the discussion around Cost Estimating Relationship (CER) development to focus primarily on its statistical parameters (e.g. correlation, equation form, etc.). An underemphasized component of CER development is the need to first visualize and explore your data. This presentation will show how integrating these processes into CER development leads to faster and better results. | 2022 | Processes & Best Practices |
| Data Management for Cost Engineering Projects | Cara Cuiule | As cost estimation data becomes more plentiful, data management becomes essential for organizing and preparing data for analysis. Even if a formal data management system is unnecessary for a project, concepts from database management can still be incorporated to ensure higher data quality. This paper will contain an overview of relational and non-relational models, along with a case study of a database that contains structured, unstructured, and semi-structured data. | 2022 | Processes & Best Practices |
| Everything You Always Wanted to Know About Affordability...But Were Afraid to Ask | Karen Mourikas, Denise Nelson | The term "Affordability" means different things to different people, depending upon one's employer, organization, function, background, etc. The term has also morphed over the years - expanded or narrowed - based on one's viewpoint. During ICEAA's OEM-COG discussion on Affordability, diverse opinions were revealed. We will assemble various interpretations & implementations of Affordability from multiple perspectives, provide historical background, and explore how our community can clarify, standardize, and promote the concept of Affordability. | 2022 | Processes & Best Practices |
| Predictive Thresholds for Schedule Execution Metrics | Michelle Jones | We called it the search for a unicorn. After an intense week of schedule analysis, checking the numbers, updating visual basic code, then presenting objective evidence that the contractor is not achieving the baseline plan, we got the action: provide the metrics of a good program. We tried, but after reviewing countless schedules, we realized that a "good" program is as rare as a unicorn, because every program deviates from the baseline plan. Although we never found a unicorn ("good program" with good metrics), we developed data driven thresholds predictive of significant milestone slip. | 2022 | Processes & Best Practices |
| The BS in BoeS – Oh the Games that are Played | Sandy Burney | A good Basis of Estimate (BoE) uses a historical comparison, and a complexity factor. These two choices are not statistically based, meaning a bias is introduced into most BoEs. This paper explores the bias in BoEs through the framework of Game Theory. A BoE goes through a creation, internal review, internal approval, and external review process, where each player in this process will likely have a different payoff matrix. Each player in this process will want to insert their bias into the BoE, which this paper terms "Bias Selectivity" (BS). | 2022 | Processes & Best Practices |
| No Estimation Without Escalation - The Inflation Revolution | Shannon Cardoza, Edward Smith | I downloaded the latest USG Inflation Indices, so my work here is done, right? OSD guidance encourages the analyst to explore program-specific escalation to maximize estimate accuracy vice defaulting to existing published indices. This briefing will provide a basics refresher background on common appropriation inflation rates, their outlay profiles, and how to interpret them. It will then explore how to improve to account for escalation, commodity-specific demand, and project-appropriate expenditure profiles increasing estimate accuracy. | 2022 | Processes & Best Practices |
| Transforming Cost Estimation Services | Kevin McKeel | In the last 10-15 years the government contracting industry has quickened the pace of technology delivery. This emerging change has forced cost estimators, as well as acquisition professionals of all kinds, to change to their own processes to keep pace. Cost estimation shops can benefit by changing their service to match customer needs and delivery tempo, add software product instruction with analyst training programs, and extend the scope of "cost" to include budget formulation. This presentation touches on transformation such cost organization infrastructure and repeatable processes. | 2022 | Processes & Best Practices |
| What Does Agile Software Development Need: Predictable Costs or Predictable Outcomes? | Christina Kosmakos | Despite ongoing laudable attempts to advance the state-of-the-art in agile software estimating processes and techniques, the software development and cost analysis communities are far from ‘cracking the nut'. There is a dire need for a cost estimating approach that accurately predicts agile project costs with the correct granularity to enable budget and execution planning. This paper will detail the exploration of three approaches that utilize extrapolation from actuals to implement a solution. | 2022 | Software & Agile |
| Simplifying Software Sizing with Simple Function Points | Daniel B. French, Carol Dekkers | Simple Function Points (SFP), modelled on the IFPUG Function Point Analysis method was introduced with the goal of simplifying the method while reducing cost, time, and difficulty estimating software size without sacrificing accuracy. We'll introduce the IFPUG version of SFP and highlight the challenges/opportunities when using to size software. We'll explore when to use SPF and key differences between SFP and IFPUG FP while providing guidance on using FP measures in software cost estimates. | 2022 | Software & Agile |
| Agile Team Performance Measurement as a Basis for Accurate Cost Estimation | H.S. van Heeringen | Agile team performance metrics Productivity, Cost Efficiency, Delivery Speed, Sprint Quality, and Product quality can be measured in an objective, repeatable and verifiable way, compared to each other, and benchmarked against industry data. The measurement data is used to recalibrate long-term effort, duration, and cost estimates based on the actual productivity delivered, resulting in increased predictability. I'll show a recent study of 4 teams of one organization, each in a different European country. | 2022 | Software & Agile |
| Dynamic Software Effort Estimation: How SWEET It Is! | William GellatlyDavid Brown, Lindsey Jones | Software estimation is a complex and diverse field that must accommodate a variety of technical inputs across the life cycle. Our team developed an Excel-based effort estimation model designed for both flexibility and transparency. It dynamically builds an effort estimating relationship based on analyst-selected data fields and source data. Our paper explores how we used data from ISBSG, statistical analysis, and practical experience to drive the development of this "clear-box" model. | 2022 | Software & Agile |
| Agile Product Roadmap Estimating and Progress Tracking | Blaze Smallwood, Ryan Blackburn | Agile project managers often struggle developing and maintaining a full project plan, given the churn in requirements and priorities typical of an agile environment. PMs often use a Product Roadmap to communicate high-level work priorities and phasing, but these often lack the quantitative information needed to enable tracking progress against overall project goals. This paper will offer an innovative solution to this problem, which has recently gained traction on an actual government agile project. | 2022 | Software & Agile |
| Let's Go Agile: Data-Driven Agile Software CERs Derived from DHS Projects | Wilson Rosa, Sara Jardine, Kimberly Roye, Kyle Eaton | This paper presents a set of effort estimating models for agile software projects using backlog data from JIRA, monthly contract reports, and requirements documents. The first set of models predict effort using either Unadjusted Function Points or Simple Function Points. The second set predicts effort using either functional requirements, stories, issues, or story points. The regression models are based on data collected from 15 agile projects implemented within DHS and DoD from 2016 to 2021. | 2022 | Software & Agile |
| Software Phasing and Schedule Growth Analysis | Daniel Long | DoD software research lacks an understanding of development phasing, effort allocation, and schedule. We evaluated conventional rules of thumb for effort allocation against projects in the SRDR database. We compared how effort allocation varies between projects with high or low schedule growth. Results showed that increasing effort in early phases consistently decreases the total schedule growth, and led to improved allocation guidelines. These findings were significant across multiple categories such as Service and project size. | 2022 | Software & Agile |
| Minding Your P's and Q's as Prices Rise | Alan Karickhoff, Brian Flynn, M. Michael Metcalf, Omar Akbik | Choices and challenges abound in combining individual prices (P's) and quantities (Q's) of labor and material into one single measure of overall escalation for a project. Popular constructs include the Laspeyres, Paasche, and Fisher. This research illuminates the issue of which index to use when in the relentless fight against Money Illusion - the tendency to think in nominal rather than real terms. Methods are offered for forecasting inflation probability distributions up to three decades out. | 2022 | Trending Topics |
| Impacts of Digital Engineering to the Cost Estimate | Brittany Clayton | Defense programs across the services are implementing digital engineering throughout the acquisition lifecycle to develop, manufacture, and sustain their platforms. How should cost estimators account for these new practices? This presentation will focus on the impact of implementing DE to the cost estimate. Based on literature reviews, discussions with SMEs, and review of policy, we will propose investments and cost uncertainties for the estimator to consider throughout the lifecycle of a platform. | 2022 | Trending Topics |
| CE^2 : Communication and Empowerment for Cost Estimators | Christina Snyder | Too often, cost estimator training focuses solely on technical abilities, largely ignoring the "soft skills". The behaviors of being a good communicator and empowering the team were shown in a 2020 ICEAA community survey to be important to cost leadership efficacy. Knowing that technical skills alone will only take estimators so far, this presentation leverages communication and empowerment training to demonstrate how all cost estimators can use soft skills to exponentially impact their analyses. | 2022 | Trending Topics |
| What is the OEM COG? | Chuck Kurtz | Cost and Affordability communities employed by Original Equipment Manufacturers (OEM) face challenges that differ from those in government or other private sector organizations. To identify and address these challenges, ICEAA formed the OEM Cooperative Opportunities Group (OEM COG) to discuss, collaborate and share best practices on topics such as Affordability, Digital Engineering, Risk and Professional Development - from an OEM perspective. Join us for an interactive session as we share lessons learned with one another. | 2022 | Trending Topics |
| Solving the Climate Crisis with Satellites, Fighter Aircraft & Nuclear Reactors | Dale Shermon | This paper examines the challenges of factory built, low cost, mass produced nuclear reactors, not by focusing on the technology, but the acquisition process. It will consider lessons learnt from satellite and fighter aircraft programmes which lead to the formation of a consortium of nations that could engage in nuclear power generation. Can we solve the problem of becoming zero emitters of greenhouse gasses while electricity demand is increasing globally? | 2022 | Trending Topics |
| DoD Cost Estimating Guide v2 | Molly Mertz, Erin E. Veltman | The DoD Cost Estimating Guide has something to offer every estimator! This session will provide an overview of the guide, with special focus on version 2. New material includes updated statute/policy references, expanded discussion of Middle Tier of Acquisition estimating, a recommended reading list, and a process-focused case study. Follow analyst Ava's progress as she navigates the challenges of a Milestone C production estimate for a fictional helicopter program. | 2022 | Trending Topics |
| Estimating Costs of Climate Change Impacts to Public Infrastructure (CIPI) | Edward Crummey, Nicolas Rhodes | What is the long-term budgetary cost that certain climate change hazards could impose on public infrastructure in Ontario, through accelerated deterioration, increased operating expenses or compromised service levels? CIPI combines climate data with mapped infrastructure data, and applies a derived climate "elasticity" to cost the impact of specific climate hazards. While quantifying the budgetary impacts of climate change on public infrastructure is relatively new, the methodology could support capital funding decisions and adaptation actions by governments now and in the future. | 2022 | Trending Topics |
| Superheavy Launch: The Coming Paradigm Shift for the Space Industry | Ryan Timm | A new generation of superheavy launch vehicles will provide mass and volume capabilities not available in the last 50 years. This paper describes how superheavy launch will transform space vehicle (SV) design and the space industry. No longer mass-constrained, SVs will be build with different materials and processes, and will include capabilities previously inconceivable. SV developers will adapt their facilities, logistics, and support equipment. Cost estimators may want to revisit parametric CERs dominated by weight. | 2022 | Trending Topics |
| Up is Up; Why Expert Knowledge Should Be Favored in a Technologically Driven World | Arlene F. Minkiewicz, William Gbelee | There have been strong advancements in technology and other tools in the data science community such as big data explosion, machine learning, and neural-network algorithms. Many organizations are losing their appreciation for subject matter expert opinion; instead, they are focusing on technological advancements in tools. We will discuss the importance of subject matter experts, the key role that they play in stitching together data and provide some cautionary tales of leaning too much on technology. | 2022 | Trending Topics |
| Cost Estimate Kick-Starters | Daniel Harper | I'd like to present several "Cost Estimate Kick-Starters" I created to "kick-start" your cost estimate, including: LaRRGE [Labor Rate Reference Guide for Estimators] Labor Rates resources including newly added Cyber Labor Rates; GRIPS [GSA Robust Infrastructure Pricing Solutions] The GSA Enterprise Infrastructure Solutions Pricer, or "Turbotax" for Enterprise Infrastructure Solutions; SLiCE [Software License Cost Estimator] lookup tool containing over 4,000 prices (and growing) for software licenses, training, etc. | 2022 | Trending Topics |
| Volunteering: Maximizing the ROI of your ICEAA Membership | Christina N. Snyder, Megan Jones | Joining ICEAA is as easy as clicking a few buttons, but to truly benefit from the value of your membership, you need to get involved. Volunteering is the first step to making your membership work harder for you. This session shows how to maximize the return on investment of ICEAA membership through volunteer opportunities with commitments ranging from short-term to long-term at the local, national, and international levels. | 2022 | Trending Topics |
| Investigating Causal Effects of Software and Systems Engineering Effort | James P. Alstad | Causal discovery can help identify factors that drive project costs. We applied causal discovery to software and systems engineering cost estimation model calibration datasets to determine the causes of effort. Due to few variables resulting from the standard causal discovery algorithms, we came up with a technique called Weak Signal Analysis to better tease out weaker causes. Finally, we estimate the numerical impact of each such causal relationship, resulting in cost estimating equations. | 2021 | Analytical Methods |
| Assessing Regression Methods via Monte Carlo Simulations | Michael Schiavoni | This presentation provides new insights while building on past research into multiplicative-error model regression methods. Prior evaluations have primarily been limited to mathematical arguments or comparisons of sample statistics using a few datasets. Conversely, this effort executes Monte Carlo simulations with thousands of iterations, thus enabling the direct estimation of measures such as true population bias. Furthermore, the capabilities/limitations of each method are explored by simulating different model forms, sample sizes, error distributions, and variances. | 2021 | Analytical Methods |
| The Missing Link: An Evolution of Portfolio Natural Selection | Catherine Dodsworth | "It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change."Charles Darwin. This research presents a conceptual framework and methodology for solving the missing link between learning curve estimates, prediction intervals, and S-curves and generating analytically-based affordable cost constraints to naturally select trade space in a portfolio. This holistic approach expands the evolution of the S-curve. | 2021 | Analytical Methods |
| Reducing Lifecycle Cost Through Aircraft Modernization | Jeremy Goucher | The Department of the Navy is planning to upgrade one of the oldest fighter jets in the inventory. To support a milestone decision, an independent Government cost estimate was developed from an engineering build up including over 1,500 individual material procurement line items. Additionally, dozens of past programs were analyzed to provide parametric inputs to the cost model. The detailed data, methodology, and risk adjusted phased results will be presented. Additionally, certain metrics will be compared to past upgrade programs to determine if there are commonalities across platforms. | 2021 | Analytical Methods |
| Adding Cost Credibility with SRL and MBSE Advanced Tools | Patrick Malone | When space based system development programs are less than 15% spent of estimated costs, traditionally almost 80% of design costs are committed! Furthermore, programs with less than the recommended GAO knowledge at program start will likely have higher risk and unexpected cost/schedule growth. Using advanced tools like System Readiness Level (SRL) and Model Based System Engineering (MBSE) methods we show how to identify vulnerabilities early in the development life cycle to mitigate these risks. | 2021 | Analytical Methods |
| SatSim: Estimating Satellite Costs via Simulation | Daniel Newkirk | Cost Estimating Relationships (CERs) are the standard for traditional cost estimators in estimating satellite designs. As new satellites and architectures are proposed, traditional CERs become less defensible as the design uniqueness increases. In order to better estimate future systems, the Space and Missile Systems Center (SMC) cost research team developed a simulation-based approach to estimate satellite costs with equivalent or better accuracy than traditional CERs. | 2021 | Analytical Methods |
| Foundation of Structured Architecture, System & Cost Modeling | Danny Polidi | Modern software packages can perform complex physics based simulations and alone typically do not consider cost as an input variable while other packages specialize in determining cost. This paper begins the development of a structured System Engineering approach to System design and defines a standardized modular diagram for a RADAR System applied to military applications in the aerospace industry. It will be demonstrated that a system can be defined in standard terms, modeled with modular blocks and be costed by those same modular blocks. | 2021 | Analytical Methods |
| Beyond the Matrix: The Quantitative Cost and Schedule Risk Imperative | Christian B. Smart | The first step in risk management is gaining an appreciation for risk and uncertainty. However, despite a long history of cost and schedule growth in all types of projects, we tend to focus on averages rather than recognize extremes. Project management is blind to risk. The fixation on averages leads to an underestimation of risk. The use of qualitative methods, including risk matrices, is prevalent, but should be analyzed with rigorous quantitative methods. | 2021 | Analytical Methods |
| Improved CERs to Estimate Commercial IaaS Costs for Federal IT Systems | William Gbelee | Previous research established vendor agnostic estimating methods for the following Infrastructure as a Service (IaaS) costs: virtual machine and storage data. New data has led to price updates to the dataset resulting in over 27,000+ world-wide data points. This includes publicly available government data. The statistical analytic approach provides validated vendor agnostic models. The analysis was conducted and verified by randomly separating the samples of data into training and test sets. | 2021 | Cyber Security & Cloud Computing |
| Planning a Cloud Migration Effort: Cost Estimating Considerations | Kyle Connor Ferris | Cloud First vs. Cloud Smart? Private Cloud vs. Public Cloud? Multi-Cloud vs. Hybrid Cloud? With the growing prevalence of cloud solutions and strategies across the Federal Government, developing a defensible cost estimate for the migration of legacy data to cloud environments becomes increasingly complex. Responding to these challenges, this presentation will demystify the fundamentals of cloud computing, and establish important cost estimating considerations when defining the purpose and scope of a cloud migration effort. | 2021 | Cyber Security & Cloud Computing |
| Cybersecurity Cost Issues Facing Today's Cost Analyst | Bob Hunt | There are three general approaches to coat analysis: Botton-up (engineering build up), Economic/Cost Benefit Analysis, and Top-down (parametric). Today Cyber cost analysis is principally done by Botton-up (engineering build up) and/or Economic/Cost Benefit Analysis. This paper will address the advantages and disadvantages of these two approaches. In addition, this oaper will propose a new parametric approach to cyber security cost analysis based on cyber "actors" and business sector. | 2021 | Cyber Security & Cloud Computing |
| Estimating Cloud Infrastructure: Requirements, Methodologies, and Uncertainty | Olivia Lindsey | The DoD is fully committed to the cloud, but where do we start? We'll explore multiple existing models available for estimating cloud infrastructure including those created by NRO CAAG and PRICE Systems, and data sheets available directly through Amazon. We'll discuss technical baseline depth needed and uncertainty considerations for each scenario while exploring economies of scale and pricing differences between security classifications for Government systems and providing ready-to-use factors and reference points for multiple cloud scenarios. | 2021 | Cyber Security & Cloud Computing |
| Comprehending Chaos: Leveraging Text to Improve Analysis | Omar Akbik | Data has become the world's most valuable industrial commodity. The defense industry is no stranger to this reality. The quantitative application of unconventional data sources, such as text, can fill gaps that exist in traditional analysis, and reduce reliance on opinion based inputs that often require significant uncertainty adjustments. This paper will explore natural language processing (NLP) for classification and clustering in cost analysis and how the authors have applied it in practice. | 2021 | Machine Learning & Data Science |
| The Algorithm of 10,000 Variables | Bryan K. Anderson | Machine learning opens the door for software cost and schedule estimation for previously unviable scenarios in traditional frameworks. A reliable software estimation requires several intricate factors and often the data is unattainable or incomplete. By training several machine learning algorithms on actual data from an issue tracking system this case study achieved a more reliable model. Factors like issue description, individuals assigned, and issue events are shown to be important features in modeling development work. | 2021 | Machine Learning & Data Science |
| Innovative Techniques for Analyzing Incomplete Data to Improve Cost Estimates | George O. Bayer Jr. | Cost estimators are often presented with incomplete data sets from which they must develop business case solutions. Understanding, interpreting, and improving the data integrity are critical factors for cost estimate accuracy. In this use case, the cost estimators analyze and interpret incomplete and subjective data sets, forecast spares depletion, and estimate obsolescence. Using innovative data mining and text analysis techniques, the estimators demonstrate how improved data can result in better cost estimates and business cases. | 2021 | Machine Learning & Data Science |
| Applying Natural Language Processing Techniques to Categorize Cost Estimation Data | Cara Cuiule | In cost estimation, datasets used for analysis can include text data that needs to be normalized to common terminologies, such as component categorizations. Natural language processing (NLP) and machine learning (ML) could potentially be used to solve this problem. This paper will define terminology and common techniques for both NLP and ML. It will also include a case study using a hardware dataset and NLP techniques using Python. | 2021 | Machine Learning & Data Science |
| Machine Learning and Parametrics for Software Joint Confidence Level Analysis | Sara Jardine | Joint Confidence Level analysis has proven to be very successful for NASA. It is typically conducted using bottom-up resource-loaded schedules. However, the use of high-level parametrics and machine learning has been successfully used by one of the authors. This approach has some advantages over the more detailed method. In this paper, we discuss the approach and provide an example of the application of machine learning and parametric analysis to software programs. | 2021 | Machine Learning & Data Science |
| Data With A Purpose: Technical Data Initiative | Jeff McDowell | Technical characteristics are known to influence cost and as such are integral to cost estimating methods. Cost-driver data is an enduring cost community need yet is often an afterthought to cost data collection. This presentation is an overview of CADE's Technical Data Report (TDR) to systemically capture this must-needed information as part of the CSDR process. Also a sample Power BI case will illustrate the powerful analysis empowered by integrating TDR data with FlexFile cost data. | 2021 | Machine Learning & Data Science |
| USCM11: an Evolution of Techniques Used to Build Cost Models | Benjamin Kwok | The development of the 11th version of USCM incorporates many innovative processes that paves the way for how future cost estimating models could be developed. This presentation will showcase analysis, processes, and techniques such as automating the generation of CERs, using machine learning techniques to classify CERs as either primary or secondary methods, using an analytics approach to evaluate your model, and more. | 2021 | Machine Learning & Data Science |
| Dealing with Missing Data: The Art and Science of Imputation | Kimberly Roye | Missing data is a common occurrence when collecting and analyzing data. Even when a data set includes many data points, many variables of interest will have omitted values. The most common way to deal with this situation is to exclude the data points from analysis. However, this is not ideal. We discuss a better way to deal with this issue, which is the use of imputation, a statistically rigorous method for filling in the holes. | 2021 | Machine Learning & Data Science |
| Continuous Enhancements: An Alternative to Maintenance | Daniel Bowers | With a rapidly evolving threat and the enemy's ability to adapt and counter our weapons and technology, gone are the days of designing, producing, and maintaining a system for 20-30 years. Modern systems need the ability to adapt and update rapidly to include the latest technology. This new paradigm changes the traditional Operations & Support methodologies and estimating techniques employed by the cost analyst. We will look at the impacts to maintenance costs and the traditional funding profile. | 2021 | Operations & Support Analysis |
| Using Analytics to Aid in Performance Based Logistics Decision Making Processes | Paul Brown | As OEMs push towards high dollar value, long-term Performance Based Logistics (PBL) contracts, decisions surrounding these types of contracts should be driven by an analytical process. PBL contracts can be worth billions of dollars and are sold as ways to improve system performance while reducing costs. Modeling and simulation can be used to evaluate the performance and cost impacts of the PBL-suggested improvements and assist decisions makers select an appropriate way forward. | 2021 | Operations & Support Analysis |
| Army Software Sustainment: Righting Our Cost Estimating Assumptions | Cheryl L. Jones | A challenge in estimating software Total Ownership Costs at the beginning of a software acquisition is estimating the cost of software after development. Rules-of-thumb are often used to estimate the annual change in the software during the sustainment lifecycle. This session will present data analysis on software sustainment profiles that show a rise and fall in the number of changes over long periods of time contrary to the practice of using a constant annual change. | 2021 | Operations & Support Analysis |
| Practical Estimating of Hardware Lifecycle Maintenance and Obsolescence Mitigation Strategies | F. Gurney Thompson III | There are many strategies that can be employed to maintain a hardware system, and many phenomena that have been observed in systems with long lifespans which can greatly affect cost and reliability. This paper will present our research approach and will examine case studies to compare various maintenance strategies, approaches to mitigating technology obsolescence issues, modeling of non-constant failure rates, effects of mid-life upgrades, and more. | 2021 | Operations & Support Analysis |
| Business Transformation of Life Cycle Cost Estimating at PEO STRI / Processes | James A. Golden | The U.S. Army's PEO STRI used the tenets of Better Buying Power and Continuous Process Improvement (CPI) to improve life cycle cost estimating, focusing on the processes and applications that produce credible data-driven, comprehensive, and reliable cost estimates. Implementation resulted in earlier and faster production of estimates; increased confidence in program estimates; and improved organizational ability to defend estimates. This presentation describes the challenges, processes, and internal controls associated with success of PEO STRI initiatives. | 2021 | Processes & Best Practices |
| Addressing Challenges in Costing Unique Large Scientific Facilities | Marc Hayhurst | The costing of large scientific facilities poses many challenges to cost estimators as these facilities are typically uniquely designed for specific types of research. However, when the facilities and hardware are examined at a lower level, there may be similar components which can be consulted to evaluate a new proposed system. Aerospace will share their experience in costing unique facilities including the importance of historical data, cost scaling, sensitivity analysis, and other evaluation techniques. | 2021 | Processes & Best Practices |
| Effective Affordability Engineering Teams Top 5 Things You Need to Know | Zach Jasnoff | Much is written on Affordability Analysis, but what does it take to implement an effective Affordability Engineering team within a DOD contractor? This presentation explores challenges / pitfalls facing establishing an effective team and the top five things needed to overcome them. Also discussed are the methodologies, technologies and best practices required to fully explore the cost / performance trade space. A Model Based Cost Engineering case study demonstrating Affordability Analysis principles will also be presented. | 2021 | Processes & Best Practices |
| Cybersecurity Cost Estimating Factors for Business IT Systems | Richard Mabe | PRICE® Systems and MITRE analysts developed an initial set of cybersecurity cost estimating factors for the 2020 ICEAA Workshop. Peer feedback on the factors identified several issues requiring additional research to resolve. This a paper presents the follow-on data updates and analysis to address these issues, and to update the factors accordingly. The results can be applied to estimate costs for cybersecurity specific activities supporting development and operations of IT systems for the federal government. | 2021 | Processes & Best Practices |
| Where To Miss, What To Give Up | Doug K Howarth | Many projects miss some or all of their targets and specifications. Unrealistic requirements force designs made with skyrocketing costs. To produce a successful product, one must know which conditions to keep and which to drop. Producers should discover what the market wants, doesn't have, and can afford. In-depth market analysis reveals configurations more likely to fall within cost limits while satisfying customer requirements. | 2021 | Processes & Best Practices |
| The Art of the Interview | Joe Bauer | Cost estimation is often called an art and a science. From an art perspective, the estimator must be able to effectively communicate with people to elicit information and deliver actionable intelligence. In this presentation, we will discuss interview techniques and other 'soft skills' that enable the estimator to effectively obtain useful information, build rapport and collaborate with a number of different audiences, to include subject matter experts, engineers, program managers, customers, and stakeholders. | 2021 | Processes & Best Practices |
| Using Integrated Program Management (IPM) Principles Effectively | Mike Thompson | Integrated Program Management(IPM) is not just managing the Technical aspects for the program and looking to achieve the program goals. Besides the technical piece, there is an entire team effort, which includes Cost Estimates and Analysis, Scheduling, and Earned Value Management, which work together to present a total picture of the program, based on separate data. In the past there has been a tendency to 'silo' these disciplines and not identify or use the them in an integrated way. The purpose of this paper is to illustrate how the use of IPM Principles. | 2021 | Processes & Best Practices |
| Forecasting of Agile deliveries | Eric van der Vliet | Agile is the most common delivery approach for software projects. Single teams are able to control their delivery, more challenging is the delivery of multiple teams or multiple release trains. How can management keep control in such a situation? How can management determine what value has been delivered? This presentation is about a Power BI based solution that consolidates the information from different teams and release trains and consolidates this information in trends for management to take decisions with respect to performance, quality and value. | 2021 | Processes & Best Practices |
| Does Cost Team Leadership Matter | Christina N. Snyder | An anonymous survey of 150+ cost analysts unanimously reported that a cost-team lead's effectiveness ultimately impacts the team's products. However, there has been minimal guidance as to what defines good leadership. Using the ten behaviors identified by Google's Project Oxygen, this paper seeks to understand what skills are necessary for successful cost leadership. The findings lead to a simple conclusion that mirrors that of Project Oxygen: improving our soft skills will improve cost leader efficacy. | 2021 | Soft Skills & Communication |
| Faster! Better! Cheaper! Improving Counting Productivity and Delivery | Sheila D. Dennis | Do you want to get the best ROI from your function point team? From a business perspective, there are five core goals for any program: 1-Effectively manage workflow; 2-Proactively manage end user expectations; 3-Accurately plan, budget and forecast deliveries; 4-Accurately estimate deliverables; and 5-Show value to the organization. A function point team should also strive for these goals. Come learn best practices, tips and techniques for building a valuable counting support unit for your organization. | 2021 | Soft Skills & Communication |
| Estimator development and process automation | Steven Glogoza | As technology, software, and tools continue to automate, it becomes increasingly challenging to develop personnel. The design and modeling of automated processes may leverage artificial intelligence, but the cost estimator will have to continue to navigate a murky gray environment that requires critical thinking, intellectual curiosity, and analytical aptitude. As we automate estimating activities and continue to develop future estimators, how do we design to capture process velocity without sacrificing insight and credibility? | 2021 | Soft Skills & Communication |
| Finding the Story in Your Data | Karen Richey Mislick | This presentation will cover data visualization and how to find the story within your data. It discusses how people process information and offers tips for creating effective graphics using data visualization principles and techniques to inform decision-making. Different approaches to visualizing data will be discussed including decluttering your graphics, choosing informative visuals, focusing the audience's attention using pre-attentive attributes, thinking like a designer, and implementing effective storytelling techniques. | 2021 | Soft Skills & Communication |
| Icebergs or Shifting Sands: What's the Key to Software Estimation? | Carol Dekkers | ICEAA is a world leader when it comes to estimating tangible programs. With CEBoK, successful estimation of large and small scale programs involving hardware, satellites, buildings, equipment and networks is not only possible, it's become a science. Now, what happens when you bring software-intensive systems into the mix? It's said that estimating software development is like estimating an iceberg, and with the new agile development, maybe even like shifting desert sands. What do you think? Can we create successful software estimates with so much uncertainty? Let's explore the topic. | 2021 | Software & Agile |
| Build It and They Will Come: Keys to Developing a Successful Software Metrics Program | Daniel B. French | Despite the pervasiveness of software development in all areas of industry today, few organizations develop and implement effective software measurement programs to manage their software development projects and leverage information necessary to make business decisions. This presentation discusses reasons why this is, addresses myths and misconceptions regarding software metrics programs, and how to develop an effective program, as well as which software metrics to develop to provide the greatest value. | 2021 | Software & Agile |
| Get to the Point. What's the Deal with Different Function Points Methodologies? | Anandi Hira | Since the development of IFPUG Function Points (FPs), many variants have emerged to simplify software sizing and improve cost estimation. Which of these sizing techniques should one use? To understand differences among methods, we compare IFPUG FPs, COSMIC FPs, and Simple FPs for effort estimation on University of Southern California (USC)'s dataset of Unified Code Count (UCC) enhancements. Additionally, we investigate automated tools to understand the correlation between Objective FPs and the other FPs methods. | 2021 | Software & Agile |
| Air Force: Standing Up Agile Under A Waterfall | Trevor Michelson | On November 2018 the 'Agile Development Manifesto' was released to parts of the Air Force to provide summary guidance to rapidly deliver software products under Agile Development. Multiple cost teams have worked closely throughout the divisions the past few years to better transition how divisions will work to incorporate Agile practices and useful metrics for software development through the release increments. This presentation will provide a journey through the usefulness of our current Agile metrics, how they will be improved and how we have made use of them throughout the DoD. | 2021 | Software & Agile |
| Cyber Mission Platform (CMP) Program: Analyzing the Full Suite of Agile Metrics | Alex Smith | Guidance, policies and task force recommendations related to agile software development best practices for DoD continue to surface on a daily basis. The DSB, DIB, GAO and OUSD A&S have all weighed in during the last two years. The OUSD A&S 'Agile Metrics Guide' provides a concise discussion of metrics that cost analysts and PMs supporting agile programs should consider. This presentation will discuss the AF Cyber Mission Platform program's agile transformation, approach to estimation, and provide techniques and visualizations for analyzing the key metrics discussed in the OUSD A&S guide. | 2021 | Software & Agile |
| Lessons Learned from Software Maintenance Datasets | Arlene Minkiewicz | Effort applied to the maintenance of software applications is thought to be between 65% to 85% of the total ownership costs of the application. Yet there are significantly fewer studies focused on maintenance as compared to development, especially based on public domain datasets. This paper presents results from a study focused on maintenance effort data from the ISBSG's Maintenance and Support (ISBSG M&S) database along with data from other sources. | 2021 | Software & Agile |
| Secure Software Development Levels and Costs | Elaine Venson | While the growing field of Software Security provides solutions to address security vulnerabilities, financial issues are still a barrier to their introduction in projects. Evaluating the cost-effectiveness of security practices requires understanding the impact of increasing degrees of security on the development effort. This session will present a rating scale for establishing levels of secure software development, along with effort estimates provided by experts, and the steps envisioned to propose a security cost model. | 2021 | Software & Agile |
| Grandma's Secret Hotdish Recipe for SW Planning: SAFe and Analytics | Leah Walker | In today's environment, programs want to develop and deliver quickly, but struggle to manage to cost and schedule baselines. Changes in scope, staffing and process require non-standard analysis of data amidst constant change. The authors employed innovative approaches, leveraging function point analysis and machine learning tools, to generate estimates and metrics to drive business outcomes. This business case explores the impacts of program ownership transition, conversion of software development methodology, and modified scope and requirements management. | 2021 | Software & Agile |
| How Green Was My Labor: The Cost Impacts of Manufacturing Personnel Changes | Brent M. Johnstone | Estimators are frequently confronted with manpower increases or decreases and asked to calculate shop performance impacts. However, existing learning curve literature offers little guidance how to do so. This paper identifies issues associated with both new hires (so-called 'green' labor) and workforce reductions and offers an analytical format. Based on a study of a large workforce expansion on a mature aircraft program, a model to analyze future manpower changes is presented as well as several example cases. | 2021 | Technical Management |
| A Perspective on Cost and Schedule Impacts of Non-Technical Project Management | Terry Josserand | Project and program management have distinct definitions, but the interplay between technical and non-technical roles within these fields has limited research. This observational study evaluates non-technical project management and its impact on the cost and schedule of major nuclear weapon modernization programs by utilizing historic data to comprehend these emerging phenomena. The study results yield multiple observations and recommendations that will assist senior leadership in their review and decision-making regarding future acquisitions. | 2021 | Technical Management |
| Lessons Learned from Implementing Global Project Control | Dale Shermon | This paper examines the challenges for organisations to initiate and maintain excellence in Project Controls working with the multi-disciplinary fields of scheduling, cost estimating, risk management, EVM, reporting and monitoring. With more than fifty global stakeholder and 150+ deliverables we will explore the highs and lows of a Project Controls change programme and the lessons learned. | 2021 | Technical Management |
| Cost and Data Issues facing Today's Cybersecurity Analysts | Bob Hunt | Cybersecurity threats and counter measures are evolving at a rapid pace. Analysts need a solution for assessing a non-static situation. Many organizations are simply costing a proposed plan without a strategic or sustainment plan. Current studies imply that it is more costly to defend against a cybersecurity attack than to execute the attack. Studies also indicate that the typical security breach is not detected until about 200 days after the breach. Analysts need to understand the scope of cybersecurity. Is it physical, computer, hardware, people, policy, or all of the above? | 2020 | 2020 SCAF/ICEAA Virtual Conference |
| The Art of Judgement | Andy Nolan | Many of us rely on "expert judgement" when estimating, but the evidence is that it can be unreliable. Based on a study of 3760 guesses, we noticed that 70% of people tend to underestimate. A new study in 2020 of 7400 guesses, showed that 69% of people tend to quote a narrow min-max range when guessing a 3-point estimate. When using judgement, most of us are too low and too precise, we are precisely wrong! | 2020 | 2020 SCAF/ICEAA Virtual Conference |
| The Quantitative Risk Management Imperative | Christian B. Smart | Risk is an important consideration for all projects. However, as a society we are risk blind. When projects plan, they tend to look only at the best-case scenario. Risk is not just a nice to have, it is a critical ingredient in project success, and it needs to be analyzed quantitatively. The use of point estimates, averages, and qualitative methods all underestimate risk. The use of quantitative methods for risk analysis is well established. We discuss how to successfully implement them in the analysis of project risk. | 2020 | 2020 SCAF/ICEAA Virtual Conference |
| The Future of IT & Software Estimating | Carol Dekkers | Wouldn't it be fun to have a crystal ball to predict the future of IT and software estimating? In this presentation, Carol Dekkers takes us on a roller coaster ride of prognostications based on where we've been, what's going well today and what trends are on the horizon. Let's have some fun connecting the dots between the world today and where we might end up in the future with ICEAA and IT and software estimating. | 2020 | 2020 SCAF/ICEAA Virtual Conference |
| An Ontology-based Cost Modelling Approach for High-Value Manufacturing | Maryam Farsi | In high-value manufacturing, the inadequacy of historical service data, and the high level of uncertainties around service cost in complex assets, make the identification of cost reduction opportunities challenging. These influencing factors also complicate the assessment of the impact of cost drivers on the total lifecycle cost. Addressing these challenges, this workshop presents an ontology-based cost model architecture with a simple data structure and minimum data requirement. Additionally, the impact of cost drivers on the total cost is evaluated and aims to examine the validity and applicability of the ontology and the cost model to different sectors and asset types. | 2020 | 2020 SCAF/ICEAA Virtual Conference |
| Empirical Effort and Schedule Models for Agile Development in the US DoD | Wilson Rosa | In the Department of Defense (DoD), mainstream agile estimation metrics (story points and user stories) are not available before contract award. This paper presents an effective approach for estimating Agile software effort and schedule using a size measure available before contract award. The analysis explores the effects of initial software requirements (functions and external interfaces), staffing, and domain on effort and schedule. The dataset contained 36 DoD agile software projects implemented from 2008 to 2019. | 2020 | 2020 SCAF/ICEAA Virtual Conference |
| Splitting Water: A Cost-Benefit Analysis on the installation of an Electrolyser at a Hospital | Simon Porter | Following the governmental commitment of achieving net carbon zero by 2050, interest in the development of the hydrogen economy has increased. Hydrogen can be produced using electricity to split distilled water into hydrogen and oxygen gases, both of which can then be used for other purposes. This study analyses the costs and anticipated benefits of installing an on-site electrolyser at a hospital. This would provide hydrogen to run a Combined Heat & Power (CHP) plant to supply electricity and heat, along with oxygen for medical usage. A number of different options for meeting the brief are discussed, along with anticipation of future trends. | 2020 | 2020 SCAF/ICEAA Virtual Conference |
| How to Make Your Point Estimate Look Like a Cost-Risk Analysis | Cara Cuiule, Amanda Ferraro, Daniel Harper, Richard Mabe | Establishing federal budgets for cloud infrastructure costs prior to selecting a cloud provider requires vendor agnostic cost estimating methods. These methods need to reflect the correlation between rates for a variety of infrastructure instances across all viable cloud service providers. This paper describes research and validation leading to CERs based on over 28,000 virtual machine and storage instances. The predictive analytic approaches presented in this paper can provide valid and verifiable vendor agnostic estimates. | 2020 | Distance Learning Series |
| Storytelling for Cost Estimators | Christina Snyder | As estimators, we advocate the importance of good data; but without context, estimates and analyses are just numbers. To give power to our work, we need to effectively pair good estimating with good communication. There is no existing best practice guidance for estimators on how to create a compelling narrative to accompany analysis. By leveraging a storytelling structure, we can inspire action, communicate our findings in a way that resonates, and ultimately become more effective. | 2020 | Distance Learning Series |
| The Fact That Your Project is Agile is Not (Necessarily) a Cost Driver | Arlene Minkiewicz | All true agile projects follow the same philosophy, they do not all apply the same set of practices, tools or processes. Agile projects are value driven, thus subject to change. There are, however, business and contractual requirements for up front estimates creating a conundrum. This paper discusses a methodology and rules of thumb for estimating agile projects, based analysis of publicly available datasets, that provides value to stakeholders and aligns with Earned Value Management requirements. | 2020 | Distance Learning Series |
| Are you Smarter than an Algorithm? | Andy Prince | Cost analysts rely on mathematical algorithms, experience, and subjective assessments to develop cost estimates. However, these analysts often disagree over what is more important: statistically derived algorithms; or experience and judgment. To try to answer this question cost estimating professionals were surveyed for their expert judgment on the complexity and new design values for 15 NASA science missions. The results may or may not be surprising, but will surely be interesting. | 2020 | Distance Learning Series |
| A 3 Market, 10 Dimension Trade | Doug K. Howarth | Any person, company, or government working across three or more related markets decides how to divide the costs between them. Often decision makers give little thought as to how those resource splits need to work when working in conjunction to a common goal. Using the example of the Prompt Global Strike (PGS) initiative, this paper studies ways to optimize costs in three connected markets (air-to-surface missiles, bombers, tanker aircraft) across ten dimensions. | 2020 | Distance Learning Series |
| Lessons Learned Implementing EVM on Government-led Delivery Efforts | Joshua Teitelbaum | Implementing Earned Value Management on projects where a Government entity serves as the Lead Systems Integrator presents unique challenges and opportunities when compared to typical EVM applications on industry vendor contracts. This paper will cover lessons learned and best practices for implementing EVM on Government-led integration projects based on field experience from a team that has helped the Government with several such efforts. This will include a description of the methods and tools the team used to baseline projects, gather data from performers, and report status to stakeholders. | 2020 | Distance Learning Series |
| The costverse for the FlexFile: Enabling Powerful Analysis in R | Benjamin Berkman, Justin Cooper | The Cost and Hour Report ('FlexFile') is a new Contractor Cost Data Reporting (CCDR) format that promises to change the world of Department of Defense (DoD) cost analysis by delivering significantly more granular cost and hour data than its predecessor, the DD 1921 series of reports. The volume of the FlexFile requires a more thoughtful approach to importing, wrangling, transforming, and ultimately communicating data than Microsoft Excel (Excel) may offer. This paper introduces three R packages that help the analyst exploit the FlexFile to its fullest extent. | 2020 | Distance Learning Series |
| Advanced Data Analytics for Maintenance & Repair Reporting | Paul Hardin, Alexander LoRusso, Tyler Staffin | The 1921-M/R (Maintenance & Repair Parts Data Report) is the DoD system for collecting actual maintenance event and repair part data in the Cost and Software Data Reporting (CSDR) system. This paper will employ the R Shiny package, which is used for the construction of interactive web applications, to demonstrate the analytical value of -M/R data. Additionally, this paper will explore the mechanics of the R Shiny framework within the environment of advanced data analytics. | 2020 | Distance Learning Series |
| Improving Software Estimating Relations for Army Software Sustainment Data | Cheryl Jones, Bradford K. Clark, James Doswell | New approaches were employed to improve Army software sustainment cost estimation: causal analysis and annualization of release data. Causal analysis examines the cause/effect relationships between factors that indicate which CERs should be derived. Converting multi-year data to annualized values has improved CERs. This presentation shows what was discovered using causal analysis and the resulting improved CERs. | 2020 | Distance Learning Series |
| Assuring Credibility in the Cost Estimate: Part II | Hank Apgar | This presentation updates the original, presented at the 2016 ICEAA International Workshop (Bristol), which traced the maturation of cost estimating attributes and focused on cost credibility. Evidence is provided in the words of government and industry executives, estimating and engineering handbooks, professional journals, and government auditing manuals. This update incorporates the impact of popular cost drivers such as system maturity and cost growth. This presentation concludes with guidance for the estimating professional. | 2020 | Distance Learning Series |
| Costing Out an Air Force Software Factory | Stephanie Quintal, Caitlin Burke, Kristen Marquette | The novel concept of standing up a software factory has left even the most seasoned cost estimator scrambling for guidance. Presented by the Kessel Run cost team, this presentation will provide insights on staffing, physical locale and other hidden stand-up costs. We will discuss real world actuals on team sizing, skill mix, and phasing methods as well as labor rate analysis and acquisition support. Lastly, a template for a generic software factory will be provided. | 2020 | Distance Learning Series |
| Diversity in Software Estimation Approaches: Perceptions to Preferences | Shashank Patil, Ria Bakhtiani | For strong foundation of Software Estimates, diversity in analytical abilities & preferences of winning teams are the key factors those must be effectively addressed. In this paper we talk about how perceptions and preferences play an important role at very early stages of a bid / pursuits lifecycle. This is an attempt to study and present various preferred approaches adopted by winning teams and their usefulness on critical parameters of Software Estimations; viz: Turnaround Time, Accuracy, Repeatability and Reproducibility. The views in this paper are based on our internal surveys carried out with SMEs and Engagement Managers at various hierarchical levels. | 2020 | Distance Learning Series |
| Navigating the Minefield: Estimating Before Requirements are Complete | Carol Dekkers | While cost estimation is challenging in hardware and manufacturing projects, software cost estimators face unprecedented obstacles when they are asked to estimate software development projects. Not only are these projects rife with uncertainty, requirements often shift, technologies change direction, and customers shift priorities and that's when they are based on solid software requirements! But, what happens when you need to create an estimate before requirements are complete? Learn how software cost estimators in leading organizations are responding to an ever-changing software development landscape and delivering estimates, that enable project success. | 2020 | Distance Learning Series |
| Software Estimating: Is the Problem Solved? Some Myths and Facts | Sanathanan Rajagopal | Software Estimating is often seen as one of the most difficult domain and an extremely difficult to estimate. However every project (S) now have software and dependency and complexity of which is growing exponentially. In this paper author will explore why people thinks why software estimating is a big issue and will try to break some myths and explore some facts about software estimating. | 2020 | Distance Learning Series |
| Is this Schedule Credible? | Jonathan Shriqui | A schedule is the life and breathe of any project. As such, it should be constructed with rigor and discipline. But how does one differentiate a credible schedule from a poorly constructed one without being a SME? The US Defense Contract Management Agency (DCMA) has created a scheduling test to objectively determine a schedule's integrity. In this presentation we will review the key concepts, pros & cons and limitation of this scheduling test. Please join me and come find out if your project schedules are transparent or is there more to them than meets the eye! | 2020 | Distance Learning Series |
| Advanced Estimating Methodologies for Conceptual Stage Development | Chuck Alexander | This research paper presents statistical techniques and cost analysis that significantly enhance legacy technology development estimating methodologies. Techniques leveraging independent variables that reflect a comprehensive set of cost drivers relating to technology scale, complexity, type, maturity, and development difficulty are presented. Highly tailored solutions including uncertainty are produced that vastly expand and refine earlier development estimating models. General R&D framework relating key milestones, TRLs and cost benchmarks is constructed and woven into an integrated solution. | 2020 | Distance Learning Series |
| Leveraging the Wisdom of Crowds in Estimating Army SW Sustainment | Christian Smart | The use of modern regression and machine learning techniques can improve predictive accuracy compared to traditional log-transformed ordinary least squares, as well as resolving issues with bias and transformation. The combination of multiple models in an ensemble and cross-validation can further increase accuracy. These techniques are discussed in detail and are applied to an extensive set of software sustainment data for 192 Army systems. Results include models based on release type, software changes, and categories. | 2020 | Data Management & Machine Learning Category Winner |
| Augustines Law: Are We Really Headed for the $800 Billion-Dollar Fighter? | Brent M. Johnstone | Augustines Law famously proposed fighter aircraft costs are growing so rapidly that by 2054 buying a single tactical aircraft will consume the entire defense budget. Is the situation really so dire? This paper examines the trend in U.S. fighter costs and relates them to generational changes in aircraft design and manufacture. It also examines the new jet fighters of the 2000s to see if Augustines Law is really unfolding as its author originally thought. | 2020 | Modeling Category Winner and Best Paper Overall |
| 13 Reasons a Cost Estimate During a Concurrent Engineering Study Could Go Wrong | Andy Braukhane | During early phase spacecraft design, the concurrent engineering (CE) approach is proven to be very efficient. But the condensed and iterative nature of CE sessions can also make life hard for a cost estimator. This work discusses 13 problem areas experienced or observed mainly during one-week, inter-disciplinary space system design studies and provides practical examples on how to tackle them, e.g. how to handle rapid data changes, wrong expectations and a diverse engineering team. | 2020 | Processes & Best Practices Category Winner |
| But Wait, Theres More! Using SFPA for Your Cost, Schedule & Performance Needs | Katharine Mann | Do you need to estimate software size? Do you want to add value to your program beyond the LCCE? Simple Function Point Analysis (SFPA) can help! We discuss how analysts can engage Program Managers to use SFPA not just for cost estimating, but for scheduling and progress tracking of software development programs. Real DHS Programs and Policies are used to illustrate the benefits of Simple Function Points to the entire organization. | 2020 | Software & Agile Category Winner |
| In Search of the Production Steady State: Mission Impossible? | Patrick McCarthy | Learning Curves are a vital tool for cost estimators when predicting the number of direct labor hours required for a production run. One challenge of utilizing learning curves is predicting when no additional improvement can be expected, otherwise known as the steady state of the production run. This paper addresses different variables to consider when analyzing data to determine when improvement is likely to cease and the steady state of the production run will commence. | 2020 | Analytical Methods & Strategies Category Winner |
| What's the Big Deal? Is Agile Software Development Really Different in the DoD Acquisition Environment? | Katelyn Barbre | Increasingly more projects are leaning towards the use of Agile as a way of mitigating the cost, schedule, and performance of complicated development efforts. Due to complicated acquisition requirements and environment of the DoD, the advantages of Agile versus non-Agile projects are blurred. This research will compare and contrast cost and technical data from multiple DoD acquisition programs to better understand the impact of Agile software development in the DoD. | 2019 | Agile |
| Scrum Agile Software Metric Analysis for AF Information Systems | Kyle Davis, Elizabeth Ashwood, Alex Smith, William Lane | Programs transitioning from waterfall to agile software development are driving change to the cost analysis process and the way we develop metrics to predict future performance. This case study discusses AF C2ISR portfolio research in a portfolio of information system projects following the Scaled Agile Framework (SAFe). Continuous development and requirements flow introduces additional challenges. This research will discuss the teams evolving approach, study of productivity metrics over time and lessons learned generated from the initial research. | 2019 | Agile |
| Agile Management for Rapid Acquisition | Maureen Deane | According to PARCA in the April 2018 Agile Program Managers Guidebook, "Agile philosophies promote rapid incremental product deliveries, provide flexibility to respond to changing requirements, and advocate close customer collaboration." While Rapid Acquisition programs may receive reprieve from the procedural requirements defined by the DOD 5000.01, program managers and the cost estimating community maintain their responsibility to manage programs effectively. This paper presents how Agile program management best practices can be applied to Rapid Acquisition. | 2019 | Agile |
| Understanding Federal Sector Agile Productivity: A Benchmark Study | Kevin McKeel, Sheila Dennis | Software estimation in the federal sector is challenging, relying heavily on team productivity. We will present the results of an Agile productivity bench marking study (2017-2018) that was performed across a variety of diverse federal projects from multiple agencies. In addition to quantitative metrics, we captured the effects of federal acquisition constraints on how Agile is implemented, drivers of successful projects, Agile estimation best practices, and other key findings that were part of the analyses and the results. | 2019 | Agile |
| Pitfalls to Avoid in Agile Fsm Productivity Measurements | Roopali Thapar, Carol Dekkers | Using Functional Sizing on Agile projects requires integration of delivery process with measurement process. The gaps, if any, can take away the whole purpose of measurement. Few key things should be kept in mind when doing functional point counting (FP) for sprints or release which are described as part of this paper. It would also provide some recommendations on how to do productivity benchmarking for same. | 2019 | Agile |
| Agile Estimation Challenges When Starting a New Team for a New Product | Eric van der Vliet | This presentation tells the story of an experienced Cost Estimator with an SPC4 certification that starts with a new team without sufficient Agile experience the development of a complete new product. The budget of the program is fixed and has a duration of two years. Because of the innovative character of the product is it impossible to start with a complete backlog although the program must be managed and controlled and of course estimated... | 2019 | Agile |
| Dont Just Use Your Data ... Exploit It! | Adam James, Jeff Cherwonik, Brandon Bryant | The Data Age is here. Data are being collected at an exponential rate and cost analysts are struggling to exploit it. Limited structured contextual information and clunky formatting are significant barriers to efficient use. This paper demonstrates the importance of establishing a modern data strategy that considers how analysts identify, access and use data in the context of an example exploitation of a large CSDR data set for the Armys Stryker Family of Vehicles. | 2019 | Analysis & Modeling |
| Production System Cost Modeling Within an MBSE Environment | Dan Kennedy, Karen Mourikas | Model-based Systems Engineering (MBSE) incorporates digital models to represent system-level physical attributes and operational behavior throughout the system life-cycle to support product development. To date, many MBSE efforts have focused on technical requirements with little emphasis on cost. Integrating cost models into MBSE provides visibility into cost impacts of design decisions. This presentation explores optimizing production-system design, manufacturing processes, and operations, by integrating various internal and industry production-system cost models into an MBSE environment. | 2019 | Analysis & Modeling |
| An Upgrade to Anderlohrs Retrograde Method for Broken Learning | Tommie Miller | Breaks in production cause havoc for managing and estimating program costs. Estimators must consider the loss of efficiencies that will result from the break as well as the expected rate of learning after the break. Anderlohrs Retrograde Method is a popular technique for estimating the cost of broken learning, but it suffers from a weakness that arise when the learning slope changes at the break. This paper describes this weakness and suggests a robust solution. | 2019 | Analysis & Modeling |
| The Beginning of the End of Traditional Analogous 'Bottoms-up' Estimating | Chris Price | Traditional bottom-up estimating methodologies are under fire due to inconsistency and inaccuracy. Leveraging advanced data-driven techniques is key for accurate, rapid cost estimating. Based on a recent endorsement provided by the DCAA in their 2018 Audit Manual, acceptance of alternative estimating methodologies other than 'bottom-up' are now acceptable. We will present a disruptive process for transitioning from a traditional costly, time-consuming analogous 'bottom-up' Basis of Estimate (BOE) bidding process to a new, more cost-effective process | 2019 | Analysis & Modeling |
| Adaptive Curve Fitting: An Algorithm in a Sea of Models | Michael Schiavoni | Adaptive Curve Fitting (ACF) is a novel technique that analyzes finite time series data such as monthly expenditures for government acquisition contracts. By intelligently fitting known resource phasing curve forms to an existing sequence of data, it generates a custom model that extrapolates the remaining values. This enables a variety of joint cost, schedule, and phasing analyses. ACF is based in theory and empirical research, shares similarities with existing techniques, and introduces several key innovations. | 2019 | Analysis & Modeling |
| Hybrid Cost Estimating: The Union of Macro and Micro-Parametrics | Dale Shermon | At the pre-concept phase of a project the nature of the solution is varied and the number of options numerous. As the project moves into the later phases the feasibility studies begin to produce more information and the number of options starts to reduce. In a hybrid cost estimating framework solution it is possible to migrate from a macro to a micro parametric cost model without need for training or skills. | 2019 | Analysis & Modeling |
| Schedule Estimating Relationship Development Using Missile & Radar Datasets | Sara Jardine, Justin Moul, Donald Trapp | This paper addresses the challenges of collecting useful historical schedule data for missiles and radars. The objective was to develop schedule estimating relationships and descriptive statistics for predicting the time between major acquisition milestones based on technical parameters. Major milestones are not always clearly defined and consistent. The difficulties of collecting and analyzing schedule data are highlighted. Various schedule drivers are discussed. The results provide a cross-check on the reasonableness of projected schedules for new programs. | 2019 | Analysis & Modeling |
| Estimating Chinas Defense Expenditures (AM08) | Jack Bianchi | While DoD releases detailed budget data, China releases virtually none. The Center for Strategic and Budgetary Assessments developed a straightforward, adaptable, and strategy-driven program, the Strategic Choices Tool (SCT), which allows users to modify planned U.S. military force structure and modernization spending over the next decade. We are developing an SCT for Chinese defense spending. This presentation addresses cost estimation methodologies for Chinas procurement, personnel, and O&M costs, and preliminary data collection and modeling efforts. | 2019 | Analysis & Modeling |
| The Legacy of Parametric Estimating | Henry Apgar | This paper chronicles the people, methods, and achievements over past centuries leading to todays acceptance of proven parametric methods for credibly predicting future costs of the worlds most significant tools, software, weapons, structures, and processes. One detailed example is how parametric methods were used to effectively estimate the construction cost of medieval European Castles. | 2019 | Communication & Visualization |
| Simplifying the Estimate without Sacrificing Quality | Jeremy Goucher | A typical Plan of Actions and Milestones for an ACAT 1 cost estimate is six months. This includes time to define the scope, define the WBS, research and analyze data, develop the estimate, iterate, brief, iterate again, brief again, and so on. Simplifying the estimate will result in shorter research and analysis periods, easier model revisions, easier quality assurance, and easier to understand presentations. This presentation will discuss various methods and processes for simplifying the estimate without sacrificing quality. | 2019 | Communication & Visualization |
| Data Visualization -a Product of Human Design | Benjamin Kwok | Cost estimates are based on data analysis and statistical methodologies that capture complex behavior in mathematical terms. One of the biggest challenges estimators face is describing their analyses in a way that a decision maker can quickly grasp and apply. Data visualization is an important skill set that analysts can use to present complex results in a clear and compelling manner. This presentation will provide an overview and show the benefits of data visualization. | 2019 | Communication & Visualization |
| The Point Estimate Is Not the Point | Jack Snyder, Joe Bauer | The purpose of cost estimating and analysis is to inform planning decisions and secure sufficient / timely funding to facilitate program execution. Or is it? The authors will explore how tangible and intangible roles of cost estimators can better inform design, acquisition strategy, and planning decisions early. Traceable, repeatable, credible and flexible analytical models help to paint the quantitative picture, enabling decision makers to accurately plan a successful program. | 2019 | Communication & Visualization |
| Clearly Communicating Your IGCE to Decision Makers | Christopher Svehlak | All that work gathering data, crunching numbers and running sensitivity tests for Independent Government Cost Estimates. Not done yet - now comes communicating them. Its more art than science. Leaders need the key details and facts. But they can get hazy amidst probability distributions, parametrics, discounting, inflation and other elements. This presentation uses a real (sanitized) cost estimate, providing tried-and-true examples of getting the message across understandably, concisely, and decision-ready. | 2019 | Communication & Visualization |
| Comparing Cloud Costs Equitably: J-Funded Capability Development | Kevin Buck, John Dubelko, Matthew Griesbach, Anthony Rojas | Government agencies often base cloud investment decisions on prices quoted in published cloud calculators and rate cards that invariably apply different service offering assumptions and cost drivers. The likelihood of making decisions based on incomplete or misunderstood data is high, and misinformed investment decisions can unfortunately be quite costly. MITRE evaluated several predominant commercial and government vendor solutions and developed a suite of vignettes that highlight challenges in comparing price quotes on an apples-to-apples basis. | 2019 | Computing |
| The 11th Commandment: Thou Shalt Migrate to the Cloud | Emily Hagerty, Orly Olbum, Brian Flynn | And God gave unto Moses on Mt. Sanai and to General Mattis in the Pentagon the 11th Commandant: Migrate to the Cloud. This paper presents research that supports mission owners, program offices, and cost components in their quest for the Holy Grail - cost effective and cyber-security compliant migration of legacy systems and data to the cloud. It offers innovative artifacts to support up-front requirements and trade-space analyses and back-end cloud design, build, testing and deployment. | 2019 | Computing |
| Predictive Analytic Estimates of Cloud Costs for Government IT Systems | Richard Mabe, Dan Harper | The White Houses "Cloud Smart" strategy encourages agencies to consider life cycle costs focused on three main "pillars": workforce, procurement, security. IT funding requires two years to approve and allocate a budget; so commercial Cloud rates will not be known as the budget is approved. Cost estimates must therefore capture historical workforce, procurement and security cost trends to estimate future solutions. Predictive analytic estimating approaches can provide effective estimates using calibrated historical data. | 2019 | Computing |
| Measurements in Cyberspace | Arlene Minkiewicz | Addressing cybersecurity and software assurance is becoming increasingly important within the DoD and beyond. With software systems constantly talking amongst themselves and across various networks, concern is growing about the protection of our sensitive data and applications. But one must also consider how much security is too much and when the "right" degree of cybersecurity has been achieved. This paper discusses cybersecurity and presents measurements suitable for accessing the efficacy of mitigation strategies. | 2019 | Computing |
| Cost Analysis Needed for Blockchain Efforts | Harvey Reed | U.S. government offices are exploring blockchain to address key challenges in their missions. The motivation stems from blockchain enabling peer-to-peer information sharing, without a centralized authority, which in turn supports execution of processes which span organizations. Currently, government lacks tools to describe blockchain projects with sufficient consistency to support acquisition and cost analysis. This presentation proposes a blockchain descriptive framework as a first step, with a focus on cost elements and drivers for blockchain projects. | 2019 | Computing |
| Forecasting Future Amazon Web Services Pricing | Hassan Souiri, Andrew Kicinski | The National Reconnaissance Office (NRO) Cost and Acquisition Assessment Group (CAAG) produces independent cost estimates to support decision making, budgeting and trade studies. Cloud service costs procured from Amazon Web Services (AWS) are becoming increasingly scrutinized. A thorough analysis was conducted to collect historical AWS prices and model the downward trend. Autoregressive time series models were fit to storage and compute service prices, resulting in annual price reduction rates to be applied to future estimates. | 2019 | Computing |
| "Big Data" Analytics in Operations Research | Cara Cuiule, Grady Noll | As the world becomes more connected and data-driven, accumulating masses of data is becoming a more common corporate strategy. This paper discusses the current PRICE® Systems data collection methodology within the company that utilizes web scraping/crawling. The team will present both lessons learned and potential shortcomings in the current method of approach, along with plans for future endeavors. | 2019 | Data Collection & Management |
| A Discussion on Data Reliability: Evaluating Qualitative and Quantitative Data | Kevin DeStefano, Faye Kim | Obtaining data is an integral part of any analysis and collecting good data that is accurate and robust can be challenging. The quality and validity of data can strongly influence the final cost position recommendation that is presented to accurately and effectively inform decision makers. This paper will discuss data reliability, how to evaluate qualitative and quantitative data, and how to create and evaluate metrics to track the quality of data to ensure accurate cost estimates. | 2019 | Data Collection & Management |
| How to Create a Cost Estimate Using Data Science and RRRrrr Studio! | Jeremy Eden | ICEAAs effort to expose the community to data science concepts and advantages to cost estimators has been popular. However, you want to know how to apply tools and techniques for creating an estimate NOW. Well prepare to create your very first data science cost estimate. There will be code, there will be data, and they will be for ye to plunder and take as your own because "not all treasure is silver and gold mate." | 2019 | Data Collection & Management |
| When Data isn't Enough | Kellie Scarbrough (Wutzke) | How can an analyst transform messy data into valuable insights? This presentation outlines several techniques in Excel and R to clean and process large volumes of data. In R, well look for important relationships among our data. Well also create a corpus to utilize an otherwise unhelpful text field and explore some text mining techniques. Finally, well explore how Tableau can help us visualize and communicate our findings. | 2019 | Data Collection & Management |
| CDMS: Developing a Database Solution for Data Management | Benjamin Truskin, Brian Wells, Tony Occhiuzzo | The NRO CAAG has a data issue...too much for Microsoft Office products to handle! The detail and variety of data the CAAG has collected for 40 years has forced a revamp of their data management and storage systems. The CAAG Data Management System (CDMS) will be the next generation database solution to store/normalize CAAG data. This presentation gives firsthand accounts of the challenges encountered and how the CAAG overcame and implemented our solution. | 2019 | Data Collection & Management |
| Growing Maintenance Costs: Understanding How Weather Impacts Maintenance. | Bryan Anderson | The in-service maintenance is often a major cost driver. This paper explores an innovative approach to identifying maintenance trends in the Federal Aviation Administration (FAA). Using NLP and other Machine Learning techniques, the FAAs RMLS maintenance text data is correlated with the ITWS weather data. The session goal is to cover one way on how to think about NLP in a cost setting. Followed by implementation with Python and other data science technologies. | 2019 | Machine Learning |
| Machine Learning Assisted Data Extraction and Normalization | Jonathan Brown, Devin Geraghty | Data collection and normalization is a key component of analysis. However, it often requires significant time and effort to properly extract quality normalized data from the raw datasets. Given the recent proliferation of larger datasets a more automated approach to data extraction and normalization is required. This research applies machine learning text classification algorithms and methods to automate historically manual data normalization tasks. | 2019 | Machine Learning |
| The Robot Forecaster--A Continued Study of Artificial Neural Network Application | Nathan Eskue | If your job includes predicting the future then you understand the difficulties of the "unknown", of all the little gremlins (sometimes called risks) that can mess up the most elegant and well thought-out plan. There are data models and statistical methods that can help a great deal, but if the conditions are right, a relatively new tool that is as accessible as it is misunderstood might just be what you need. We will cover the basics of Artificial Neural Networks and I'll share the progress I've made since ICEAA 2018 in putting these digital brains to work. | 2019 | Machine Learning |
| Don't be Scared, Machine Learning is Easy | Mary Johnson, Dakota Shafer | Thus far the field of cost estimating has only explored the pedagogical utilization of machine learning within cost estimating. The novelty of machine learning makes it difficult to find real use cases demonstrating the efficient application of machine learning to the cost estimating process. This paper and accompanying presentation will document a case study exploring the data cleaning and modeling process for a cost estimate done on DoD installations. | 2019 | Machine Learning |
| Machine Learning and Natural Language Processing Applications for Cost Analysis | Karen Mourikas, Jose Lemus, Enrique Serrot | How can machine learning and natural language processing enhance the traditional methods of cost analysis? By analyzing larger datasets more easily than the human brain can, and by automating many of the manual, time-consuming tasks needed to cleanse and prepare the data for analysis. This presentation examines several applications in which we integrated Natural Language Processing using python libraries, with various Machine Learning methods, in particular trees, randomization, and boosting, to improve prediction accuracy. | 2019 | Machine Learning |
| Beyond Regression: Applying Machine Learning to Parametrics | Kimberly Roye, Christian Smart | Cost estimating has relied primarily upon regression analysis for parametric estimating. However, regression analysis is only one of many tools in data science and machine learning, and is a small subset of supervised machine learning methods. In this paper, we look at a variety of methods for predictive analysis for cost estimating, including other supervised methods such as neural networks, deep learning, and regression trees, as well as unsupervised methods and reinforcement learning. | 2019 | Machine Learning |
| Data Impacts on System Readiness and Cost | Paul Brown | Operations and support costs are becoming increasingly in focus as system readiness is addressed. This paper will discuss the impact data has on forecasting readiness and the corresponding effects on cost. Identifying "correct" data and sources, such as CSDRs and OEM data reporting systems, plays an important role in modeling both cost and readiness. Different approaches to improve readiness and their impacts on cost and data needs will be discussed. | 2019 | Methods |
| A New Approach When Cost/Capability Trades Matter Most | Jeffery Cherwonik, Adam James, Richard Bazzy | Consideration of life cycle costs (LCC) early in the acquisition process, specifically prior to Milestone A, is paramount to credible requirements generation and improves the probability of program success. This paper describes a parametric universal wheeled ground vehicle LCC model based on CSDR data and designed for integration into a modeling and simulation environment that produces alternative vehicle designs. The LCC model uses outputs from a requirements-driven preliminary design model to generate informed cost vs. capability/requirements trades. | 2019 | Methods |
| Five Steps for Improving the Accuracy of Rough Order of Magnitude Estimates -Bell V-280 Wing Predictive Cost Analytics Case Study | Zachary Jasnoff, Ross Raburn | According to the GAO Cost Estimating and Assessment Guide, ROMS are "developed when a quick estimate is needed, and few details are available". This leads to challenges with ROMs especially when estimates become "locked in" early, based on the least accurate data. We will discuss five enabling steps that estimators can take to improve the accuracy of ROM estimating. This includes using a predictive analytic data driven methodology for leveraging and extrapolating limited data sets. | 2019 | Methods |
| Intellectual Property Valuation: An Overview | Cynthia Prince | Intellectual property (IP) refers to inventions, literary and artistic works, designs, and symbols, names and images used in commerce. IP is protected by patents, copyright and trademarks, which enable earning of financial benefit from these inventions or creations. While most cost estimators already know that, what do we know about IP valuation? If asked to derive a cost estimate for IP valuation, what approaches could be taken? This overview will answer that question and more. | 2019 | Methods |
| PAC Template - Program Acquisition Cost (PAC) Template | Horace White, Jim Cain, Raymond Garrido | One of the goals of our young and upcoming cost organization was to create a universal cost model template. Fast forward 6 years and the template we developed introduced standardization, a starting point for new analysts, minimized tedious work, and reduced common errors. We would like to share the success story of the cost model template and the lessons learned through the 6-year development process. | 2019 | Methods |
| Self-Organizing Markets And Time | Doug Howarth | Markets demonstrate statistically significant self-organization concerning how they respond to changes in prices and the product features offered to them. The nature of these self-organizing activities changes over time. What works for a market now may not work a few years from now. Being able to characterize market self-organization now and in the future is key to optimizing financial success, which this paper examines. | 2019 | Planning & Strategy |
| Contracting for Agile Projects | Blaze Smallwood | The nature of agile, with its flexible approach to requirements and stress on incorporating feedback from frequent stakeholder engagements, makes contracting for agile projects particularly difficult. This paper proposes a new idea that uses a capability-based incentive structure, which is currently being considered on upcoming Department of Defense contracts. The paper will present the mechanics of implementing this approach, its pros and cons, and a comparison to other approaches recently used on similar projects. | 2019 | Planning & Strategy |
| Enhance Estimator Success & Organization Competitiveness in Supplier Assessment | John Swaren | This presentation reviews the use of an effective methodology to perform Supplier Assessment. Estimator success and organizational competitiveness are enhanced when supplier quotations can be assessed and validated, leading to stronger negotiation positions as well as basis for preferred suppliers. This paper offers a quantitative methodology to benchmark, monitor and evaluate potential suppliers of a customer-specified technology. Once established, a Supplier Assessment capability creates the outcome of timely consistent bid evaluations as well as ceiling prices for negotiation. | 2019 | Planning & Strategy |
| Application of Conjoint Regression to Requirement Analysis | Jacob Walzer | Conjoint analysis provides a simple method of measuring how the tradeoff between different capabilities can affect the perceived value of a product. Normally associated with marketing research, it can also be utilized to help estimate the cost of developmental items based on their expected requirements. This presentation explores the best practices, common pitfalls, and validation techniques associated with conjoint regression, and includes a notional example derived from an application of this technique in a USMC tradeoff analysis study. | 2019 | Planning & Strategy |
| Measuring Portfolio Value for Government Programs and Initiatives | George Bayer Jr., Bryan Anderson | Transformational government initiatives which require major capital investments or acquisitions are complex, difficult to measure, and challenging to articulate to decision-makers. Considering the complex dependencies and implementation risk of individual programs, risk-adjusted cost estimates for portfolios are often overstated. This paper examines how cost estimators and analysts measure portfolio value and generate stakeholder advocacy for major government initiatives, which require multiple acquisitions for implementation and agency policy changes to change behaviors and realize value. | 2019 | Processes & Best Practices |
| 2020 Census Cost Estimation - GAO Audit Lessons Learned | Edward Kobilarcik, Neala Jones | The 2020 Census LCCE is a large, complex estimate with a projected $15 billion total cost. In 2016, the GAO assessed the estimate as only partially meeting any of the GAOs four characteristics of a quality cost estimate. Using the GAOs Cost Estimating and Assessment Guide, the 2020 Census LCCE team used a data-driven analysis to improve the LCCE and its supporting documentation. The result was a drastic improvement in the 2017 GAO assessment. | 2019 | Processes & Best Practices |
| GAO Updated, New, and Coming Best Practice Guides | Jennifer Leotta, Brian Bothwell | This session will go through GAO's updated, new, and forthcoming best practice guides. This includes a review of the best practices described in the cost guide, schedule guide, technology readiness assessment guide, and Agile assessment guide. | 2019 | Processes & Best Practices |
| Contractually Speaking: The Story of DOD Contracts & Potential Consequences | Orly Olbum, Stephanie Lee, Peter Braxton | A contract is the legal document holding government and contractor accountable for their responsibilities and can play a significant role in cost and schedule issues throughout its lifetime. This paper utilizes the Contracts Database and CADE to investigate whether or not contract vehicles themselves contribute to these issues. We also dive into the acquisition process to determine the efficiency of different contract vehicles and how profit may impact a contracts success or failure. | 2019 | Processes & Best Practices |
| Business Case on the Cost/Benefit of U.S. Government Support of Contractors in order to Maintain Competition | Todd Pardoe, John Stedge, Chad Bielawski, Jaimie Smith | The DoD is facing shrinking industrial bases in its acquisitions due to contractor mergers, specific military applications, and scarcity of materials. National security, strategic posturing, and Congressional interest make it vital to keep these industries viable and competitive. Mitigating the risk of "shrinking industrial bases"offers unique opportunities in structuring these acquisitions. This study looks at the costs and benefits associated with strategies for acquisition of defense systems within the constraints of shrinking industrial bases | 2019 | Processes & Best Practices |
| Engineering the Acquisition Process: Better Value Through Mechanism Design | Christian Smart, Britt Staley | As both the regulator and the only buyer in a market that measures over $500 billion a year in acquisitions, the Defense Department has tremendous leverage but does not make full use of it. Limited competition in the prime contractor market results in higher prices and lower quantities purchased. There have been some recent strides made to achieve better value, but much more can be done if the government will pursue strategic approaches. In this paper, we consider the use of mechanism design to achieve better value. | 2019 | Processes & Best Practices |
| Creating a Cost Driver S-Curve | Sandy Burney | S-Curve analysis is not just for Risk Analysis. It can be used for both Contractor and Government Cost Driver analysis. The presentation will: 1) describe the similarities and differences between Cost Driver and Risk Analysis; 2) provide a rational for doing an S-curve on Cost Drivers early in the procurement lifecycle; 3) provide detailed steps to developing a Cost Driver S-Curve; and 4) display examples of Cost Driver S-Curves and additional outputs that result from the analysis. | 2019 | Risk |
| The 3-Point Method Redux: Estimating Cost Uncertainty Given Only a Baseline Cost | Marc Greenberg | Leveraging a 2007 paper entitled "Estimating Cost Uncertainty when only Baseline Cost is Available," this presentation revisits the 3-point method: a scenario-based application of the Analytical Hierarchy Process (AHP). This presentation steps through an overview and notional example on applying the 3-point method to demonstrate how pessimistic and optimistic costs are derived from a given baseline cost. These three costs, in turn, serve as the requisite three parameters of the triangular distribution. | 2019 | Risk |
| When is Less More? Level of Detail in Cost and Schedule Risk Assessment | Laura Krepel, Zachary Pirtle | When performing cost and schedule risk assessment (C)SRA, the level of detail in the analysis schedule dictates the time to build, validate, and review the model. In this brief, we will discuss the pros and cons of using a more or less detailed analysis schedule as the basis for these models, whether 10s or 10,000s of lines. We will explore the level of effort, accuracy, and value (stakeholder understanding) of each approach through the use of real-life scenarios, based upon the authors' experience developing models for NASA and other federal agencies. An overview of the basic requirements for any SRA or JCSA, including detailed schedule for execution, monitoring, and control of the schedule to validate model critical path will also be provided. | 2019 | Risk |
| Risk: Intentional Interaction With Uncertainty | Kai Lemay, Raymond Britt, Jessica Laiti | In this session we review how Risk Management is too often perceived as a compliance exercise rather than a management tool. Because of this outlook, the vast majority of risk registers record too few risks. The limitations and pitfalls of restricting the amount of risks recorded leads to a reliance on point estimates, an incomplete risk mitigation plan, and an absence of tangible impacts and analysis. Its not the big risk that lead your project to failure, its the 30 little risks that occur in a unique sequence of events that spirals into a perfect storm. | 2019 | Risk |
| Impact of Scope Changes on Schedule Growth | Daniel Bowers, Geoffrey Driskell, Gail Flynn, Kelsey Anderson | Can cost analysts predict and thus estimate schedule growth based on scope changes? Is there a relationship between the two? Leveraging from the same research and data as the 2018 ICEAA Workshop paper "Impact of Scope Changes on Software Growth" by Jonathan Brown and Gail Flynn, this research focuses on the possible schedule impacts due to pure growth and scope changes. Analysis was performed from a scheduling perspective to show the impact scope changes can have on the program schedule. | 2019 | Scheduling & Programming |
| The Programmatic Estimating Tool (PET) | William Laing, Erik Burgess | The Programmatic Estimating Tool (PET) provides a method for adjusting cost estimates in scenarios where programs face rigid schedule and/or budget phasing constraints. PET integrates program cost, schedule, and budget phasing into a single user-friendly tool. Using historical correlation between cost, schedule, and phasing model residuals to generate a tri-variate conditional distribution, PET can be used to estimate the impact of: Schedule and/or phasing constraints on cost, Cost and/or phasing constraints on schedule | 2019 | Scheduling & Programming |
| A Program Managers Guide to Reliable Subcontractor Reporting | Patrick Malone, Garth Edwards | Prime Government contracts are executed with significant subcontractor content. Objective measurement of their performance can be difficult leading to methods such as percent spent or level of effort. These methods can result in erroneous progress masking true performance. Using discrete earned value best practices provides prime contractors and Government agencies realistic subcontractor performance for early corrective action. This paper investigates how to implement low risk objective EV techniques to promote reliable and effective subcontractor reporting. | 2019 | Scheduling & Programming |
| Blending Contract and Project Management to Achieve Cost Savings | Teresa Price, Gerald Jones, Wilfred Tagud, Andrew Drennon, Tia M Barnes | This study examines the waterfall that occurs between segregating projects and contract management, and how interdependencies drive project cost savings. This interpretive study will visit the discussion around the pitfalls of segregating these interrelated activities, and how the process can impact your agency bottom line. It will exam how cost avoidance can be achieved and provide recommendations to assist in the minimization of costs, schedule, and risk. | 2019 | Scheduling & Programming |
| EVM Visualization: The Radar Tool | Tyler Staffin, Brian Flynn | This research presents an innovation in EVM data analytics and visualization that gives acquisition managers the potential to better understand system level risk and to laser-focus their attention on critical elements of cost. The tool transforms a massive set of disaggregated data into a manageable form, and extracts "critical" elements by filtering on a set of user-defined, rules-based criteria. The tool holistically displays absolute growth, percent growth, and percent complete in one radar graph. | 2019 | Scheduling & Programming |
| The 3C's of Measurement and Cost Estimating Success: Create, Confirm, Convince | Carol Dekkers, Dan French | To many, it may be obvious that good cost estimating is fundamental to software development"¬. However, most technology organizations remain unaware of the benefits of formal cost estimating and measurement - even when they are technology-centric and use defined project management methods. This presentation promotes a 3C approach to increasing awareness, gaining acceptance and understanding why formal cost estimating is critical to organizational success. | 2019 | Software |
| From Point A to Point Estimate: How Requirements Become Function Points | Dan French, Carol Dekkers | IFPUG function point analysis has been successfully used to size software by organizations for 30+ years. However, there remains confusion as to how the process works. This presentation demystifies the Function Point Analysis process and educates those interested in how function points are counted and/or are considering the use of FP in their organizations. The presenters will discuss functional/non-functional requirements, good/poor requirements and how an FP count is developed based on the functional requirements. | 2019 | Software |
| Estimating and Managing Modern Software Development Programs | Bob Hunt, Dan Galorath, Ian Brown | The recent final report of the Defense Science Board (DSB) Task Force on Acquisition of Software for Defense System (February 2018) outlines key challenges facing Defense with resect to software acquisition. The summary report states, "Software is a crucial and growing part of weapons systems and the Department needs to be able to sustain immortal software indefinitely. " Identified issue range from poor initial estimates to the Federal Governments inability to utilize modern developmental tools, such as the "Software Factory". This paper will discuss thes | 2019 | Software |
| Army Software Sustainment Cost Estimating Results | Cheryl Jones, James Doswell, Brad Clark, Robert Charette, Paul Janusz | The Army has conducted a study over the past six years to improve the estimation accuracy of software sustainment systems cost. Based on an extensive data call of 192 Army systems, data analysis revealed several types of cost estimating relationships based on release type, release rhythm, and three categories of data. Analysis of a sustainment cost risk model was also conducted. This presentation will show the study results including what worked and did not work. | 2019 | Software |
| The Journey to Better ERP Estimation | Jon Kilgore, Jenna Meyers, Arlene Minkiewicz, Cara Cuiule | Software estimation is never easy, especially when implementing large complicated Enterprise Resource Planning (ERP) Systems comprised of significant off-the-shelf (OTS) components combined with glue code interfaces to numerous legacy systems. Data to support such an estimate is sparse, incomplete, and often inconsistent. This paper presents the first phase of a journey toward improvements to this situation, utilizing analysis of historical data to inform future estimates, focusing on process and lessons learned. | 2019 | Software |
| Software Cost Estimation -Why Is It Different? | Harold van Heeringen | Software Cost Estimation is often considered the most difficult part of an integral cost estimate. Unrealistic software estimates often result in disasters. Examples are new tunnels, metro lines or factories that cant start their operation because everything is ready, except for the software part. One of the main reasons for this is often an inaccurate and optimistic software cost (and schedule) estimate. Why is estimating software differently and how is ICEAA dealing with this? | 2019 | Software |
| Adapting Existing Cost Model to Estimate Section 804 Rapid Prototyping and Rapid | Rick Garcia, Jesse Celis | Analogous prototype program cost data doesnt readily exist and the development of prototype specific cost databases is a lengthy process. Our immediate Cost Estimating needs must address estimating prototype satellites using existing cost models. Our evolving process elicits and applies explicit cost model adjustments to existing cost models to directly address the explicit differences between prototypes and operational systems, specifically in the areas of CERs to Model Commercial-Like Programs and %New Design and Heritage. | 2019 | Space & Missiles |
| The Efficacy of NASA's Joint Confidence Level Policy | Andy Prince | In January 2009 NASA policy established the performance of a Joint Cost Schedule Confidence Level (JCL) analysis as the basis for project funding and external commitments. This paper examines the efficacy of that policy relative to reducing cost overruns. Projects executed before the policy implementation are compared to projects approved post-policy implementation. Various statistical techniques are used to perform comparative analyses and alternative explanations are explored. | 2019 | Space & Missiles |
| "Phase 0" Space Mission Estimates | Michel van Pelt | How to prepare ROM estimates for future space missions, within little time and using even less inputs? Based on experience with preparing estimates for the very first Principal Investigator proposals for new ESA Science missions, this paper provides rules-of-thumb, observed high-level cost drivers and trends, checklists, comparisons with later-phase cost estimates, potential pitfalls and general lessons learned. | 2019 | Space & Missiles |
| An Analytical Explanation for Vertical Integration Behavior in the Marketplace | Caleb Williams, Jack Semrau | Vertical integration is a costly and difficult-to-reverse strategy with significant risks, yet it has experienced a sharp rise in popularity within the satellite sector over the last decade. Building on previous research, this session demonstrates how commercial parametric tools can be used in combination with business-case analysis to understand why aerospace companies are increasingly pursuing vertical integration strategies. Additionally, it discusses how the framework for this analysis can be applied across other industries. | 2019 | Space & Missiles |
| Estimating Missile G&C Development Cost: An Important Advance | James York, Paul Hardin III, Jeffery Cherwonik, Olivia Collins | This paper describes improved cost estimating methodologies for tactical missile Guidance and Control (G&C) Development Engineering (DE), generally the largest contributor to missile development cost and typically one of the most challenging cost elements to estimate. These methodologies represent an alternative approach to commonly used cost factors (e.g., factor of prototype production cost) which have error metrics that normally exceed 50%. The research developed CERs with error metrics below 50% and as low as 20%. | 2019 | Space & Missiles |
| A Comparative Analysis of Nuclear Security Enterprise Estimates. | Terry Josserand, Leone Young, Wendy Lee | Various government publications recommended adapting multiple methods and approaches to enhance the quality, validity and reliability of a program cost estimate. This study utilizes a COTS software tool and two approaches based on different system architectures, for a Nuclear Security Enterprise estimating effort. The cost estimate results exhibit a phenomenon of near-homogeneity. Model techniques, critical parameters, and cost drivers are analyzed and evaluated to help comprehend the model behaviors. | 2019 | Space & Missiles |
| Quantifying Annual Affordability Risk of Major Defense Programs | Thomas J. Coonce, David Tate | Using DoD Selected Acquisition Reports (SARs), the authors use a variety of statistical techniques to estimate the distribution of possible funding profiles that might result from a new program, given its initial estimates, program characteristics, and funding climate. The authors then extend this method to address the remaining annual cost risk of partially-completed programs. The results allow analysts to assess the annual confidence levels of a given program. | 2018 | Acquisition & Operations |
| Developing an Independent Government Cost Estimate | Richard Shea, Shavaiz Saood | The Independent Government Cost Estimate (IGCE) is an unbiased cost estimate based upon Government requirements and inputs without input from potential contractors. The IGCE is a tool used by the Government during source selection as the basis for reserving funds for the contract, comparing costs propose by submitting contractors, and used as a guideline to determine contract proposal price reasonableness. This presentation discusses the steps building the IGCE and the next steps in the process. | 2018 | Acquisition & Operations |
| Diversity of Maintenance Logs and Delay | Bryan Kenneth Anderson | Maintenance activities play a leading role during the In-Service Management phase in the FAA's AMS. This paper explores the potential relationships between FAA maintenance activity and cost incurred by the public. The paper objective is to determine if there is a correlation between facility type, maintenance activity incurred by the FAA, and flight delays impacting the public that could drive reclassification of facility type. | 2018 | Acquisition & Operations |
| Portfolio Analysis: Estimating the UK Defence Budget | Dale Shermon | This paper will consider how to build a portfolio analysis of a government budget and, once established, what questions can be considered through this analysis. I will examine the capability gaps and the cost of replacement for obsolete capabilities. As an example the paper will consider the UK MOD budget for current and future expenditures with lessons learnt trying to conduct this ambitious analysis. | 2018 | Acquisition & Operations |
| Estimating Software Sustainment Costs | Arlene Minkiewicz | Software sustainment costs can make up as much of 90% of the total ownership cost of a program, yet the software industry continues to struggle with the best way to predict these costs. Traditional cost drivers used for acquisition estimates don't necessarily apply for sustainment. This paper discusses an on-going research project collecting cost and technical data from evolving software systems in an effort to determine the best sustainment cost drivers and cost estimating relationships. | 2018 | Acquisition & Operations |
| Tabular CARDs: Orderly Data for the Cost Community | Jeff McDowell | Recent CAPE guidance has brought positive exciting change to the Cost Analysis Requirements Description (CARD). This change is a new focus on standardized program data content via workbook templates balanced with reduced-scope narrative content. This paper demystifies the new CARD templates by describing each of the tables emphasizing how they were designed to satisfy enduring and recurring cost estimator needs. Also their relevance to MIL-STD-881D Product Extensions and CSDR Plan Standards will be presented. | 2018 | Acquisition & Operations |
| Exploring How Systems Age (and Fail) and the Impact on O&S Cost Estimates | Patrick McCarthy, Alex Krencicki | Significant time is spent estimating the development and production costs of weapon systems. However, estimating how systems age and fail during the O&S phase, as well as trying to identify optimal sustainment and maintenance strategies can be every bit as challenging and have a significant impact on the estimated lifecycle cost. This paper will address obstacles associated with predicting fleet aging profiles, including dynamic failure distributions and the integration of right-censored data into the estimate. | 2018 | Acquisition & Operations |
| Cost and Competition in U.S. Defense Acquisition | Eric Lofgren | The cost estimator has a major role in determining the price, and therefore value, of major systems acquisition in the Department of Defense. Two primary costing methodologies include 'should cost' and 'will cost' analysis, and are affected by 'must cost' realities. This paper explores the history of these costing methods and places them in a theoretical context, first with respect to the meaning of competition, and second with respect to the nature of cost. | 2018 | Acquisition & Operations |
| Using Army Software Sustainment Cost Estimating Results | Cheryl Jones, James Doswell, Bradford Cark | The Army has completed an initial analysis of software sustainment cost and performance data collected from ~250 Weapons, C4ISR, and ERP systems. The analysis addresses primary resource distributions and cost estimating relationships across multiple functional domains, and establishes a foundation for efficient resource allocation decisions across the Army systems portfolio, and projected policy and process changes. The results, including the detailed statistical analysis, will be made available for use by participants. | 2018 | Acquisition & Operations |
| Using Sustainment CSDR Data | Sandra B. Enser | Operations and support costs can exceed 60% of the total Life Cycle Cost of aerospace and defense systems. This work is increasingly performed by contractors rather than by organic facilities. This paper will demonstrate how the Sustainment Cost and Software Data Report (CSDR) data collected by DCARC can be used to estimate future sustainment costs, support Performance-Based Logistics business case analyses, identify maintenance and repair problems, streamline review of hours and costs in negotiations, and evaluate contractor profits. | 2018 | Acquisition & Operations |
| Potential Impacts of Non-Major Program Data Collection on Cost Estimating | Brandon S. Bryant, Stephanie Myrick | The FY 2017 NDAA requires the armed services to collect cost data on all acquisition programs over $ 100 M. The requirement creates a challenge of balancing data collection standardization and flexibility for programs that have not been required to follow traditional data collection strategies. However, this new requirement is an opportunity to improve DOD cost analysis. This paper provides implementation strategies, how to leverage current data collection tools, and potential impacts on cost estimating. | 2018 | Acquisition & Operations |
| Maturing Cost Estimation in a Rapid Acquisition Environment | Jennifer Manring, Natalie Faucher | DoD is moving toward rapid acquisition approaches to deploy critical capability to the warfighter quickly. Rapid acquisitions have shortened timelines and less definition than traditional acquisitions. These constraints challenge traditional cost estimation practices. Multiple case studies were conducted to understand the challenges to cost estimating processes in a rapid acquisition environment. Easily implementable recommendations are made to help ensure credible and confident cost estimates are developed, even within the constraints of these challenging environments. | 2018 | Acquisition & Operations |
| The Concepts of Size and Productivity in an Agile World | Harold van Heeringen | Many organizations have moved to agile development software development. One of the cornerstones of one of the main Agile methods, Scrum, is to de-professionalize estimation and to use an arbitrary effort unit of measurement, the story point, to estimate projects. Because of a lack of understanding of the matter, management in these organizations believe that this is the way to go, but now they are facing huge challenges in estimation and budgeting. it is explained why agile has resulted in lower estimation maturity in the industry, and the ways this can change again using industry standards. | 2018 | Agile |
| Scaled Agile Deliveries: Do We Still Need Estimates? | Eric van der Vliet | Organizations prefer agile software development methodologies because of the high level of agility of requirements and the focus on delivering business value. Self-controlling teams deliver as much as possible value within the available budget. Attention for estimation is reducing what has a direct impact on the control of agile deliveries. This presentation explains what amount of estimation is required in what stage of a scaled agile delivery to provide the right level of delivery control. | 2018 | Agile |
| An Implementation of Automated Structural Design-To-Cost in a Model Based Engineering Environment | Christopher Price, Leonor Hagberg, Brig Bjorn, Apinut "Nate" Sirirojvisuth, Daphne Biddle | The integration of Computer-Aided Design (CAD) tools with Cost Analytic tools in COTS Integration Environments allows designers to quickly and easily estimate the cost of structural designs as they are designing, thereby directly supporting Design-To-Cost, in a Model Based Engineering (MBE) environment. This paper will demonstrate how quick and easy it is to estimate modifications to a structural assembly using such an integration environment. | 2018 | Agile |
| A Perfect Marriage: Mr. and Mrs. Agile-Cloud? | Teresa Price, Wilfred Tagud, John Sullivan, Andrew Drennon | What mandate brought together no one should tear apart. This interpretive study analyzes the relationship between Cloud computing services and Agile Methodology to determine if it's a right fit for achieving cost avoidance. In addition, the study includes experience-based input from the FAA's Chief Information Technology officers as well as other agencies. The research will conclude with the challenges and identified lesson learned when both are implemented together. 'I can't say, 'I Do' without you: ' | 2018 | Agile |
| Accurate Agile Estimation | Kevin McKeel | Estimating software development costs in an Agile development environment is challenging due to undefined scope, lack of planning beyond the next few sprint cycles, and distrust of traditional software estimation techniques within the Agile world. This presentation will detail an IRS case study, and describe how these challenges were overcome. | 2018 | Agile |
| Agile Software Development Cost Risk for Information Technology Programs | Adam Letcher | This paper addresses the issue of how to analyze and quantify cost risk within an Agile software development environment. Because of the unique nature of Agile software development, some of the traditional cost estimating and risk analysis practices should be altered. This paper proposes a process for assessing Agile development risk using a capability sizing metric and presents the governing equations for doing so. | 2018 | Agile |
| Using Function Points to Manage Agile Product Backlog: Fact vs. Fiction | Daniel French, Carol Dekkers | The Agile framework continues to grow in popularity as IT organizations struggle to deliver projects on time and budget. Agile is not a silver bullet that many think it is and the primary reason for this is that Agile estimation methods are completely subjective. This presentation details how IFPUG function points and other rule based size metrics can be effectively used to help Agile teams estimate sprint sizes, calculate velocity and manage the product backlog. | 2018 | Agile |
| Agile Software Development Cost Factors: A Case Study | Blaze Smallwood | The lack of data on government agile software development programs has made estimating costs for new agile development programs challenging. This paper seeks to address this challenge through a case study of several completed DoD agile projects with cost, schedule, and performance data. Several relevant metrics will be examined, including cost per story point, cost per requirement, scope growth rates, impacts of team size changes on velocity and productivity, and various others. | 2018 | Agile |
| Processes of Weapon Systems Acquisition | Eric Lofgren | One major outcome of recent reforms in defense acquisition has been the organizational separation of research and development from production and sustainment. But is the reform simply a change in organization charts or does it have real implications for the cost estimator? This companion to a 2017 paper will build a historical understanding of the defense innovation and procurement processes and suggest a proper role for the cost estimator under the new organization. | 2018 | Comprehensive Perspectives |
| Investigation of the Disconnect Between the Use of CSDRs Cost and Contracting Communities | Marc Stephenson, Brian Davis | Today, the organization of project offices separates the program manager (PM) from the contracting officer. It might seem that for this reason, the PM's cost team often fails to adequately support contract negotiations. However, this paper will demonstrate that a more fundamental problem is the disconnected structure and flow of information to each of the functional teams. It will investigate the ties and prospective improvements to bring the cost estimator back into the contracting cycle. | 2018 | Comprehensive Perspectives |
| Where Have All the Estimators Gone? | Cris Shaw, Tom Dauber, Andrea West, Bryan Kenneth Anderson | The declining budgets, expanding oversight, sequestration challenges, and acquisition changes faced by Government programs has resulted in the increased importance of cost estimating and analysis, and higher value of qualified cost estimators. However, like many associations and organizations, ICEAA has seen a decline in membership despite expanding the overall focus of the organization into new areas. We will identify key influencing events, analyze recent ICEAA membership trends and recommend potential areas for membership growth. | 2018 | Comprehensive Perspectives |
| Being Certain About Uncertainty: Part 2 | Andy Prince, Christian B. Smart | This paper addresses the difficult and pervasive challenge of identifying extreme cost growth early in a project's life cycle and preventing it before it happens. The paper examines how DoD and NASA have implemented policies and practices to minimized or eliminate extreme cost growth and why those policies can sometimes fail. Finally, we propose some remedies that could help and identify some warning signs that a project may be headed for trouble. | 2018 | Comprehensive Perspectives |
| WBS vs CES: Navigating Different Structures for Software Systems | Brad Dahlin, Bakari Dale | Army estimates are created using either a Work Breakdown Structure (WBS) and the Cost Element Structure (CES). Determining which structure to use for your cost estimate can be a challenge. Through this presentation, the authors will present when to use which structure, and what information and level is required for each and provide information as to when a WBS/CBS is required. | 2018 | Comprehensive Perspectives |
| Introduction to the Organizational Cost Community Framework: Update | Terry Josserand, Edwin P. Chamberlin, Leone Z. Young | One specific area of emergent research and assessment, with respect to organizational role and responsibility overlaps and gaps, exists within an organization's ability to generate sustainable and defensible cost estimates. This paper will assist organizations facing the challenge to improve their cost estimation and analysis capabilities through the general characterization of the four interdependent cost functions, their processes, and the participants all within the systematic Organizational Cost Community Framework. | 2018 | Comprehensive Perspectives |
| Have All the Cost Estimates Already Been Done? Data Science in Cost Analysis | Jeremy Eden | Data Science and 'big data' or the analysis of large quantities of consolidated and searchable information are having great impacts on operations, industries, scientific fields, and analytical disciplines, including cost estimating. This paper will provide a basic understanding of data science, discuss some current issues and solutions for cost estimators using data science, and suggest some steps ICEAA can take to incorporate data science with current cost estimating training and certifications to capture and shape the benefits of the data science revolution in cost analysis. | 2018 | Comprehensive Perspectives |
| An Empirical and Visual Tale of a Cross-Country Bicycle Adventure | Rick Collins, Maggie Dozier, Orly S. Olbum, Paul Lanier Hardin III | The 'road' from Anacortes, Washington to Bar Harbor, Maine is paved with amazing landscapes, small towns, interesting people and cold beer. This 74 day cross-country cycling trip was an ideal opportunity to collect data that might explain daily riding speed. This presentation (and companion paper) paper describes the journey and post-ride analysis of the data using influence diagrams and constrained optimization (via Excel Solver) and will hopefully inspire others to get on a bike and experience the 'power' of cycling. | 2018 | Comprehensive Perspectives |
| Data Science Cost Estimating Challenges: Cut Your Time in Hack! | Joe Rohner, Lorraine Feury | The increasing velocity and variety of cost analysis data demands holistic data science approaches performed in near-real time. How can you discover multiple innovative, unbiased, cutting-edge solutions in just three days? Host your own Data Science Hackathon cost estimating challenge! Hackathons are environments that incorporate external open-source tools coupled with super-smart data scientists to solve challenges impeding success. Learn Hackathon best practices that serve as innovative catalysts and transform how your organization solves complex problems. | 2018 | Economic/Data Analysis |
| A Robot Brain Might Be the Best Forecasting Tool Possible | Nathan Eskue | Do you have to predict the future as part of your job'revenue, headcount, demand, production, etc.? Is it frustrating that something always seems to mess up your estimates? The robot brain'Artificial Neural Network (ANN)'can predict patterns in the midst of chaos. I'll share its mechanics/purpose, my experience using ANN's to self-learn (what could go wrong?), and how that same technology can produce forecast accuracy better than any other method. | 2018 | Economic/Data Analysis |
| Demand, Recurring Costs, And Profitability | Douglas K. Howarth | Customers in all markets collectively abide by their self-imposed demand curves, which dictate their responsiveness to changes in price and the maximum quantities of products they can absorb. Concurrently, producers in all markets face recurring costs, which typically fall over time due to a variety of factors. Producers can effectively model demand and recurring costs before product launch. Understanding how demand curves relate to recurring costs is key to enhancing profitability, which this paper examines. | 2018 | Economic/Data Analysis |
| The Art of Employing Data Science to Improve Cost Data Analysis | Richard Shea, Shavaiz Saood | While large data sets are highly desired and used frequently, there is not a universally accepted standard format for large data extractions and transfers. This presentation focuses on gaining insights into large amounts of data, spotting inconsistencies, and getting data into a usable format. This presentation examines the possibility of putting data into a standard database format so it can be easily manipulated and cross compared against other datasets that may have comparable elements. | 2018 | Economic/Data Analysis |
| Robust Non-Design, Code, Test, and Integration Cost Estimating Relationships | Britt Staley, Nicole Robertson | Computer program development (CPDEV) models are founded in accurately estimating the design, code, test, and integration (DCTI) of the computer program. Given that non-DCTI support costs can account for 50 percent or more of the estimate total, it is also critical to accurately capture these indirect costs. This paper analyses almost two decades of historic data from nineteen CPDEV efforts to derived timely non-DCTI cost estimating relationships for use in these types of estimates. | 2018 | Economic/Data Analysis |
| Calibrating Use Case Points Using Bayesian Analysis | Kan Qi, Anandi Hira, Elaine Venson, Barry Boehm | Use Case Points (UCPs) has been widely used to estimate software size for object-oriented projects. Yet, many research papers criticize the UCPs methodology for not being verified and validated with data, leading to inaccurate size estimates. This paper explores the use of Bayesian Analysis to calibrate the use case complexity weights of the UCPs method to improve size and effort estimates. Bayesian Analysis integrates prior information (in this study, we use the weights defined by the UCPs method and the weights suggested by other research papers) with parameter values suggested by data. The effectiveness of the calibration approach has been evaluated by our empirical study of 34 use case driven projects. We found that the Bayesian estimates of the use case complexity weights consistently provide better estimation accuracy compared to the weights proposed by the original UCPs method, the empirically calibrated weights, and the expert-based weights. | 2018 | Economic/Data Analysis |
| Improved Cost and Technical Data Collection for Contractors | Greg Kiviat, John Swaren | As a follow on to the 2017 ICEAA Workshop presentation 'Lessons Learned in Leveraging Historical Cost, Schedule and Technical Data' the authors present results of a Part II post-study enhancement applying standardized CARD (Cost Analysis Requirements Description) to existing data collection process, per audience feedback. The team has applied this enhanced approach and is exploring how contractors can leverage government standard process to improve the existing approach and gain insight beyond model calibration (e.g. CER development, analogous programs, etc). | 2018 | Economic/Data Analysis |
| A Case Study for Future Munition: An Analysis of Alternatives | Faye Kim, Meagan Gadreault | An Analysis of Alternatives (AoA) is essential in making a rational funding decision particularly before MS A decisions. An AoA was conducted to determine the Total Ownership Costs (TOC) for 21 possible Courses of Actions (COAs) for the design of a future munition. Multiple options within the future munition's three main hardware sections and software section create the COAs, and differences in these options within the COAs drive the TOC that impact the overall AoA decision. | 2018 | Economic/Data Analysis |
| Projecting Future Costs with Improvement Curves: Perils and Pitfalls | Brent M. Johnstone | Improvement curves are one of the most common projection tools used by cost estimators. Their use is surrounded however by perils and pitfalls. Common errors include: the fallacy of 'straight edge and graph paper' projection, the dangers of recovery slopes, failure to understand how development and production environments differ, and the dangers of using learning curve slopes to measure production line efficiency. This paper examines these potential pitfalls and proposes ways to avoid them. | 2018 | Economic/Data Analysis |
| Diamonds in the Rough: How to Normalize Cost Accounting Data | Niatika Griffin, Jason DeLorenzo | Every government agency has a cost accounting system. For cost estimation purposes, this is a wealth of actual cost data that can be used as a reference for estimates. Unfortunately, the needs of financial analysts differ from cost analysts. This paper explores how to normalize cost accounting system data using FAA's Delphi system as a case study. We conclude given certain assumptions the data can be normalized into usable source data and customized reports. | 2018 | Economic/Data Analysis |
| Applying Economic Theory to Cost Recovery in DoD Working Capital Funds | Kathryn Connor, Michael Vasseur | A DoD working capital fund must recover its costs, but some customers have questioned the cost of these services, which are perceived to be too high and include readiness and other costs not directly tied to customer demand. This study identifies and tailors best commercial practices in order to better align customer incentives with DoD goals and readiness needs. We recommend that this DoD working capital fund implement preferential pricing for those activities that contribute to readiness. | 2018 | Economic/Data Analysis |
| Integrating Excel Cost Models and MS Project Schedules | Melvin R. Etheridge, Jr. | The FAA requires non-risk-adjusted 'resource loaded' schedules as part of its Milestone Decision process. Integrating cost and schedule risk in MS Project schedules and Excel cost models is challenging. These challenges stem from the fundamental differences between a cost estimate, which is a tree, and a schedule, which is a network and the dependency of some cost elements to duration. This paper investigates techniques for linking risk-adjusted schedules and cost estimates. It highlights the need for close collaboration between cost estimators and schedulers. | 2018 | Management, EVM & Scheduling |
| Adventures in Using Contractor Cost Data Reports for Wheeled and Tracked Vehicles Analysis | Kimberly Roye, Jennifer Scheel | The Cost Assessment Data Enterprise (CADE) contains multiple types of contractor cost data reports (e.g. CDSRs, FCHRs, CPRs) that cost analysts may find useful in developing cost estimating relationships. Understanding the best report(s) to use depends on the analysis being performed and the optimal choice is critical to meaningful analysis. This paper will provide lessons learned to help analysts successfully navigate available cost reports and avoid common pitfalls. | 2018 | Management, EVM & Scheduling |
| The Living Estimate: Leveraging LCCEs Throughout the Program Lifecycle | Sriram Krishnan, Kevin Schutt | While cost estimates are crucial for program planning and approval after program approval, where does the estimate go? This paper examines how the LCCE can support and inform a program office beyond the planning stages. We will also look at the cost analyst's role in the Program Control and Business Management functions, and how s/he can add value to all aspects of the Program Control team. | 2018 | Management, EVM & Scheduling |
| Modern Methods for Budget-Constrained Schedule Analysis | Nick DeTore | There's a need across the community for methods to optimize schedules based upon constraints imposed by the available budget. Work cannot be performed until funding is available, yet current SRA/JCL tools don't reconcile this discrepancy. The cost/schedule analysis communities understand the prevalent and vital concept of a budget-constrained schedule and yet research has been inconclusive. This paper introduces a robust method of adjusting the schedule within complex program plans in order to satisfy a budget. | 2018 | Management, EVM & Scheduling |
| Your Schedule is in Shambles and This is Why: A Systematic Approach to Why So Many Programs Fail | Kai Lemay, Patrick Myers | In this presentation, we explore the root causes for why programs fail to achieve schedule targets, either set by them or set by external stakeholders. In this presentation we review how requirements, contracts and contract requirements, program management approaches (or lack thereof), scope creep, unrealistic estimates, and not taking a data driven decision approach to decision making all come together to form a perfect storm that prevents programs from achieving the schedule targets. | 2018 | Management, EVM & Scheduling |
| Proactive Estimating: The Analysis of Sixth Generation Aircraft | Dale Shermon | This paper will explore some of the options and alternatives which as a cost community we should be pursuing for new projects. It will examine the big, first order assumptions which we should be considering to ensure that we have a voice and that the cost is considered at the forefront of the decision process. As an example the paper will consider the options for a sixth generation fighter capability. | 2018 | Methods & Models |
| Design Life Study | Alex Wekluk, Colleen Adamson, Matt Reiley | The ODNI/SRA/CA division studied Design Life (DL) impact on the cost of traditional satellite acquisition strategies and associated launch costs. If launch costs continue to decrease it may be prudent to move away from constellations of extremely reliable vehicles to constellations of less reliable vehicles at an affordable price. A constellation of shorter DL vehicles tolerates a higher risk at the unit level and increases flexibility to evolve ahead of a changing technology landscape. | 2018 | Methods & Models |
| Commercial Applications of Predictive Analytics: Requirements-Driven Forecasting | John Swaren | Our job is often to fine-tune a predictive capability, process or system by best applying knowledge to justify key drivers for forecasting. Regardless of methodology, internally-developed or licensed, the common objective is creating valid/defensible estimates based on actual historical data as well as performance parameters. This paper will examine four commercial case studies where unique Predictive Analytics methods were implemented to both leverage proprietary knowledgebases as well as reflect requirements metrics to create justify forecasts. | 2018 | Methods & Models |
| Estimating Future Air Dominance | David Stem | Estimating the cost of aircraft programs early in development presents special challenges. The process requires a consideration of the content of the program and various methods to be employed. The Air Force Cost Analysis Agency recently provided cost advice to the Scientific Advisory Board on how aircraft programs are estimated, recent historical experience/lessons learned, and approaches to reduce life cycle costs while the program is in the conceptual design stage. | 2018 | Methods & Models |
| Just-In-Time Cost Estimate in a Multidisciplinary Design Environment | Apinut "Nate" Sirirojvisuth, F. Gurney Thompson III | Poor cost estimates cause inefficiency in the program and run the risk of increased bottom-line, program delay and cancellation. One remedy is to have an effective cost management framework that can evolve and adjust as the program matures. In this research case study, we present the formulation of cost-capability trade space that automates multidisciplinary collaboration resulting in a just-in-time cost estimate from any design change or technology infusion. An aircraft redesign example will be presented. | 2018 | Methods & Models |
| Unmanned Space Vehicle Cost Model: Past to Present | Ben Kwok, Chinson Yew | Space, the final frontier. These are the cost estimates that employ the Unmanned Space Vehicle Cost Model (USCM). USCM's 45+ year mission: to assist in the development of cost estimates, to seek out new data sources and new methodologies, to boldly evolve to meet the ever changing landscape of the cost world: Take a journey into USCM's development, a model traveling at warp speeds since the 60's that has undergone major evolutions over the past decades. | 2018 | Methods & Models |
| Building Dynamic Cost Estimating Models | Miranda Jones | Organizations are driving requirements for both cost estimators and analysts to be agile and deft in the application of dynamic, heterogeneous data efficiently with process repeatability. Developing versatile estimating models saves time, reduces errors, and drives consistency. This presentation will provide an overview of the 'what' and 'why' of dynamic estimating models and the techniques for building adaptable models that can be applied to any type of cost estimating discipline. | 2018 | Methods & Models |
| An Approach Towards Determining Value Through the Application of Machine Learning | Christopher Hutchings | Within the field of data analytics, machine learning is a method used to develop algorithms that lend themselves to prediction; often known as predictive analytics. These analytical models allow 'interrogators' to expedite the production of reliable and repeatable results and uncover hidden insights through learning from legacy relationships and trends in the available data. The intention of this paper is to demonstrate a value based engineering application of this embryonic field of analysis. | 2018 | Methods & Models |
| Learning Rate Sensitivity Model | Timothy P. Anderson, Nichols F. Brown | In space cost estimates, learning curves are used to estimate cost of small quantity acquisitions. Recently, spacecraft providers have started proposing unprecedentedly large constellations. The authors have developed a methodology to test assumptions about learning rates versus proposed cost estimates, providing a data-driven assessment of whether a proposed learning rate/cost combination is feasible or even likely, and further describes the learning rate would be necessary in order to meet a proposed cost estimate. | 2018 | Methods & Models |
| Enhancing Risk Calibration Methods | Christian B. Smart | Calibration methods such as the Enhanced Scenario-Based Method allow analysts to establish cost risk analyses that are based on objective data. Some methods currently in use rely on the normal and two-parameter lognormal. Empirical data, however, indicates that a three-parameter lognormal is more appropriate for modeling cost risk. We discuss three-parameter lognormals and how to calibrate cost risk using this distribution. We compare the results with traditional calibration to two-parameter normal and lognormal distributions. | 2018 | Risk & Uncertainty |
| Calculating a Project's Reserve Dollars from its S-Curve | Marc Greenberg | A probabilistic method was developed to calculate funds a project would need if it exceeded the project's point estimate. This conditional cost reserve is depicted as the amount of funds held in reserve 'above the project of program. The method requires three data inputs to calculate cost reserve: (1) the project cost dispersion (measured by its coefficient of variation), (2) the project point estimate, and (3) the confidence level of the project point estimate. | 2018 | Risk & Uncertainty |
| Estimating the Cost of Pharmaceuticals: Managing Cost and Expectations | Thurman D. Gardner, Robert Hunt | Recently headlines have been addressing the high price of pharmaceuticals. Contrary to what it may seem, greed does not drive pricing, it is the unknown and recovery of costs. Science provides a big piece of this, but the regulatory pathway is almost as big a factor, and in some cases bigger. Estimating new products requires a solid methodology and forward-thinking approaches to adequately bound projected costs and expectations. | 2018 | Risk & Uncertainty |
| Risk-Adjusted Contract Price Methodology | Peter J. Braxton, Keith S. Hetrick, Orly S. Olbum | Traditional risk approaches rely on program-level risk and uncertainty benchmarks from SARs, but governments and other buyers need to assess and manage risk at the contract level. For the first time, the Risk-Adjusted Contract Price Methodology explicitly models both 'off-the-shareline' risk and 'on-the-shareline' risk to present a complete and accurate distribution of final contract price. Drawing from a robust CLIN-level database of cost, fee, and price changes over time, it incorporates historical benchmarks for contract changes and other growth. | 2018 | Risk & Uncertainty |
| To Monte Carlo Or Not To Monte Carlo: That Is The Question | Joe Bauer | In cost estimating, the two most often used risk / uncertainty analysis methodologies are Monte Carlo simulation and Method of Moments. Have you ever stopped to wonder about the difference between the two? Fear not! In this presentation, the authors will compare and contrast the two risk / uncertainty analysis methodologies through several case studies. The authors will also share several unique methods for allocating risk dollars across the program phases. | 2018 | Risk & Uncertainty |
| Establishing Standards as the Basis for Effective Measurement and Affordability | Pete Pizzutillo | Measurement of application development output has long been a controversial topic. Yet as contracting relationships within industry and public sector become more strategic, buyers and sellers of software development and sustainment services require consistent and effective measures of application development output to provide: objective visibility into application development output; a rational basis for Application Development and Maintenance (ADM) investment decisions; and vendor and buyer accountability supported by data, not subjective judgments. | 2018 | Software Estimating |
| Software Data Collection and Analysis for Proposal Evaluation | Ken Rhodes, Eric Hippensteel | Software data collection forms provide necessary information for the Government to assess the validity of proposed software development effort, productivity, sizing, and schedule. This paper discusses use of the forms with an RFP to validate contractor proposals and compare across multiple bids. This data collection and evaluation approach helps ensure selection of the best vendor and defend against protests, and was recognized by Defense Procurement and Acquisition Policy (DPAP) as a DoD acquisition best practice. | 2018 | Software Estimating |
| Cost of Software Obsolescence Resolutions | Sanathanan Rajagopal | Software plays an important role in defence. Almost every project in defence has software elements with various degrees of complexity and dependencies. This has brought its own challenges to the availability-based contracts. The challenges to both the contractors and the suppliers is that they have to have a good understanding of the whole life cost of the product and have confidence in the whole life cost model at the time of negotiation and contract signing. This paper will look into the ways to estimate the cost of Software Obsolescence resolutions of real-time defence software. | 2018 | Software Estimating |
| Software Made Simple: Effort Adjustment Factors and the Accuracy of the Estimate | Jeremy Goucher | This research investigates the sensitivity of estimated hours for software development in relation to effort adjustment factors (EAFs). The analysis highlights the importance of performing original data analysis for new estimates, rather than relying on rules of thumb or industry standards. Accuracy and precision metrics are used to demonstrate the applicability of the ESLOC method to a variety of software projects, including agile projects. Finally, the analysis is supported with data from over thirty historic programs. | 2018 | Software Estimating |
| Predicting Maintainability for Software Applications Early in the Life Cycle | Cara Cuiule | Maintainability is defined as the difficulty of altering a software system's source code, thus it is tied very closely to the concept of software maintenance. The following research is an investigation of the methods used for measuring this characteristic (including the Maintainability Index). Guidance on how maintainability affects maintenance effort will be proposed. This will be followed by a discussion of which metrics could possibly predict maintainability early in the life cycle. | 2018 | Software Estimating |
| Impact of Scope Changes on Software Growth | Jonathan Brown, Gail Flynn | The SEI DoD Software Factbook summarizes MDAP/MAIS SRDR data for DoD programs. The mean value reported for ESLOC growth is 106%. While accurate, the SEI's and other similar analyses capture total software growth, including the impact of scope changes. This paper introduces 'Pure Software Growth' which differentiates planned scope changes from traditional software growth. Several programs are analyzed from this perspective to show the difference between pure and total growth and the unexpected impact this could have on estimates. | 2018 | Software Estimating |
| Why Does Software Cost So Much? Toward a Causal Model | Michael D. Konrad, Robert Stoddard | How can we control the cost of software-intensive systems? To contain costs we need to better understand which factors truly drive costs versus those merely correlated with cost. In this talk, we will share results from application of a newly developed causal discovery and modeling tools to evaluate these potential causes of code quality and effort: number of requirements, software size, schedule, team experience; programmer productivity and error-proneness; and architecture pattern violations. | 2018 | Software Estimating |
| A Probabilistic Method for Predicting Software Code Growth: 2018 Update | Eric M. Sommer, Bopha Seng, David LaPorte, Michael Ross | Software estimating is challenging. SMC's approach has evolved over time to tackle this challenge. Originally based on Mike Ross's 2011 DSLOC Estimate Growth Model, we've updated our model to include more recent SRDR data and an improved methodology (Orthogonal Distance Regression). Discussions will focus on non-linear relationships between size and growth, unique growth for new, modified, and unmodified DSLOC, as well as correlation between DSLOC types and future efforts to include space flight software data. | 2018 | Software Estimating |
| Estimating the 'Total Cost of Ownership' of Cybersecurity in an Increasingly XaaS world. | Zachary Jasnoff, David Cass, Richard Mabe | Converging technology trends in XaaS have profound effects on how organizations are evaluating decisions regarding XaaS outsourcing and hybrid deployments as more business functions move to the cloud. Most organizations have a security skills gap that XaaS and moving to the cloud can help solve giving choices on insourcing or outsourcing cybersecurity. This paper explores how XaaS impacts the TCO of cybersecurity and also deliver guidance on the estimating the cost of the DFARS cyber policy to defense programs. | 2018 | Technology & Innovation |
| Modeling Technology and System Readiness Level Impacts on LCC | Apinut "Nate" Sirirojvisuth, Bakari Dale | Technology Readiness Levels (TRLs) is an important construct used to assess maturity of a technology and set criteria for inclusion in a program with an implied level of risk. In this research, we attempt to quantify the risk implications in terms of cost and schedule impacts. A new cost model will be introduced that utilizes technology maturity assessment of constituent components of a system as a metric to determine cost, schedule, and uncertainty response. | 2018 | Technology & Innovation |
| Application of K-Means Clustering Methodology to Cost Estimating | Jacob Walzer | Cluster Analysis provides an efficient method of analyzing large datasets and grouping them based on similar characteristics. While not suitable for all datasets, clustering allows analysts to easily classify data while considering all relevant variables. This presentation explores the best practices, common pitfalls, and validation techniques associated with clustering, and includes a notional example derived from an approved application of this technique utilized in a USMC estimate. | 2018 | Technology & Innovation |
| Machine Learning & Non-parametric methods for Cost Analysis | Karen Mourikas, Nile Hanov, Joseph King, Denise Nelson | The world of big data opens up new opportunities for ICEAA, such as machine learning and non-parametric methods. These methods are more flexible since they do not require explicit assumptions about the structure of the model. However, a large number of observations is needed in order to obtain accurate results. Hence, the use of big data. This presentation examines several non-parametric methods, with examples related to our community, and discusses opportunities and limitations going forward. | 2018 | Technology & Innovation |
| Costs Considerations in Refreshing Vulnerable IT Networks | John Leahy | Virtually all IT networks must deal with the growing threat of cybersecurity intrusion and yet retain sufficient features to meet mission needs. Many IT network components are close to, if not past, their end of life. This presentation will provide a high level view of alternatives analysis focused on the cost of a network refresh with secure, state of the art components. Topics include different approaches to network refresh and assessment of stakeholder interests. | 2018 | Technology & Innovation |
| Social Media and Submarines: How Machine Learning and Unconventional Methods Can Change Cost Estimating | Omar Akbik, Jeffrey Pincus | As technology advances and analytical techniques evolve, machine learning and big data systems stand to dramatically change the cost industry- these changes are coming sooner than we may think. Industry requires timely, efficient and defensible analysis, and new data sources are allowing previously unexplored opportunities to meet these goals. We explore how automated algorithms can identify market trends to forecast program cost with greater accuracy, and in less time than historically required by an analyst. | 2018 | Technology & Innovation |
| Agile Delphi Estimating and Analyzing Schedule/Resource Realism for Software Projects | Blaze Smallwood | One of the most important tasks for a software project manager is to accurately forecast the schedule and resources needed to complete the project. This paper will describe a novel approach to this problem that combines Monte Carlo-based uncertainty analysis with an innovative new technique for estimating software requirements, which utilizes an agile-style Delphi estimating methodology. This method has been successfully implemented on several government projects and is a viable option for any software project. | 2017 | Agile |
| Agile: You're Doing it Wrong (or how to know you're doing it right) | Daniel B. French | Agile is a hot topic in the IT world today. If your organization is using Agile, or considering it, there are a number of things you need to know. This presentation will discuss the pros and cons of Agile, how to determine if your organization is ready for it, proper Agile implementation, hybrid methodologies, and the organizational challenges when trying to make an organization Agile. | 2017 | Agile |
| Agile and GAO Cost Estimating Best Practices | Jennifer Leotta, Karen Richey | This paper will examine how GAO's cost estimating process can be applied to programs that are using an Agile framework. First, it will provide a brief overview of Agile processes and methods. Second, it will examine each of the 12 steps in the GAO cost estimating process and how those steps relate to an Agile framework. Finally, it will discuss how Agile artifacts can be leveraged to fulfill cost estimating documentation needs. | 2017 | Agile |
| A Cost Model for Early Cost Calculation of Agile Deliveries | Eric van der Vliet | Agile software development methodologies provide the IT industry with the flexibility they need to keep up with the faster change of business requirements. In agile software delivery, upfront detailed specifications are absent, yet investment decisions need budget input. The challenge is to build a cost model that takes essential (size) and additional cost drivers into account. This presentation explains a cost model that supports the cost calculation for agile delivery early in the process. | 2017 | Agile |
| Agile 'Mumbo Jumbo' | Jeremy Eden | You don't put any stock in this Agile "mumbo jumbo" do you? Well, Agile is here to stay and is gaining popularity.This paper will discuss current issues and some solutions for cost estimating Agile projects, clear up common misconceptions about Agile, and Introduce an Agile approach to cost estimating, including a draft Agile Cost Estimating Manifesto with suggestions on how to incorporate it with current cost estimating training, so ICEAA and its members can BE Agile. | 2017 | Agile |
| Beyond the Manifesto: Tracking Agile Performance in a DoD Environment | Gordon M. Kranz, Michael Thompson | The use of Agile methods are becoming prolific within the DoD Enterprise and Weapons systems development environment. Agile supports the incremental discovery of end system capability within the constraints of the DoD parameters. This paper investigates, through real world experience, the use of earned value techniques to strategically manage the Agile development process. One of the essential elements key to success is traceability of user requirements to Agile features. | 2017 | Agile |
| How Should We Estimate Agile Software Development Projects and What Data Do We Need? | Tom Coonce, Glen Alleman | This paper proposes and demonstrates how to estimate agile software projects using collected data from prior similar projects. The authors discuss traditional ways of estimating software projects and show why these approaches are not feasible in estimating agile projects. They demonstrate estimating agile projects using actuals from historical features. They recommend a few additions to DoD's Software Resource Data Report (SRDR) as well as a Standard Feature Breakdown Structure (FBS) to permit using this approach. | 2017 | Agile |
| CMMI and Agile Development: A Binary Choice? | Arlene F Minkiewicz | The Capability Maturity Model Integration for Development has a long and impressive history for progressing the cause of process improvement. Agile development is a paradigm for software projects that are characterized by collaborative, cross functional teams working closely with customers to deliver functionality regularly. Many say these two approaches can't work in tandem. This paper explores this question with examples of both successes and failures. | 2017 | Agile |
| Assessing ERP Cost, Schedule and Size Growth | Hasetetsion Gebre-Mariam, Abishek Krupanand | This study will examine percentage changes in cost, schedule, and size across Milestones A, B, C, and full deployment for DoD Enterprise Resource Planning (ERP) programs. The analysis is based on 9 fielded systems collected from DoD authoritative data sources. Cost contributors, drivers, and factors by major cost elements will also be examined. Results may be used for crosschecking cost estimates or business case analyses at an early phase to inform funding decisions. | 2017 | Data Analysis |
| Lessons Learned in Leveraging Historical Cost, Schedule and Technical Data | Greg Kiviat, John Swaren | The process of applying historical cost, schedule and technical data to develop new program estimates is often more difficult than textbooks suggest. The paper provides real-world insight to the issues that arise when capturing, analyzing and applying data for the next estimate. The goal is to integrate information from multiple products, systems and subsystems. Lessons learned include experience and best practices when working with financial, engineering, manufacturing and program management. | 2017 | Data Analysis |
| Technology Readiness Level (TRL vs. Percent Development Cost) | James Linick | Advanced technology acquisition often proceeds in steps characterized by TRL of systems or components. The relationship between TRL ordinal number and percent of total acquisition cost could be used as a factor in estimating the cost of new acquisitions at each phase from concept definition through production decision. A study of historical data will be performed to establish such a relationship based on a specific definition of TRLs, which vary across government agencies. | 2017 | Data Analysis |
| Machine Learning Approach to Cost Analysis | Karen Mourikas, Joseph A. King, Denise J. Nelson | At ICEAA 2016, the keynote speaker presented a challenge to the cost estimating & analysis community: to develop cost estimating methodologies based on machine learning. Machine learning provides an alternate to parametric modeling. At Boeing, we are experimenting with several machine learning techniques for cost estimation and analysis; Random Forest prediction methodology, in particular, has shown encouraging results. This paper describes our cost applications of Random Forest. | 2017 | Data Analysis |
| Technology Development Cost and Schedule Modeling | Chuck Alexander | A tangible need exists in the scientific, technology, and financial communities for economic forecast models that improve new or early life-cycle technology development estimating. Industry models, research, technology datasets, modeling approaches, and key predictor variables are first examined. Analysis is then presented, leveraging a robust industry project dataset, applying technology and system-related parameters to deliver higher performing parametric cost and schedule models. | 2017 | Data Analysis |
| DHS Cost Analysis Overview | Darrin DeReus | Cost Estimating and Analysis is a vital resource for program and project managers, regardless of agency or activity. July 2014 the cost analysis responsibilities and personnel were transferred from the DHS's Office of Program Accountability and Risk Management to the Office of the Chief Financial Officer. This presentation will provide a history of the creation of a cost analysis capability in a new agency, changes over time, lessons learned and current successes of the Cost Analysis Division. | 2017 | Data Analysis |
| Production Rates: Do They Really Matter? | Brent Johnstone | Production rate is widely assumed to be an important contributor to unit cost -- higher production rates lead to lower unit costs, and vice versa. Examination of published data, however, leads to a more ambiguous picture. This paper examines the impact of rate by functional cost element, including the impact on learning curves. It concludes production rate impacts are real, but the impacts are uneven and sometimes reveal themselves in surprising ways. | 2017 | Data Analysis |
| Using Real Options to Quantify Portfolio Value in Business Cases | George Bayer, | With the challenges in quantifying program risks, interdependencies, and the business case impact on portfolios, cost estimators can use real options to estimate probabilistic value in business cases and assess portfolio value. Many business cases not only add value as stand-alone investments, but they provide the opportunity for subsequent investments. Cost estimators can measure this incremental program value and the impact of a specific investment on a larger portfolio by using real options. | 2017 | Economic Analysis |
| Discount Rate for Government Investment | Ron Beheler, Laura Barker | We often use a discounted-cash-flow construct to evaluate the Economic Viability of an investment. OMB-94 provides discount-rate guidance for internal investment (treasury rates) and public investment/regulatory programs (7% - rate of return on an average private sector investment). Since any resources consumed by the government must be foregone by the private sector, we will argue that the discount rate should be based on a rate that at least proxies the value of private investment. | 2017 | Economic Analysis |
| Are the Rates Right? Benchmark Protection Against Escalation SWAG | Brian Flynn, Brian Torgersen, Greg Mihalek, Adam James | Medical escalation rates continue to trend higher than general inflation - with significant impact on defense firms and acquisition affordability. Modest deltas in rates translate into billions of additional costs. This research presents benchmark escalation values for employer costs of employee medical compensation. The benchmarks, in turn, support company planning & collective bargaining and liberate government cost estimates from their traditional reliance on solely FPRPs and FPRAs. | 2017 | Economic Analysis |
| The Rate You Want " Rethinking Cost Escalation in FAA Development | Sriram Krishnan, Andrew Drennon | Like most civilian agencies, the FAA has used 'raw' OMB escalation indices to produce then-year costs for its life cycle cost estimates. However, as an agency which incrementally funds many development programs, could the FAA benefit from recognizing time value of money principles at the estimation stage? We look at outlay data from across the agency to determine whether defense appropriations styled weighted indexes would significantly impact then-year FAA solution development costs. | 2017 | Economic Analysis |
| Terminal Facility Realignment: A Business Case Approach | Deji Oladipupo, Ayn Smith, Melvin Etheridge, Richard Shepro | The Federal Aviation Administration applies a business case analysis approach to the decision process for realigning and consolidating Terminal facilities. A Microsoft Excel model has been developed to produce these analyses. It uniquely displays both costs and benefits in one place and includes graphical outputs of NPV over time. A key output of the analysis shows interesting cyclical trends of NPV rising, falling, and rising again over time in almost all scenarios. | 2017 | Economic Analysis |
| Kennedy Space Center's Transformation to a Multi-User Spaceport | Terry Lambing | In the past, launch pads were used almost exclusively for government missions. To support a growing private sector space economy, NASA's Kennedy Space Center has transformed into a multi-user spaceport capable of handling the needs of a variety of companies from launch processing and operations through recovery. This presentation will provide an overview of the transformation and the NASA Business Case requirements needed in order to gain approval, and successfully establish these unique partnerships. | 2017 | Economic Analysis |
| NRO CAAG Spacecraft Test Schedule Estimating Relationship | Daniel Barkmeyer | The NRO CAAG's parametric model for estimating spacecraft testing schedule duration has been updated. The updated model incorporates data from government and commercial spacecraft contracts. The intent of this presentation is to share the model with the ICEAA community. The latest model will be presented, along with a case study of a satellite program that illustrates the importance of the chosen schedule drivers to system test schedule. | 2017 | EVM & Scheduling |
| Fully Integrated Cost & Schedule Method (FICSM) Analysis Schedule Implementation | Jonathan Brown, Benjamin T. Unruh | The Joint Agency Cost Schedule Risk and Uncertainty Handbook released in March 2014 introduced the FICSM which integrates cost, schedule and risk estimating. Key to the FICSM process is creation of an analysis schedule including duration uncertainty. This study leveraged data from an active program to develop an analysis schedule as part of building a FICSM model. This paper describes the methods used to develop the analysis schedule and duration uncertainty used for this program. | 2017 | EVM & Scheduling |
| Beyond RIFT: Improved Metrics to Manage Cost and Schedule | Nicholas DeTore | Risk-Informed Finish Threshold (RIFT) presented an innovative solution to the problem inherent in schedules that risk analysis results (time) cannot be allocated the same way as in cost models (dollars). Developing RIFT validation methods inspired an exploration into analyzing simulation data more meticulously. Methods described here provide unique insight into cost and schedule uncertainty results while introducing powerful new techniques to improve a project's potential to complete on time, on budget. | 2017 | EVM & Scheduling |
| Adopting DOD Best Practices in IPM to non-DOD Needs | David L. Wang, Daniel Schwartz, David T. Chiang | Non-DoD federal acquisition agencies have their own unique acquisition requirements and acquisition culture. However, many non-DoD acquisition agencies face similar challenges, specifically in the areas of acquisition program execution and program affordability. To address these challenges, there has been increased interest to consider tailoring DoD best practices in Program Management, Earned Value Management and acquisition lifecycle schedule management. This presentation discusses a systematic approach to tailoring DoD best practices for non-DoD acquisition needs. | 2017 | EVM & Scheduling |
| Cost Associated with Acquisition Complexity and Differing Levels of Mission Assurance | Erik Burgess | In Government acquisition programs, the cost of what we buy is largely driven by how we buy. For satellites, this includes contracting, funding, oversight, reporting, testing, parts assurance requirements, and other factors. All of these are related, and no single attribute defines the cost curve. The NRO has developed and implemented a method that quantifies the cost of these factors and circumvents the usual debate about whether or not an acquisition is streamlined or commercial-like. | 2017 | Methods |
| Don't Dis LOE: Modeling Production Sustaining Labor Across Multiple Lots | Sandy Burney | The estimation of sustaining labor is often given little analysis; just assuming some constant level. However, in estimating production hours on large Programs, spanning multiple lots, where each lot is negotiated separately, conflicts often occur over the cost of sustainment labor. This briefing will look at different approaches to estimating and modeling sustaining labor, based upon varying cycle times for completing a single unit, and the number of units being procured in each lot. | 2017 | Methods |
| Automated Data Collection Using Open Source Web Crawling Technology | Anna Foote | For many reasons, data gathering is often one of a cost estimator's biggest challenges. Even in cases where data is available, the logistics of data collection can be daunting. This paper explores a methodology developed by PRICE utilizing free, open source web crawling technology to automatically seek and find commodity pricing for information technology hardware and software items. This presentation shares the nuts and bolts for everyone's benefit. | 2017 | Methods |
| A Framework to Price and Cost IT Network Services | John Leahy | The correct pricing of an IT network offering aimed toward internal users can be an involved process. Different services targeted to some, but not all customers makes proper cost allocation challenging. Considerations such as mirroring the structure, if not the rates, of commercial offerings; reconciliation of differences between cost allocations and rational pricing; and influencing customer purchasing behavior adds to the complexity. An approach for costing and pricing IT services that address these requirements will be presented. | 2017 | Methods |
| Don't Get Caught in the Learning Curve Vacuum | Patrick McCarthy | Production environments featuring multiple end items benefit from a manufacturer designing products and processes with a high degree of commonality and standardization. When estimating labor hours using learning curves, the estimator often overlooks commonality when considering quantities, slope, production rate and lost learning. This paper will analyze alternative approaches to defining commonality and how to account for commonality using learning curves in integrated production environments. | 2017 | Methods |
| Modeling Hardware Development Cost in a Low TRL / Pre-Acquisition Environment | Jack Snyder | Estimating the cost of pre-concept programs, often with very immature TRL's, is always challenging. This case study highlights the use of analogous programs and predictive modeling to overcome the challenge. We will explore the strategy of finding analogous components of a program, even when the overall program is unrelated. Predictive models can easily be tailored for the inherent uncertainty of these early programs, when we are unable to find relevant data. | 2017 | Methods |
| Exploring the of Results of two Methodologies for Unmanned Space Estimation | John Swaren | Independent validations of two separate predictive analytics methodologies were performed over the last two years. In both cases, common detailed assessments of past missions integrated technical and programmatic requirements to mimic a grass-roots bottom-up methodology. This presentation compares the process and top-level results from both approaches as well as explores several lessons learned. | 2017 | Methods |
| Deployment Cost Estimation for Electronic/IT Systems | F. Gurney Thompson III | This paper will discuss our research into cost estimation for the deployment of Electronic and IT systems, also known as Operational/Site Activation. The deployments can span multiple sites/locations, and often require technical studies, end user working groups, and site survey visits in addition to the actual installation activities. This paper will discuss our research approach, sizing metrics, cost drivers, model structure, and future work. | 2017 | Methods |
| Budgeting for Canadian Shipbuilding " Examining Predictive Cost Analytic Methods | Peter Weltmann Zachary Jasnoff | Predictive Cost Analytics plays a major role Canada's Shipbuilding Strategy in developing cost and budget estimates. However, these estimates must be based on sound historical cost data. This paper discusses challenges faced by the Canadian PBO in developing reasonable estimates for the JSS, AOPS and CSC shipbuilding programs based on the quality of historical data available, the methodologies employed to address these challenges, and the influence of these estimates on decision-makers | 2017 | Methods |
| Decision Trees and Cost Estimating | Josh Wilson | Decision trees are predictive modeling approaches that recursively partition data, and apply simple prediction models to the resulting subgroups. Decision tree based prediction approaches have the benefits of being easy to interpret and explain, requiring little data preparation or cleaning, and the ability to model complex nonlinear relationships and variable interactions. This presentation explores the applicability of decision tree based prediction methods to cost estimating. | 2017 | Methods |
| Cost Estimating Canada's Future Surface Combatants | Rod Story | Canada has not built a new surface combatant since it completed the last of its 12 Halifax class frigates in 1996. Currently Canada is in the middle of a request for proposal to build, using a common hull, 12 new frigates and 3 new destroyers. This paper presents the first public cost estimate of these ships highlighting both the challenges that were encountered and the solutions that were used in determining a reliable cost estimate. | 2017 | Methods |
| Modeling the Influence of System and Application Complexity on the Cost of Cloud Hosting | Daniel J. Harper, Kevin Buck | To refine cost estimates produced within a Cloud Total Ownership Cost model, MITRE is now prototyping an additional modeling feature that estimates the cost impact of migration and hosting complexity. While many cloud cost models rely on a limited set of cost drivers, other factors associated with general complexity significantly influence costs and are often ignored. Within the prototype, complexity assessments are plotted using intuitive spider diagrams for ready comparisons of candidate systems and applications for hosting in various cloud solutions. | 2017 | Methods |
| Marco-Parametrics; Its Unique Capability and Application | Dale Shermon, Arlene F Minkiewicz | The Family of Advanced Cost Estimating Tools (FACET) has been used around the world as a macro parametric cost model at the early stages of a project life cycle. It has a unique capability to seamlessly transition from performance to design based estimating. FACET will also combine uncertainty in the inputs with those of the algorithm to produce a true uncertainty range. Now implemented in TruePlanning the FACET model is easy to access and apply. | 2017 | Methods |
| Projecting Program Spare Parts Sustainment with Incomplete Data | Bryan Anderson, George Bayer | Analyzing a government acquisition, the team discovered that the existing agency supply chain system lacked transparency, and historical failure data was unavailable due to implementation of a new supply chain system. The team examined how to accurately project parts demand using incomplete data by conducting a comparative analysis between data sources, using subject matter experts to estimate upper and lower bounds, and applying statistical analysis fit curves to project spare parts needs. | 2017 | Operating & Support |
| Sustainment Cost Data Collection | Sandi Enser | Operations and support costs can exceed 60% of the total Life Cycle Cost of DoD systems. Currently, the individual services have their Visibility & Management of Operating and Support Costs (VAMOSC) systems. However, with many sustainment functions performed by contractors, improved reporting on sustainment contracts is essential. This paper presents several data collection initiatives for sustainment efforts and their impacts on estimating O&S costs. | 2017 | Operating & Support |
| Expanding the Range of Your Data: A Small Ships Case Study | Kathleen Hudgins, Robert Nehring, Elizabeth Koza, Anna Irvine | While the Navy has comprehensive Operating and Support (O&S) data for current Navy ships, there are a limited number of smaller boats available for inclusion. With increasing technological advances and a renewed effort to reduce personnel, smaller boats are receiving increased interest. This paper explores using Coast Guard O&S data to supplement Navy data for use in estimating O&S costs for smaller boats. Topics include data sources, normalization, and comparisons of data between the services. | 2017 | Operating & Support |
| Army Software Maintenance Cost Estimating Relationships | Cheryl Jones, James Doswell, John McGarry, Jenna Meyers | For the past four years, the Army, under the leadership of DASA-CE, has been collecting and analyzing Army system software maintenance cost and technical execution data to support the development of more accurate cost estimation methods. The presentation will present the cost methods and cost estimation relationships developed from the analysis of the execution data sets. The results of the Army's analysis efforts, including the detailed statistical analysis will be made available. | 2017 | Operating & Support |
| Implementing Additive Manufacturing Technology into the Logistics Supply Chain | Patrick K. Malone, Bruce Fad | Recent explosive growth in Additive Manufacturing (AM) or 3D printing is providing logistics supply chain economic opportunities. We investigate self-sufficient repair and maintenance capabilities for isolated environments, impacts on strategic readiness, increasing responsiveness and cost efficiencies not available in traditional supply chains. Our research will evaluate and contrast legacy logistics architectures against AM elements that will drive cost downward over lifecycles to meet current and future affordability goals within the Government and commercial organizations. | 2017 | Operating & Support |
| Data Driven Confidence Regions for Cost Estimating Relationships | Christopher Jarvis | Cost estimating models typically contain many uncertain parameters. The uncertainty of the parameter values drives the uncertainty of the model outputs. In this paper we review and compare the techniques to compute the confidence regions for linear and nonlinear regression methods. Application to two common CERs is presented with comments regarding the practical implementations and limitations within the current cost estimating environment. | 2017 | Parametrics |
| Moment One, Please | Chad Krause, Erik Burgess | Cost models constructed without statistical correlation are designed to underestimate the mean, unless they are simple sums. Any model that uses a factor or other instance so functional correlation will miss the first moment, not to mention others, unless statistical correlation among the CERs is also applied. An internal audit of 13 recent satellite cost estimates comprising of 52 CERs shows the mean cost changes by as much as 9% when this correlation is applied. | 2017 | Parametrics |
| Modeling with Gumby: Pros and Cons of the Weibull Curve | Michael Mender, Ann Hawpe | The Weibull function is used to model various cost phenomena. This popularity is driven by its significant flexibility, driven by its nature as a two parameter function. Additional parameters result in flexibility, which while advantageous, also places incorporating estimates at risk of over fitting (i.e., a model that fits the observed data well, but has little predictive power). Presentation relays theory and exhaustively reviews pros and cons for cost estimating. | 2017 | Parametrics |
| Maximum Likelihood Estimation for Regression of Log Normal Error | Christian Smart | The use of Log-transformed Ordinary Least Squares (LOLS) has been criticized for the use of transformation, which results in biased estimates. LOLS is a maximum likelihood estimate (MLE) of the median when the residuals are lognormally distributed, for which we provide evidence. We discuss MLE and show how to use this technique to directly model untransformed lognormal error. We also discuss two other popular methods, ZMPE and MUPE, in the context of MLEs. | 2017 | Parametrics |
| Novel Manufacturing Methods ~ Characterizing the Impacts to Cost | Zachariah Sayre, Frank Campanile | Average unit procurement costs for military aircraft are growing at a rate that far exceeds inflation. Efforts such as the Air Force Research Lab's Composite Affordability Initiative and the Advanced Composite Cargo Aircraft have demonstrated alternative manufacturing methods utilizing composites to achieve significant part and fastener count reductions. This analysis highlights the concepts of large composite unitization in fixed wing aircraft manufacturing and its resulting effects to cost. | 2017 | Parametrics |
| Assessing Confidence Levels in Funding and Budgeting NASA Science Missions | Robert E Bitten, Charles D. Hunt | NPR7120.5E requires that NASA utilize joint confidence level (JCL) analysis to set budget and funding guidelines for projects within its portfolio of missions. This paper addresses analysis that was conducted to determine the effect of different confidence levels on the performance of different cases of mission portfolios. The results show that the most effective confidence level varies depending on the case. An overview of the policy, methodology, cases and results are discussed. | 2017 | Parametrics |
| Assuring Credibility in the Cost Estimate | Henry Apgar | Credibility can be the most important attribute of a cost estimate. This paper traces the evolution of quality metrics that assess cost credibility in the words of senior government executives, industry leaders, estimating and engineering handbooks, professional journals, and government auditing manuals. The presentation concludes with recommendations for the estimating professional. | 2017 | Policy & Standards |
| In Theory Or In Practice? The Optimistic World of Pessimistic Cost Estimators | Erin K. Barkel | In 2015, Andy Prince won the Best Paper award for The Psychology of Cost Estimating, a piece which highlighted a number of cognitive biases which impact the work of cost estimators. Of these cognitive biases, the one that seemed to resonate most strongly with the community was optimism bias. However, not everyone is worried that the glass is half-full. Where do negativity and pessimism bias fit? How does our understanding of these biases change how we communicate with stakeholders? | 2017 | Policy & Standards |
| How to use Predictive Analytics for a DCAA-Compliant Estimating System | Anthony A DeMarco | The DCAA ensures that Defense contractors use an acceptable estimating. They establish acceptable estimating system compliance criteria and audit's contractors to determine compliance. Non-compliance can be devastating. This presentation demonstrates a data-driven estimating methodology that complies and reduces the time and labor necessary to produce estimates. It illustrates the roles and procedures necessary for success. Six key compliance tenants are identified and key benefits quantified. | 2017 | Policy & Standards |
| Developing a Cost Capability Roadmap | John Fitch | Developing cost estimating tools and databases is critical to analyst productivity and the quality of analyses. Historically, organizations struggled to systematically improve capabilities due to a lack of a long-term plan, unstable budgets, and priorities that shift with changes in leaders. This presentation walks through how one organization confronted this challenge by developing a roadmap that links the mission to requisite tools, data, and training; and a plan to achieve the roadmap. | 2017 | Policy & Standards |
| The Shortcomings of Cost Estimating Templates | Meagan Gadreault, Faye Kim | Templates are seen as a best practice in many industries to standardize workflow and reduce redundancy. However, templates may not be the ideal solution for cost estimating as they can suppress innovation and constrain analysts. Instead, best practices should be determined using process improvement methods and implemented to create customized estimates. This paper explores leveraging Lean Six Sigma techniques to determine alternatives to templates in order to put forth the best estimates. | 2017 | Policy & Standards |
| The Art of Cost: Sun-Tzu's Strategic Insight in Cost Estimation | Brian A. Gillespie | Strategy is the process employed to determine how political purpose is translated into action. Clausewitz in his grand work "On War" argued that war serves a political purpose. So too does cost estimation, for at its most fundamental level, cost estimation attempts to determine the level of resources needed to achieve a political outcome. As one of the greatest enduring works on military strategy, what does The Art of War teach us about cost estimation? | 2017 | Policy & Standards |
| Coherence & Oddities: A Retrospective of Cost Estimating Publications 1978-2016 | Ross A. Jackson, Bradley C. Boehmke | Intellectual sedimentation can operate unobserved within professions. Assessing publication trends serves to make such conventional wisdom explicit. Our analysis of articles from the Journal of Cost Analysis and Parametrics (and its predecessors) provides insight regarding research coherence and oddities within the cost estimating community. This knowledge is essential for envisioning alternative futures and could be of benefit to those engaged in the praxis of cost estimating or its research. | 2017 | Policy & Standards |
| A History of Thought in Defense Acquisitions | Eric M. Lofgren | As Congress debates another round of defense acquisition reform, the necessary role for the cost estimator is affirmed. But how did this role come about and what are future implications? From the famed RAND systems analyses of the 1950s to the introduction of data reporting systems still in use today, this paper will explore the rich history of thought in defense acquisition, giving a special eye to controversies and continuing challenges that affect cost estimators. | 2017 | Policy & Standards |
| Establishing and Implementing Cost Estimating Standards | Hetal Patel, Denise Dulee, Danielle Spencer | As part of its efforts to create a world-class cost estimating capability, the Missile Defense Agency (MDA) wrote a cost estimating handbook and established an independent assessment team to ensure adherence to the handbook. As a result of this, MDA has received positive feedback from GAO in its progress towards this goal. This briefing describes in detail the processes and procedures that MDA has developed. | 2017 | Policy & Standards |
| Estimating Challenges and Solutions at NASA: Past, Present, & Future | Cabin Samuels, Jeff Brown | NASA's cost estimating landscape is fraught with unique challenges. Missions often incorporate cutting-edge technology and operate in harsh environments with no margin for error. Goddard's Cost Estimating, Modeling, & Analysis (CEMA) Office has developed robust methodologies to create realistic, transparent cost estimates for missions with little or no cost heritage. This paper discusses past and current states of cost estimation at Goddard and provides insights to shape future improvement. | 2017 | Policy & Standards |
| Anatomy of the Future DoD Cost Estimator | Tamiko L. Ritschel, Jonathan D. Ritschel | The need for a specialized, properly qualified DoD cost estimating workforce has resulted in a migration of civilian billets from a financially focused job series to the 1515 Operations Research series. We analyze the origins of this change, implications for the current workforce, and then discuss the anatomy of the future DoD cost analyst. While we focus on recent changes occurring within the U.S. Air Force workforce, the future implications are relevant to all Services. | 2017 | Policy & Standards |
| Masters in Cost Estimating and Analysis | Dan Nussbaum, Greg Mislick | This presentation provides an update on the all-distance Learning Master's Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS). | 2017 | Policy & Standards |
| Contract Geometry Best Practices for Incentive Contracting | Peter J. Braxton | With the current emphasis on Incentive Contracting, the cost analyst plays a vital role in establishing target cost, fee, and shareline. Drawing from the Joint Contract Price Database, this paper examines actual contract geometries across a wide range of programs and their effectiveness in encouraging cost management. Using the published Risk-Based ROS methodology, it provides a framework for implementing incentive arrangements for both competitive and negotiated procurements. | 2017 | Risk |
| A 'Common Risk Factor' Method to Estimate Correlations Between Distributions | Marc Greenberg | A 'common risk factor' method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared to estimate common risk factor 'intersections' that are, in turn, proxies of correlation. | 2017 | Risk |
| The End of S-Curve Alchemy: Gold from a New SAR Database | Todd Andrews, Jeffrey Pincus, Brian Flynn | S-curves typically address only "known unknowns." They're built upon an analysis of cost risk and uncertainty" element by element. But, they fail to address the ambiguity of the curve itself, or the degree of confidence that its proffered probabilities are actually correct. This confidence factor, of course, is hard to measure. The "unknown unknowns," in other words, are not explicitly captured since their probabilities are a mystery. This research fills the gap in knowledge. | 2017 | Risk |
| Being Certain About Uncertainty, Part 1 | Andy Prince | Doing cost risk analysis is hard because we don't really know what a good cost risk analysis looks like. In this paper we will explore the challenges to doing good cost risk analysis and discuss ways to know if your cost risk analysis is any good. We will also examine the phenomena of extreme cost growth and lay the groundwork for future work. | 2017 | Risk |
| Integrated Cost-Schedule Risk Analysis Improves Cost Contingency Calculation | David T. Hulett | The main benefits of integrated cost-schedule risk analysis are improvement of the estimates of cost contingency and identification of the main risks to cost for mitigation purposes. The main focus will be on estimating the cost contingency needed and identifying risks to cost, which may be independent of schedule or indirectly due to schedule risk. New simulation software including iterative risk prioritization will be used to illustrate these points. | 2017 | Risk |
| Using Quantum Theory and Monte Carlo Multiverse to Manage Risk | Nathan Eskue | Need to better predict your EAC? Tired of being surprised? Great! Not an expert on quantum theory or the multiverse? No problem! This hands-on session will give a high level view of how quantum theory can help us understand our EAC by viewing it as a multiverse. We'll then explore an Excel model that takes your EAC and uses Monte Carlo analysis to reveal all possible futures. Attendees get a copy of the model! | 2017 | Risk |
| Software Projects Estimation & Control | Alain Abran | This talk presents 1) an outline of the design of the '2nd generation' COSMIC Function Point method for measuring a functional size of software; 2) Industry evidence of its versatility & value in software project benchmarking & estimating in business, real-time, infrastructure, component, mobile, IoT, cloud software, via traditional & agile methods; 3) how it can be used for early and rapid sizing at estimation time; 4) how size measurement can be automated with very high accuracy. | 2017 | Software |
| Bottom Up Methods of Estimating Software SE/PM and Non-DCTI Costs | James Black | Systems Engineering/Program Management (SE/PM) and additional non-Design, Code, Test, and Integration (Non-DCTI) activities performed during software development efforts are often significant and drive estimates of total project costs. Yet, cost estimates often omit the detailed research and analysis needed to adequately model SE/PM & Non-DCTI costs. This brief will present bottom up methods useful for understanding and estimating these costs and share analysis of recent SE/PM & Non-DCTI data. | 2017 | Software |
| Reliable Non-Design, Code, Test, and Integration Cost Relationships | Jeremy Goucher, Brittany Staley | Software cost estimates require ratios derived from historic cost reports for non-design, code, test, and integration (NDCTI) cost elements. Since NDCTI accounts for as much as 50% of the estimate, a comprehensive historical data set is critical to ensuring an accurate estimate. The authors have recently analyzed over ten years of actual cost data from DoD command and control systems to develop a new set of NDCTI ratios. The results also bring new insight into "fixed" versus "variable" cost. | 2017 | Software |
| Objective SLOC: An Alternative Method to Sizing Software Development Efforts | Andrew Kicinski | Equivalent Source Lines of Code (ESLOC) is the basis of methodology used by many organizations for collecting and estimating software development costs. Selecting ESLOC parameters requires insight into the software reuse. Too often data collectors are unable to verify the appropriateness of the assigned ESLOC parameters and validate their implementation. This paper examines the drawbacks of ESLOC, and presents an alternative and more objective method to estimating software development effort. | 2017 | Software |
| Software Effort Estimation Models for Contract Cost Proposal Evaluation | Wilson Rosa, Corinne Wallshein | This study will introduce regression models and benchmarks for predicting software development effort using input variables typically available at contract bidding phase. The analysis is based on 200 DoD projects delivered from 2004 to 2016. The first set predicts effort using initial software requirements along with peak staff, application domain, and other categorical variables. The second set predicts effort specifically for agile software development using data from 16 agile projects. | 2017 | Software |
| Software Size Growth | Marc Russo, Wilson Rosa | Software cost estimating relationships often rely on software size growth percentages. Actual delivered source lines of code (SLOC) may be predicted with categories of early code estimates such as new, modified, reuse, and auto-generated SLOC. Uncertainty distributions will be presented to represent growth by code category for use in cost modeling. | 2017 | Software |
| Analysis of Software Cost Estimation Models Using Normalized, Stratified Data | Dan Strickland | Software Resources Data Reports (SRDR) are the DoD's effort to improve cost estimation efforts by collecting metrics on software programs. Existing SRDRs were normalized in a manner consistent with previous efforts and productivities were analyzed using three popular software cost estimation models (SLIM, SEER, and COCOMO). Calculated values were tested against model thresholds and the results were tested for accuracy. Findings indicate some models outperform others and schedule impacts results. | 2017 | Software |
| The Journey from "Bottom-up" to Predictive Modelling BOE | Lori Saleski | Predictive Cost Modeling has the potential to save significant money over conventional bottom-up bidding with better accuracy. This paper outlines a process to implement predictive cost analytics in an organization that been dependent on bottom-up estimating for decades. Walk with us down the path discussing implementation, change management, adoption and stakeholder challenges. Explore architecting the solution for maximum leverage at ROI. | 2017 | Software |
| Rapid Cost Estimation for Storms Recovery Using Geographic Information Systems | Rolando A. Berraos-Montero, Steven M. F. Stuban, Jason Dever | The present study introduces a new approach to estimate the recovery costs of public property in the aftermath of a storm, by integrating geographic information systems. Estimating recovery costs for a disaster is a current concern for emergency responders. This work focuses on applying economic indicators, population, and storm event tracking to geographic information systems for rapidly estimating recovery costs. | 2016 | Journal of Cost Analysis and Parametrics |
| Review of Quantitative Methods for Designing Availability-Based Contracts | Amir Reza Kashani Pour, Peter Sandborn, Qingbin Cui | Understanding the total life-cycle cost is an essential part of all sustainment contracts. Sustainment constitutes 70% or more of the total life-cycle cost of safety-, mission-, and infrastructure-critical systems. For many types of systems, availability is the most critical factor in determining the total life-cycle cost of the system. To address this, availability-based contracts have been introduced in the governmental and non-governmental acquisitions space (e.g., energy, defense, transportation, and healthcare). | 2016 | Journal of Cost Analysis and Parametrics |
| Tooth-to-Tail Impact Analysis: Combining Econometric Modeling and Bayesian Networks to Assess Support Cost Consequences Due to Changes in Force Structure | Bradley C. Boehmke, Alan W. Johnson, Edward D. White, Jeffery D. Weir, Mark A. Gallagher | Current constraints in the fiscal environment are forcing the Air Force, and its sister services, to assess force reduction considerations. With significant force reduction comes the need to model and assess the potential impact that these changes may have on support resources. Previous research has remained heavily focused on a ratio approach for linking the tooth and tail ends of the Air Force cost spectrum and, although recent research has augmented this literature stream by providing more statistical rigor behind tooth-to-tail relationships, an adequate decision support tool has yet to be explored to aid decision-makers. | 2016 | Journal of Cost Analysis and Parametrics |
| Forecasting the Unit Price of Water and Wastewater Pipelines Capital Works and Estimating Contractors Markup | Rizwan Younis, Rashid Rehan, Andre J. A. Unger, Soonyoung Yu, Mark A. Knight | Municipalities and water utilities need to make realistic estimates for the replacement of their aged water and wastewater pipelines. The two main objectives of this article are to present a method to forecast the unit price of water and wastewater pipelines capital works by investigating inflation in their construction price, and to quantify the markup that contractors add to bid a project price. The Geometric Brownian Motion model with drift is used for investigation. | 2016 | Journal of Cost Analysis and Parametrics |
| Generalized Degrees of Freedom | Shu-Ping Hu | Two popular regression methods for the multiplicative-error model are the Minimum-Unbiased-Percent Error and Minimum-Percentage Error under the Zero-Percentage Bias methods. The Minimum-Unbiased-Percent Error method, an Iteratively Reweighted Least Squares regression, does not use any constraints, while the Minimum-Percentage Error under the Zero-Percentage Bias method requires a constraint as part of the curve-fitting process. | 2016 | Journal of Cost Analysis and Parametrics |
| Development of Unit Cost Indices and Database for Water and Wastewater Pipelines Capital Works | Rashid Rehan, Rizwan Younis, Andre J. A. Unger, Brendan Shapton, Filip Budimir, Mark A. Knight | The objective of this work is to develop a unit cost database and index for water and wastewater pipelines capital works, and estimation in their construction cost. This was accomplished by analyzing tender summaries and progress certificates from the cities of Niagara Falls and Waterloo, Ontario, Canada, that span the period from 1980 to 2008, as well as using data from RS Means construction cost database. | 2016 | Journal of Cost Analysis and Parametrics |
| Using Pre-Milestone B Data to Predict Schedule Duration for Defense Acquisition Programs | Christopher A. Jimenez, Edward D. White, Gregory E. Brown, Jonathan D. Ritschel, Brandon M. Lucas, Michael J. Seibel | Accurately predicting a realistic schedule for a defense acquisition program is a difficult challenge considering the inherent risk and uncertainties present in the early stages of a program. Through the application of multiple regression modeling, we provide the program manager with a statistical model that predicts schedule duration from official program initiation, which occurs at Milestone B, to the initial operational capability of the programs deliverable system. | 2016 | Journal of Cost Analysis and Parametrics |
| Dynamics of New Building Construction Costs: Implications for Forecasting Escalation Allowances | Michael T. Dugan, Bradley T. Ewing, Mark A. Thompson | Construction projects often require multiple years to complete and the costs of supplies, materials, and labor may increase substantially during a projects time span. As a result, construction contracts often include an escalation clause to account for cost increases. This article examines the time-series properties of new building construction costs using several producer price indexes. | 2016 | Journal of Cost Analysis and Parametrics |
| Balancing Expert Opinion and Historical Data: The Case of Baseball Umpires | Ricardo Valerdi | Many decisions benefit from situations where there exist both ample expert opinion and historical data. In cost modeling these may include the costs of software development, the learning curve rates for specific manufacturing tasks, and the unit rate costs of operating certain products. When making forecasts we are often faced with the decision to base our estimates on either expert opinion or historical data. | 2016 | Journal of Cost Analysis and Parametrics |
| Using Robust Statistical Methodology to Evaluate the Cost Performance of Project Delivery Systems: A Case Study of Horizontal Construction | Dares Charoenphol, Steven M. F. Stuban, Jason R. Dever | The objective of this study is to demonstrate the application of the bootstrapping M-estimator (a robust analysis of variance [ANOVA]) to test the null hypothesis of means equality among the cost performance of the three project delivery systems (PDS). A statistical planned contrast methodology is utilized after the robust ANOVA analysis to further determine where the differences of the means lie. | 2016 | Journal of Cost Analysis and Parametrics |
| Multiproduct Cost-Volume-Profit Model: A Resource Reallocation Approach for Decision Making | Gabriel Soares Zica Bergo, Bruna Hoffmeister Lucas, Vinicius Amorim Sobreiro, Marcelo Seido Nagano | This work addresses the problem of reallocating productive resources to maximize proat. Most contributions to the topic focus on developing or improving the Cost-Volume-Proat model to obtain solutions that provide an ideal mix of products before the data is given. In particular, some algorithms are available for the problem, such as the ones proposed by Kakumanu and Shao and Feng. | 2016 | Journal of Cost Analysis and Parametrics |
| NASA Commercial Crew Cost Estimating - A Look at Estimating Processes, Challenges and Lessons Learned | Lance Cole, Rick Battle | To support annual PPBE budgets and NASA HQ requests for cost information for commercial crew transportation to the International Space Station (ISS), the NASA ISS ACES team developed system development and per flight cost estimates for the potential providers for each annual PPBE submit from 2009-2014. This paper describes the cost estimating processes used, challenges and lessons learned to develop estimates for this key NASA project that diverted from the traditional procurement approach and used a new way of doing business. | 2016 | Government Processes |
| Health CERs - Using an interdisciplinary approach to estimating the cost of caring for Canada's veterans | Erin Barkel, Carleigh Malanik | Reports from the Veterans and the National Defence and Canadian Forces Ombudsmen have brought attention to the prevalence of mental illness among veterans. However, there are no published estimates for the cost of treating this growing population. A key obstacle to this research is that the relevant data is dispersed across several government organizations. We fill this research gap by leveraging actuarial and epidemiological studies of the veteran population to create new cost estimating relationships. | 2016 | Government Processes |
| Federal Shared Services: The future Shared-First and challenges facing the Federal Shared Services Providers (FSSPs) and their customers | Ruth Dorr, Marilyn Fleming, Sarah Pope | This paper will discuss the progress being made in the "Shared-First" approach for sharing of administrative services. Federal Agencies are being asked to execute their missions to ever increasing standards in a resource constrained environment. Administrative shared services have been identified as a way to reduce redundancy and improve efficiency in Federal business solutions. The identification of shared services as a Presidential cross-agency priority along with the designation of several providers for administrative shared services are recent developments in the movement toward shared services within the Federal government. | 2016 | Government Processes |
| Lessons learned from applying cost capability curves on an Air Force AoA | Robert Georgi | New direction from Air Force leadership established a requirement for the presentation of life cycle cost versus capability tradeoff analysis for all AoA final reports. This presentation examines the approach that was used for a recent AoA that was a pilot program for the Cost Capability Curve process. When appropriately implemented cost capability analysis can provide significant advantages. There are lessons learned from attempting to implement this approach. We will also discuss examples where this approach does not apply. | 2016 | Government Processes |
| Mining for Cost Estimating Relations from Limited Complex Data | Mark Jacobs, Shawn Hayes | NASA's robotic Earth and space science missions cover a diverse group of projects addressing complex science objectives that include challenging implementation approaches. Progress applying Principle Component Analysis techniques covering project management, systems engineering, mission assurance, integration & test, and spacecraft subsystems is described. Supporting data analysis efforts include a large detailed set of technical and programmatic input candidates that are analyzed to identify the primary spacecraft cost drivers. | 2016 | Government Processes |
| Manufacturing Assembly Plan (MAP) Tool: Bridging the Gap between Performance and the Construction Process | Richard Lee, Haitham Ghannam, Edward Walsh | This presentation explains the four module build plan for the VIRGINIA Class Submarine (VCS) program and provides an in-depth look into labor hours by Section 1/2A, Section 2B/5, Section 6/7, Section 8/9, Final Assembly and Test (FAT), and Post Shakedown Availability (PSA). Further understanding these concepts will not only allow cost estimators and program managers to better understand the current construction build strategy, but it will also aid in better aligning cost estimates with reality at the deck plate. | 2016 | Government Processes |
| It Ain't Easy Being Green... Sustainable Manufacturing with an eye on Cost Avoidance and Stewardship | Kammy Mann, Jessica Boatwright | The DoD, one of the largest energy consumers in the world, has committed to sustainability. An important element of sustainable weapon systems acquisition currently not under the purview of the DoD's strategic plan is sustainable manufacturing. This paper will leverage current research and industry best practices to identify the impact of sustainable manufacturing on cost avoidances relating to human health and environmental benefits. Recommendations will also be provided for complying with the FAR sustainability requirements. | 2016 | Government Processes |
| Business Case Analysis Wizard | Lauren Nolte, Cassie Capots, Marcie Pfeuffer | This paper describes the purpose and design of a "Business Case Analysis (BCA) Wizard" that was created for the Air Mobility Command (AMC) Enterprise Learning Office (ELO). As the value of BCAs within the learning transformation at AMC has grown, ELO identified the need to develop a tool that allows "non-cost analysts" to develop high-quality BCAs. This "BCA Wizard" guides users through a standard process of conducting a BCA via a user-friendly interface. | 2016 | Government Processes |
| Masters in Cost Estimating and Analysis | Dan Nussbaum, Greg Mislick | This presentation provides an update on the all-Distance Learning Masters Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS). | 2016 | Government Processes |
| Methods Used in Pricing and Conveying NASA Assets for Use by Commercial Programs | James Roberts, Torrance Lambing | Different agreement types including EUL's, Use Permits, Land Leases and others are used to convey NASA assets no longer needed by programs such as Space Shuttle and construction of the Space Station. By law, some situations prohibit NASA from pricing facilities in competition with the local market. Costs are estimated either via market-based or actual cost methods, according to circumstances. This paper will explain and clarify how these costs are estimated and pricing is established. | 2016 | Government Processes |
| ATC Zero: Estimating the True Impact of the Chicago Control Center Fire | Kellie Scarbrough (Wutzke) | For seventeen days, the Chicago Air Route Traffic Control Center went offline after a contractor set fire to the intricate communications network that controls some of the busiest airspace in the country. Workers scrambled around the clock for seventeen days to restore functionality to the center. This paper seeks to determine the true economic impact of the event, including the costs of time, operations, and environmental impacts. | 2016 | Government Processes |
| Where Does Your Cost Estimate Go? | Emily Stafford, Christopher Metz | You have spent several months developing your cost estimate and presenting it to the customer. Now what? This paper will track what happens to your cost estimate after it has been delivered to the customer. It will follow the estimate through the budget formulation process, ultimately becoming a part of the President's Budget. It will also discuss the cost estimator's role during the budget process, and the influence of the budget process on future estimates. | 2016 | Government Processes |
| Facilitating Predictive Cost Analytics via Modelling V&V | John Swaren | We are often asked to "fine tune" estimating software to unique situations. That "calibration" effort is key to Verification & Validation of parametric models. In this presentation, we show an example of using Predictive Analytics to develop a method to predict a Software-estimating model's inputs as a function of Key Performance Parameters (KPPs). We demonstrate how a calibrated knowledgebase can best be used to V&V an estimating approach reflective of past and future requirement metrics. | 2016 | Government Processes |
| Hyperbolic Discounting and Defense Acquisitions | Eric Lofgren | Cost estimation has largely been approached as an engineering problem which can be solved using data and techniques. While these are undoubtedly invaluable considerations, the broader production process is an inherently open-ended system subject to human interactions which are ill-defined and non-deterministic. This paper explores how behavior and institutions affect cost outcomes through the avenue of hyperbolic discounting, a time-inconsistent model of personal valuations. It will be argued that evidence for hyperbolic discounting in both government agents and contractor management have negative effects on cost estimates and will proffer some solutions based on commitment schemes. | 2016 | Government Processes |
| Analysis of Alternatives from a Cost Estimating Perspective | Kirby Hom, Kenny Ragland | Subsequent to the Weapon Systems Acquisition Reform Act of 2009, there have been other significant DoD policy changes that require Service sponsors to conduct an Analysis of Alternatives (AoA) prior to Milestone A. "About three-quarters of a program's total life cycle cost is influenced by decisions made before it is approved to start development" [GAO-09-665]. From a cost estimating perspective, a successful AoA rests with the study approach, overall costing methodologies, key insights from trades analyses, and lessons learned. | 2016 | Government Processes |
| The Navy Modernization Program: Estimating the Cost of Upgrading AEGIS Guided Missile Cruisers | Jeremy Goucher | The Navy is well into its 20 year, $16B (CY10$) plan to modernize 84 AEGIS warships. This cost estimate covers eleven of the planned ships and accounts for maintenance and upgrades to the ship and the combat systems. The estimate leverages recent contract and shipyard performance data and interviews with engineers, resulting in a detailed study and recommended cost savings initiatives. The methods and data in this estimate will assist ship modernization cost efforts across the fleet for the foreseeable future. | 2016 | Government Processes |
| Cost Estimating Challenges in Additive Manufacturing | Joe Bauer, Patrick Malone | Additive manufacturing possibilities are virtually endless: from rapid prototyping, to printing food, to recreating human tissue. This presentation highlights the challenges of estimating 3D printing processes and offers some solutions to the estimator. Challenges include: amortizing non-recurring tooling across multiple programs, capturing software-centric nature of 3D printing, printing speed, and printing media limitations. This presentation highlights the use of existing cost estimating tools and areas for future development. | 2016 | Methods & Models |
| Long Term Impact of Ship Concepts on Operating and Support Cost Affordability | Eric Buller, Stuart Merrill, Bryan Miller | Achieving affordability can be challenging especially once a program enters full rate production. Developing accurate Operating and Support (O&S) cost estimates that reveal tradeoff impacts and sustainment risks provides decision makers with affordability analysis during key acquisition events. We intend to demonstrate through analysis the impact of legacy and new ship concepts on O&S costs and program affordability. We will examine the cost tradeoffs for each ship concept in each O&S cost element. | 2016 | Methods & Models |
| The Peculiar World of Sole Source Contracting; Maybe There is a Better Way | Sandy Burney, Shawn Larson | In 2014, 21% of U.S. Government contracts were sole source, accounting for 31% of total contracted value. They typically require negotiation and certification of all cost estimates, which can take up to 18 months. This light-hearted presentation will explore the peculiar world of developing and negotiating sole source estimates, including differences from budgetary and competitive estimating, as well as offer suggestions for achieving fair and reasonable sole source prices for less time and money. | 2016 | Methods & Models |
| Migration of Microsoft Excel Tools to Next Generation Platforms: "Can You Hear the Footsteps" | Jeremy Eden | Analysis tools are popular solutions for extended deployments and at the same time an acceleration of platform migrations is occurring to keep up with rapidly changing technology and end user demands. This paper provides specific examples of platform migration challenges, resulting failures, and how they can be best addressed. It will also provide specific best practices for ICEAA member take-away that can be followed to help ensure smoother transitions across platforms. | 2016 | Methods & Models |
| Generalized Degrees of Freedom (GDF) | Shu-Ping Hu | Minimum-Percentage Error/Zero-Percentage Bias (ZMPE) method is commonly used for multiplicative-error models. But ZMPE users do not adjust degrees of freedom (DF) for constraints included in the regression process. This generates misleading ZMPE CER fit statistics and underestimates the CER error distribution variance. Hence, ZMPE results are incomparable with those derived without constraints. This paper details why DF should be adjusted and proposes a Generalized Degrees of Freedom measure to compute fit statistics for constraint-driven CERs. | 2016 | Methods & Models |
| Bottom Up Methods of Estimating Depot Level Repairables | Tim Lawless, James Black, Dave Goldberg | Depot Level Repairables (DLR) costs comprise a significant share of total Operations and Support costs as they capture the scope of repairing and/or replacing failed hardware items. This paper will provide detailed descriptions of bottom up methods to estimate DLR costs using programmatic technical baseline documentation and acquisition costs as well as supply system requisitions and repair price data sources. Emphasis will be placed on producing data-driven analysis to inform cost estimates and estimate customers. | 2016 | Methods & Models |
| Seven Degrees of Separation: The Importance of High-Quality Contractor Data in Cost Estimating | Crickett Petty | The popular notion that any two people are linked to one another on average via a chain with "six degrees of separation" is based on a relatively small sample. Using massive data sets, researchers have since discovered that the average degrees of separation is closer to seven. This highlights the need for complete and thorough data in any analysis, and cost estimating in particular. Despite its importance the collection of high-quality contractor cost data is challenging. Processes for meeting these challenges are presented. | 2016 | Methods & Models |
| Interviewing Subject Matter Experts | Melissa Teicher | As estimators, we often have to obtain technical information from Subject Matter Experts (SMEs) in order to provide thorough and accurate analysis. But sometimes, getting the information you need is easier said than done. This paper discusses the steps for interviewing SMEs in order to make the most of their time and your time. It explores the benefits of building a cooperative relationship of mutual respect and the "dos" and "don'ts" of SME consultations. | 2016 | Methods & Models |
| The Impacts of Design Change on Reliability, Maintainability, and Life Cycle Cost | Andreas Viberg, Oskar Tengo | When making decisions about a design change to a technical system, it is essential to understand how it affects reliability and maintainability, as that directly impacts both overall performance and cost of operations and maintenance throughout the life cycle. LCC analysis based on modeling and simulation of relevant scenarios is an effective way to gain that understanding. This paper presents an approach for such analysis and illustrates its successful application in a recent project. | 2016 | Methods & Models |
| A Prime Contractor's View Using JCL Tools and Techniques | Chip Woods | For several years now, Lockheed Martin has been integrating cost and schedule data into a single model to perform integrated analysis. This paper will discuss Lockheed Martin's JCL process, the tools used to develop JCL curves, and how this analysis can be used to shape and steer the program-including modeling funding constraints. | 2016 | Methods & Models |
| Class List of Characteristics: Changing the Landscape of Estimating by Removing Bias | Paul Breon | CLOC (Class List of Characteristics) is a coding system wherein each digit of the code indicates one of the cost drivers for the product/activity. CLOC gives the ability to "Call a thing by what it really is" and creating a universal language that can be utilized for any purpose. CLOC does NOT include Bias like quantity, cost, etc. These things are used in conjunction with the code to do any kind of analysis needed. | 2016 | Methods & Models |
| Achieving Affordability through high fidelity ROM cost estimates | Eric Buller | Achieving affordability can be challenging especially if early decisions are made without proper cost estimates. Developing high fidelity ROM cost estimates and identifying capability tradeoffs during the requirements generation phase can be critical to the success of achieving long term affordability. We intend to demonstrate through analysis that preforming high fidelity ROM cost estimates with capability tradeoffs allow decisions makers to informed choices on mission requirements that achieve affordability for future programs. | 2016 | Methods & Models |
| Building an Integrated Aerospace Electronics Development Parametric Model from the Ground Up | David Bloom | This paper presents the results of a multiple year effort to create a business specific set of integrated parametric cost models for complex electronics hardware development in such a way that all disciplines and activities that participate in that development process have a piece of the model and a stake in the data collection that drives the model. This "grounds up" approach provides that added benefit of cost allocation guidance once the development is approved. | 2016 | Parametrics |
| Using Historical Cost Reports to Derive System Specific Factors | Jocelyn Tague, William Banks | Non-Recurring Engineering (NRE) is often a cost driver for new, complex systems and must be given high attention during Milestone B cost estimating. Our team developed an approach for developing system specific Cost Estimating Relationship (CER) using actual costs for from combat weapon systems with shared capabilities, resulting in a credible defensible, yet flexible approach that can be replicated for a wide range of estimates. | 2016 | Parametrics |
| Improving Parametric Software Cost Estimation Through Better Data Collection | Bob Koury | In 2010 PRICE Systems LLC in conjunction with PEO STRI agreed to work together to improve the data being collected to support software development cost estimation in the command. The paper will compare and contrast the experiences of two other collection efforts with those of original effort. The paper will identify core findings which may be used universally to improve the quality, quantity, and applicability of data being collected. | 2016 | Parametrics |
| A Regression Method for Sparsely-Populated Datasets | Daniel Barkmeyer | A method is presented for including sparsely-populated independent variables in a regression analysis without introducing subjectivity or statistical bias. The method is shown to reproduce the result of a traditional regression method for a fully populated dataset, and to converges to the same result on a sparsely-populated dataset as that dataset decreases in sparseness. Application of the method to develop a new, empirically-derived model for spacecraft testing schedule duration will also be presented. | 2016 | Parametrics |
| Improvement Curves: An Early Production Methodology | Brent Johnstone | Learning slope selection is a critical parameter in manufacturing labor estimates. Incorrect ex ante predictions lead to over- or understatements of projected hours. Existing literature provides little guidance on ex ante selection, particularly when some actual cost data exists but the program is well short of maturity. A methodology is offered using engineered labor standards and legacy performance to establish a basic learning curve slope to which early performance asymptotically recovers over time. | 2016 | Parametrics |
| The NRO CER Analysis Tool | Donald MacKenzie | The NRO Cost and Acquisition Assessment Group (CAAG) uses its CER Analysis Tool, CERAT, to inform CER development. CERAT analyzes CERs for sensitivity to influential data points (IDPs). The impacts on estimated costs for a target data point when data points are removed are the primary outputs. The changes in recorded CER constants when data points are removed also help in assessing overall CER "stability." CERAT also provides several other aids to CER developers. | 2016 | Parametrics |
| A Parameter Driven World | Jeffrey Platten | Parametric estimating is gaining acceptance. DFARS requires DOD contractors to maintain an estimating system. Parameters are the characteristics, features, or measurable factors that define a product or program. What makes a good parameter? What are the pros and cons of parametric cost estimating? How do you develop and maintain a parametric estimating database/model? What does the future hold, beyond just estimating? How do you gain parametric control over touch labor, support labor, and parts costs to improve affordability? | 2016 | Parametrics |
| Generating a Semi-Automated "Similar To" Basis Of Estimate from a Complex Parametric Hardware Cost Model for Antennas | Danny Polidi, David Bloom | This paper discusses the development of a "Similar-To" Basis Of Estimate (BOE) generation tool used in conjunction with a Complex Parametric Antenna Cost Model. Starting with the parametric model based on quantifiable sizing mechanisms which are designed to quickly and accurately calculate the "top-down" cost for all engineering and operations disciplines and functions required for antenna development and test, cost is generated, "similar to" programs are identified, and a BOE can be auto-populated. | 2016 | Parametrics |
| Macro-parametrics and the applications of multi-colinearity and Bayesian to enhance early cost modelling | Dale Shermon | This paper will consider the spectrum of parametric cost models from cost estimating relationships (CER) to micro and finally macro-parametric models. This will lead to a description of the Family of Advanced Cost Estimating Tools (FACET) parametric suite of models and their top-down capability to estimate costs incorporating the application of multi-colinearity and bayesian techniques. | 2016 | Parametrics |
| A CER Development Process for Spares Estimating | Cheryl Wilson | This presentation shows the direction of a group of Spare Parts Estimators and how we are advancing our statistical skills and CER development processes. It shares the progress in our CER development journey attained through education, research, analysis and teamwork. | 2016 | Parametrics |
| Essential Views of the Integrated Program Management Reports (IPMRs) | Thomas Coonce, Glen Alleman | During this session, the authors will describe key views of the IPMRs and other data that will allow a government stakeholder to a) understand the cost, schedule and technical status on applicable contracts, b) investigate the nexus of performance problems, and c) observe likely future problems. The presentation represents a "synthesis" of existing traditional earned value management metrics and other metrics designed to provide an integrative picture to assist in keeping the program "green". | 2016 | Program Management |
| Meeting Fiscal Constraints: the Evolving Role of Performance Measurement | Steve Green, Kevin Davis, Kurt Heppard | Performance measurement tools can reflect the strategic intent of management and help them prioritize initiatives to meet strategic objectives. In fiscally austere times, the Balanced Scorecard can help organizations address these tough decisions. Applying concepts from the Balanced Scorecard to cost analysis and budgeting, this paper develops an integrated aligned system of strategic goals, performance metrics, and cost parameters at a university facing pressure to manage costs. There may be lessons learned for the DoD. | 2016 | Program Management |
| The graph is always greener on the other side: examples of how to interpret and fix bad visual representations of data | Stephen Ketcham | In my role performing independent reviews of estimates, I've seen a quality spectrum of different types of graphs, charts, and forecasts. These visuals summarize key points and remain in decision-makers minds, often providing the convincing point to win funding approval for the project. Unfortunately they can also be poorly crafted, employ scare tactics or even frame the data in a misleading way. This presentation provides bad examples of visuals, and discusses possible improvements. | 2016 | Program Management |
| Seeing What We Need to See vs. What We Hope to See: Visualizing Integrated Cost and Schedule Data for Earned Value Analysis | Brian Leach | When applied properly, data visualization can serve analysis by showing us what is otherwise difficult to see (and not always what we hope to see). With the emergence of the UN/CEFACT XML standard, integrated cost and schedule data can be shown in exciting new ways. This paper will examine and explore innovative new ways to show integrated cost and schedule data which have practical applications in earned value management system analysis. | 2016 | Program Management |
| Different Incentives in Government Contracting | Jennifer Leotta | This paper shows stock prices are used as the primary incentive for executives of publicly traded government contractors rather than profit sharing contracts. Looking at the reaction of companies' stock prices to public and private sector news, this paper shows stock prices do not capture "bad news" for both the public and private sector customers compared to overall market indicators; and would make poor contract incentives. Finally, alternate methods of contract incentives will be explored. | 2016 | Program Management |
| Agile Project Management Controls: Issues with EVM and a Viable Alternative | Omar Mahmoud, Blaze Smallwood | As more government software programs adopt agile practices, program managers are trying to find effective ways to monitor and control cost and schedule. The challenge is that traditional project control techniques, like EVM, are not well suited to deal with the requirements fluidity and new terms/metrics that come with agile projects. This paper will describe the inherent issues with using EVM on agile projects and offer a recommendation for a more effective solution. | 2016 | Program Management |
| The Psychology of Cost Estimating | Andy Prince | Research into human psychology has yielded amazing findings into how we process information and how we use information to make decisions. Cost estimators can use psychology and behavioral economics to improve not only our cost estimates but how we communicate and work with our customers. By understanding how our customer's think, and more importantly, why they think the way they do, we can have more productive relationships and greater influence. The end result will be better decisions by the decision makers. | 2016 | Program Management |
| Cost Management | Georges Teologlou | Cost Management has a key role in project management and is the management trend due to the actual financial and economic worldwide constraints. Cost Management solution was built to be in line with the recommendations of the GAO (US Government Accountability Office) that stated that a large number of Defense projects are at risk of experiencing cost overruns and schedule delays. GAO found that the two major causes of project failure were first an inaccurate initial estimate and secondly poor subsequent cost control. | 2016 | Program Management |
| Using Total Lifecycle Cost as the Basis for Proposal Cost Evaluation | Ryan Timm | This paper discusses benefits and obstacles of using the TLCC of an entire system instead of the bid price for the purpose of proposal cost evaluation. An example acquisition of a space vehicle (SV) ground system explains the impact to TLCC of portions of the ground system developed by the SV manufacturer, SV operations, external software interfaces, government furnished IT infrastructure, and legacy sustainment until transition. Using TLCC allows higher-level requirements and enables innovative architectures. | 2016 | Program Management |
| A Maturity Analysis of the American and European Cost Community | Mark Gilmour | The Cost Engineering Health Check (CEHC) is a cost capability benchmarking tool used to identify strengths and weaknesses within organizations costing functions. In 2014 the authors of this paper conducted a CEHC at the ICEAA conference in Denver. That workshop used e-voting to anonymously gather the views of the audience in attendance. This paper presents a summary of these results and contrasts them against data gathered from European workshops, providing a snapshot of the international costing capability. | 2016 | Program Management |
| Presenting Estimates | Melissa Teicher | Decision-makers rely on cost estimators to guide them in making the most educated judgments on what is best for the program or organization. A thorough and well-organized presentation can make the difference between an easy decision and a difficult one. This paper discusses how public speaking skills contribute to an effective estimate presentation. It also delivers tips and tricks on how to make an estimate presentation more visually pleasing and easier to understand. | 2016 | Program Management |
| Using Stochastic Optimization to Improve Risk Prioritization | Eric Druker | Minimizing the cost of complex programs is critical for government agencies trying to meet their missions in today's fiscal. Today, identifying cost-savings measures is a manual procedure where an analyst must make an educated guess as to what risks to mitigate. This process is manual and slow. This paper will outline a methodology for using the emerging field of stochastic optimization to automate the identification of cost-savings measures on complex programs. | 2016 | Risk |
| Why Are Estimates Always Wrong: Estimation Bias and Strategic Misestimation | Dan Galorath | The biggest source of estimation error usually comes from people, either by accident or strategically. This is a disaster because viable estimates are core to project success as well as ROI determination and other decision making. Most people don't know how to estimate. Those who estimate are often optimistic, full of unintentional bias and sometimes strategically mis-estimating. This paper discusses the issues of estimation bias and strategic mis-estimation as well as how to mitigate these issues. | 2016 | Risk |
| A "Common Risk Factor" Method to Estimate Correlations between Distributions | Marc Greenberg | A "common risk factor" method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared leading to an estimated 'intersection' of pair of common risk factors. This intersection is then assumed to be a proxy of correlation. | 2016 | Risk |
| Data-Driven Guidelines for Correlation of Cost and Schedule Growth | Sidi Huang, Mark Pedigo, Chris Shaw | When running Monte Carlo simulations, one of the most difficult aspects to account for is correlation between the independent variables. Ignoring correlation generally reports overly optimistic analysis results; industry standard suggests that when no additional insight is available, a default value of 0.2 can be used. This study analyzes historical NASA program data to develop data-driven guidelines for correlation. Moving forward, this study can be replicated to other industries and organizations with critical schedules. | 2016 | Risk |
| Risk Mitigation: Some Good News after the Cost / Schedule Risk Analysis Results | David Hulett | The results of your cost-schedule risk analysis say you are likely to be overrun on both schedule and budget. Are you are out-of-luck? No. Do not forget the risk mitigation phase. Risk mitigation typically proceeds from a prioritized risk list developed from analysis of the risk and uncertainty-loaded schedule. Complete mitigation of risks is not usually feasible, and uncertainty is usually deemed to be irreducible. Some good news comes from the risk mitigation actions. | 2016 | Risk |
| A Mathematical Approach for Cost and Schedule Risk Attribution | Fred Kuo | The level of sophistication in cost estimate has progressed substantially in the last decade. This is especially true at NASA where, spearheaded by the Cost Analysis Division (CAD) of NASA Headquarters, cost and schedule risks in terms of confidence level have been codified. In this paper, the author attempts to employ the portfolio approach in calculating risk attribution; first applying to a portfolio of cost risks and extend the same concept to schedule risks analysis. | 2016 | Risk |
| Triangular Distributions and Correlations | Jennifer Lampe | Triangular distributions are often used in estimating cost risk because the math is relatively simple. The concept of an estimate having a lower limit, an upper limit, and a mode is easy to understand. We show the math behind triangular distributions and correlations. What does it mean to say that two risks are 30% correlated? How do risk analysis programs handle triangular distributions and correlations? How can you use random numbers in Excel to generate a Monte Carlo simulation for any defined triangular distribution? | 2016 | Risk |
| The Impact of Selected Assumptions and Core Tenets on Schedule Risk Assessment Results (A Progressive Model Comparison) | James Quilliam | In the quest to ensure the sound representation of Schedule risk assessment (SRA) simulations this case study will provide a progressive model comparison of schedule risk assessment assumptions and core tenets. The elements of this approach will focus on the: methodology and tools; the progressive assumptions and core tenets applied; conclusions and lessons learned for practitioners. This will greatly enhance the understanding and confidence that leadership and project teams have in the schedule risk assessment results. | 2016 | Risk |
| Wine as a Financial Investement | Michael Shortell, Adiam Woldetensae | The purpose of this presentation is to look into wine as an investment and analyze its characteristics as a financial asset. The goal here is to look into vintage wines through time, analyze its profitability and predict how investing in wine could be a good source of diversification. Our objective is to detect the types of wines that are a good source of investment and their associated risk. We aim to identify a good investment portfolio for a wine collector in order to maximize returns and minimize the associated risk. | 2016 | Risk |
| Beyond Correlation: Don't Use the Formula That Killed Wall Street | Christian Smart | Risk models in cost estimating almost exclusively rely on correlation as a measure of dependency. Correlation is only one measure (among many) of stochastic dependency. It ignores the effect of tail dependency, when one extreme event affects others. This leads to potential outcomes that do not make sense such as a program with a large schedule overrun but no cost overrun. This method has been widely blamed for the financial crisis a few years ago. The copula method is introduced as a way to overcome these deficiencies. | 2016 | Risk |
| Connecting the Dots: Integrating the risk cube with the POE | Christina Snyder | DoD guidance instructs program managers to include risk and uncertainty analysis in their cost estimates. Program managers also maintain a risk cube that tracks known program risks and assigns a likelihood and consequence. Risks, regardless of if they are programmatic, technical, or schedule defined, will have an effect on the overall program cost. This paper investigates the linkage between identified risks and POE cost and shows how to translate the risk cube into dollars. | 2016 | Risk |
| Methodology for Integrating Risk mitigation Activities into JCL analysis | James Taylor, Justin Hornback | The intent of this whitepaper is to outline a an improved methodology for capturing risk mitigation activities for both cost and schedule into a Joint Confidence Level (JCL) analysis and assessing mitigation effectiveness. | 2016 | Risk |
| The Continual Pursuit of the One True Software Resource Data Reporting (SRDR) Database | Remmie Arnold, Peter Braxton | This paper will demonstrate the ability for one database to store Software Resource Data Report (SRDR) source data in its original form, as submitted by WBS element and reporting event. This database allows evaluations, interpretations, and annotations of the data, including appropriate pairing of Initial and Final reports; mapping of SLOC to standard categories for the purposes of determining ESLOC; normalization of software activities to a standard set of activities; and storage of previous assessments. | 2016 | Software Estimating |
| Agile Estimation Using Functional Metrics | Thomas Cagley | Agile methods have matured and are now being integrated into many different approaches to the development of software. Estimation has been problematic for all methods, agile to plan based and therefore it tends to be a lightning rod for experimentation and synthesis such as is being described in this paper. Agile Estimation Using Functional Metrics has presented a path for integrating the discipline found in functional metrics with the collaborative approaches found in agile parametric estimation. | 2016 | Software Estimating |
| Put some SNAP in your estimating model | David Lambert | An effective estimating model is dependent upon having the capacity to accurately size the problem domain for the solution being developed. Function Point Analysis is a widely accepted industry and ISO standard for sizing functional requirements; however, it does not take into account the sizing of non-functional requirements. Learn how the newly released Software Non-Functional Assessment Process (SNAP) can provide a more complete and accurate assessment of project size. | 2016 | Software Estimating |
| Forecasting Software Maintenance | Alea DeSantis | Providing a defensible estimate for software maintenance can be complicated. By overlaying historical data with lifecycle events one can derive event driven factors and use trend analysis to forecast future Software Problem Reports. Topics addressed are: common-cause variation, special-cause variation, severity level, complexity level, and decay rate. This method moves away from using a general defect rate calculation to a more complete product-specific calculation. | 2016 | Software Estimating |
| Effective use of Function Points for Analogous Software Estimation | Daniel French | Analogous estimation is often used to estimate software projects, especially in the early phases, because it is believed to be quicker and easier than other techniques. However, many organizations do not employ this technique effectively and correctly. One pitfall is improperly applying the technique & basing these estimates on Software Lines of Code (SLOC). This presentation will discuss common mistakes made when using analogous estimates as well as how the use of IFPUG function points can greatly improve this estimation technique | 2016 | Software Estimating |
| NASA Software Cost Estimation Model: An Analogy Based Estimation Method | Jairus Hihn | Software development activities are notorious for their cost growth. While there has been extensive work on improving parametric methods there is very little focus on the use of models based on analogy and clustering algorithms. In this paper we will describe the methodology being used in the development of a NASA Software Cost Model using data mining clustering algorithms and evaluate its performance by comparing it to estimates from COCOMO II, a calibrated COCOMO II, liner regression, and K-nearest neighbor models. | 2016 | Software Estimating |
| Estimating Agile Software Development | Bob Hunt | The use of Agile Software Development continues to grow in large and small Federal programs. These programs often utilize Story Points as a key size metric. This presentation will roovideman update on Agile Cost Estimating techniques and emphasize how to,utilize Story Points as a key size metric. | 2016 | Software Estimating |
| How Much Does Software Maintenance Cost? | Cheryl Jones, James Judy, John McGarry | To accurately estimate, budget, allocate, and justify the resources required to meet evolving mission and service affordability requirements across the system life-cycle we must be able to answer the fundamental question: How much does software maintenance cost? This presentation provides a summary of the findings from an ongoing U.S. Army study that is focused on developing improved software maintenance estimation processes, cost relationships, and associated models. | 2016 | Software Estimating |
| Exploring DoD Software Growth: A Better Way to Model Future Software Uncertainty | Nicholas Lanham | This presentation highlights trends and estimating relationships derived from the April 2014 SRDR data. This analysis provides the estimating community with several new variables to better model future software growth uncertainty. In addition, the analysis includes growth by contract type, domain, and program type as well as introduces a multivariate model to predict final hours. All of the relationships included within this slide set have been derived from the best 400+ data points, were approved by DCARC, and are DoD-specific. | 2016 | Software Estimating |
| Considerations in Data Center Migration Life Cycle Cost Estimating: A Joint Information Environment (JIE) Perspective | Dennis McKeon | The potential benefits of consolidating data centers has led to the creation of initiatives such as the DoD Joint Information Environment (JIE). The goal is to reduce IT infrastructure costs by consolidating system support functions, improve hardware utilization, and lower energy and facility costs. This paper is from the perspective of an organization migrating to a consolidated data center. It will describe the migration steps, cost estimating approach, benefits, risks, and organizational challenges from their perspective. | 2016 | Software Estimating |
| Early Phase Software Cost and Schedule Estimation Models | Wilson Rosa | Software cost estimates are more useful at early elaboration phase, when source lines of code and Function Points are not yet available. This study introduces effort estimation models using functional requirements as a size predictor and evaluates development schedule using estimated effort based on data from 40 military and IT programs delivered from 2006 to 2014. Statistical results show that estimated functional requirements and peak staff are significant contributors and that estimated or actual effort is a valid predictor of development duration. | 2016 | Software Estimating |
| Software Size Growth | Corinne Wallshein | NCCA will present their findings including cost estimating relationships (CERs) of software size growth using selected subsets. CERs will be compared using initially estimated parameter distributions against CERs using software size. Other parameters include requirements, staff size, hours, and duration. As the 2014 update of the Joint Agency Cost and Schedule Risk and Uncertainty Handbook advises use of standard error for uncertainty bounds, NCCA will compare bounds for recommended CERs, by selected data subsets. | 2016 | Software Estimating |
| At the Intersection of Technical Debt and Software Maintenance Costs | Arlene Minkiewicz | Ward Cunningham introduced the technical debt metaphor in 1992 to ease business leaders understanding of the cost of making poor decisions during software development. While this author has seen many different definitions of technical debt, it is basically an indication of the amount of 'should fix' issues that remain in production code. This paper reports on an on-going research effort to understand technical debt and its implications on software maintenance cost and effort estimation. | 2016 | Software Estimating |
| Software size measures and their use in software project estimation | Harold van Heeringen | Using the right size measure is extremely important for software cost engineering purposes. However, the number of software size measures is growing rapidly and there is a lot of confusion in the industry with regard to the methods that should be used. The advantages and disadvantages of the major software size measures are discussed and recommendations are given on what size measures should be used to estimate the different types of software systems. | 2016 | Software Estimating |
| The ABCs of the UCF: A look at Section L and Section M | Jennifer Leotta | This paper will examine the government cost analyst's role in the acquisition/contracting process. Specifically, it will look at the Uniform Contract Format (UCF) and how to create effective language for Section L that provides clear directions for offerors and includes all information necessary to complete a thorough proposal evaluation. It will also look at developing Section M language for varying contract types (cost plus, fixed price, etc.) that is consistent with the Federal Acquisition | 2016 | Acquisition |
| A Weapons Acquisition Case Study: Cost Overruns and Schedule Slips | Randy Bowen, David L. Peeler, Jr. | This story is true. Certain details have been omitted to "stump the audience" and protect the guilty. The trials and tribulations of an acquisition program cost overrun are explored for persistent lessons and future applications. This saga includes materiel difficulties, workforce issues, congressional interest, funding and schedule perturbations. The aim is to educate, as the persistence of fundamental acquisition problems and the insignificance of acquisition reforms are framed. | 2016 | Acquisition |
| GAO-COFC Concurrent Bid Protest Jurisdiction: Are Two Fora Too Many? | James Linick | The Government Accountability Office (GAO) and the Court of Federal Claims (COFC) are the bid protest fora where a disappointed offeror may challenge an agency's procurement decision. Critics claim that concurrent bid protest fora, which have differences in jurisdiction, standards of review and remedies, creates inefficiency in the procurement system, e.g. serial protests. This presentation examines whether concurrent bid protest fora are healthy for the US federal procurement system. | 2016 | Acquisition |
| Early Stage Cost Estimating for Radars and Sensors | Jeremy Goucher | The majority of costs for programs are locked in even before a program enters production, which makes accurate early stage cost estimates vital for effective resource management and program success. This study proposes a method for analyzing combat system cost prior to a complete requirements description. The data set includes old, new, big, small, ground, sea, air, domestic, and foreign systems. The result is a model that requires limited data and is widely applicable. | 2016 | Acquisition |
| Federal Agency Independent Government Cost Estimate | Cassandra M. Capots, Lauren Nolte, Lavanya Yeleswarapu | To achieve mission success in today's constrained fiscal environment, federal agencies must establish a fair and reasonable cost baseline to make informed decisions. This baseline is commonly called an Independent Government Cost Estimate (IGCE). We have developed a customizable process that forms a starter kit for any agency that wishes to quickly and effectively perform IGCEs. We discuss best practices, lessons learned, and implementation tips for both DoD and civilian agencies. | 2016 | Acquisition |
| Observation of Production Rate Effect observed in Historical Weapon Systems Cost | William Banks, Timothy Lawless | Production Rate Effect has been observed in several weapon systems program's historical cost data during the procurement phase of the acquisition life-cycle. The main contribution of this paper is to illustrate real life observations of production rate effect and the economies of scale shown when purchasing end items from major defense contractors. | 2016 | Acquisition |
| Analyses of Alternatives for Space Systems | R. Alex Wekluk | This presentation provides an overview of recent experiences performing satellite analyses of alternatives, including technical and cost trades. Our team developed a scalable mechanism to perform satellite sizing using physics and stochastic modeling. Once modeled, an array of costing methods is applied to develop repeatable and defensible estimates. The team made great strides in presenting the large number of nuances to decision makers. | 2016 | Business Case Analysis |
| Training Cost Analysts, a Cohesive Pedagogical Framework for Success | Kammy Mann, Danielle Lainez | How do we train cost analysts? This paper will take a critical look at peer-reviewed academic research regarding pedagogy, or the method of teaching. Leveraging existing knowledge, current best practices and resources available, a framework for teaching and training will be proposed to the cost community. By establishing a consistent methodology for educating our professional workforce we can ensure that new members of the cost analysis field have the tools and skills to succeed. | 2016 | Business Case Analysis |
| 3D Printing in the DoD; Employing a Business Case Analysis | Nicole Santos, Richard Shea, Robert Appleton | 3D Printing is the family of technologies that enables users to produce items on demand from CAD files or 3D scanning. The potential benefits to military logistics include cost savings, weight reduction, and responsiveness to the warfighters' needs. To demonstrate and measure the benefits in the Department of Defense (DoD), a rigorous Business Case Analysis (BCA) will identify benefits and challenges to implementation including evaluating its costs, risks, and benefits. | 2016 | Business Case Analysis |
| Fixing the Flawed Nature of DoD BCA's | William G. Williamson | The government is not a business that is profit-oriented. Monetary business metrics as a means of determining what items to consume are inherently flawed for the government since metrics such as Return on Investment's assume the organization is a provider of the goods or services, not a consumer. A better approach for the government would be to measure the areas that the government most values, i.e. schedule and requirements. Using those areas as anchors, a best value metric can be developed, pointing the government to smarter acquisition decisions. | 2016 | Business Case Analysis |
| Developing a Dynamic Expense-Volume-Profit Model to Determine Break-Even point | William Kent | Analysts must be able to determine when a program will be profitable. This insight is used to drive contract negotiation, pricing, and strategy. Many factors need to be evaluated: expenses, revenue, and volume of sales. These factors must be determined based on limited historical data and extrapolated over a time horizon. This paper introduces a framework for analyzing these factors using learning curves, regression, and financial statement analysis to determine break-even. | 2016 | Business Case Analysis |
| ROS vs. ROC: Government Can Lower Price by Raising Fee | Sandy Burney | Many Government Contractors measure profitability using Return on Sales (ROS), which includes Fee in the denominator. The FAR regulates fee percentages or Return on Cost (ROC). In negotiating sole-source contracts, this metric difference can result in the Contractor negotiating a higher risk adjusted cost and price to offset lower fees offered by the Government. The briefing provides a mathematical example of the tradeoff between Cost and Fee for a CPFF contract. | 2016 | Business Case Analysis |
| Bridging the Gap between Capital Investment Valuation in Private & Public Sector | George Bayer, Jr. | The public and private sectors have different valuation techniques and considerations when evaluating capital budgeting decisions. Understanding the difference between private and public sector capital investment analysis (discounted cash flow, cost/benefit, tax impacts, stakeholders) helps decision-makers make better informed decisions. In the cost estimating community, understanding investment distinctions makes us better stewards of information and more effective Finance professionals. | 2016 | Business Case Analysis |
| Begin With a Joke? What John Oliver Can Teach Us About Communicating Cost Estimates | Erin K. Barkel | Are you having a hard time holding management's attention when you explain the results of your estimates? It's time to change your approach. Consider this: millions of people tune-in and click to watch John Oliver talk about cost estimates gone wild, and they stay tuned-in for 20 whole minutes. What can you learn from a comedian about explaining cost estimates? Maybe you should start with a joke. | 2016 | Business Case Analysis |
| Liars! Why Cost Estimators and Budgeters Inflate and Underestimate Costs! | Travis Winstead, Ann Hawpe | Projects and programs regularly suffer from poor cost estimating techniques that result in insufficient budgets and unanticipated cost overruns. To issue a contract or set a realistic budget, the cost analyst must verify and validate the mountain of cost information and determine a fair and reasonable price for a contractor's goods and services. This paper will discuss strategies to assist program analysts in reviewing cost estimates and strategies used to support Government negotiations. | 2016 | Management, EVM & Scheduling |
| Fiscal Sustainability of Canada's National Defence Program | Peter Weltman | This report provides PBO's estimate of the fiscal gap between the status quo budget allocations and the cost of sustaining Canada's status quo national defence forces. Parliamentarians may wish to examine scenarios that will reduce or eliminate the gap between the cost of maintaining the current force structure and the amount of funding being allocated to paying for it. | 2016 | Management, EVM & Scheduling |
| The Performance Metrics Model and Study | Derreck Ross, Haitham Ghannam, Richard C. Lee | The Performance Metrics Model and Study (PMMS) is a comprehensive technique for selecting a performance metric TCPI. The PMMS uses historical data of completed products to determine the best EVM estimation technique for similar products that are in progress. The result is an auditable and repeatable approach that increases the objectivity of an Estimate at Completion. This approach can be applied across any program that has historical data. | 2016 | Management, EVM & Scheduling |
| Cost Consistency and Completeness as an Impossible Exercise | David L. Peeler, Jr. | In previous papers, Hilbert's Problems were used to propose and revisit Hilbert's Problems for Cost Estimating. This paper employs Godel's theorem w/respect to Hilbert's application onto cost. What can we learn about ourselves as estimators and where can we exert greatest impact with our estimates? Using Godel's two theorems of undecidability as catalyst, we explore the effect and utility of exacting math and other notions on cost estimates specifically and programmaitics generally. | 2016 | Management, EVM & Scheduling |
| Preventing Program Management Pitfall Using Portfolio Estimating | Nick Morales, Christopher Dewberry | Estimating the effort, time, and resources needed to complete project activities is one of the most challenging tasks that project managers must face. Along with the inherent uncertainty associated with managing these activities, additional management uncertainty occurs when there are multiple incremental efforts associated with that project. In order to handle the complication of managing several program increments, each with their own budget lines, portfolio estimating is a method that helps to provide program managers with a fully comprehensive overview of the resource requirements across multiple projects. Program managers can use the portfolio estimate as a tool to help make informed management decisions at the appropriate level and identify how those decisions impact the entire program. | 2016 | Management, EVM & Scheduling |
| Putting Schedule Quality Checks to the Test | Eric M. Lofgren | Analysts often use the 14-Point Assessment for Schedule Quality as a benchmark for determining the overall reliability of a schedule. But how much of the variation in schedule error can be explained by the results of the 14-Point check? This paper will use actual defense contract data to find the correlates of schedule reliability, measured using both the accuracy and the timeliness with which the schedule slip is predicted. | 2016 | Management, EVM & Scheduling |
| Provide Agile Analysis to Stakeholders Using Category Tagging Approach | Eric Hong | It is becoming more and more important for organizations to use data analytics for improved decision-making to create a competitive advantage and/or operate efficiently. This paper will describe the challenges in creating and implementing a strategy. It will identify potential reporting requirements and how to leverage existing work products as input into a standard structure feasible to perform analysis for different stakeholders. The proposed solution will increase the accuracy and agility of an organizations' analysis. | 2016 | Management, EVM & Scheduling |
| Applying Earned Value to Agile Development Programs | Bob Hunt, Michael Thompson | Agile Software and Agile Development continue to dominate the Federal acquisition arena. This presentation will address current trends in estimating and applying Earned value to Agile programs. | 2016 | Management, EVM & Scheduling |
| EVM's Potential for Enabling Integrated Cost-Risk Management | David R. Graham, Bryn Turner | Integrating the cost estimating, EVM analysis and technical risk management disciplines to realize comprehensive cost-risk management has proven elusive. Recently developed EVM cost-risk tools offer the potential to successfully enable integrated cost-risk management. The paper will illuminate the process of how integrated cost-risk management can be realized through cooperation of cost estimating, EVM analysis and technical risk management disciplines and briefly describe these new EVM cost-risk tools. | 2016 | Management, EVM & Scheduling |
| Conducting Root Cause Analysis for Should Cost Management | Nancy A. R. Droz, Mr. Robert L. Hernandez | Root-cause analysis (RCA) is typically performed after a technical failure or after a program anomaly is found. However, the technique is key to the development and selection of effective initiatives to reduce program cost. RCA is a critical aspect of successful Should Cost Management execution. The presentation will provide a how to approach of applying root cause analysis during the evaluation of program data to develop more effective costs reductions. Lessons learned will be presented. | 2016 | Management, EVM & Scheduling |
| Organization Cost Estimation as a Human Intensive Systems Engineering Problem | David Bloom, Robert Wright | This paper will compare and contrast the difference between Intensive and Non-Intensive Human Interactive Systems. Further, this paper will investigate the role of Governance, Culture, Process and Tools (GCPT) in each of the 2 systems, Intensive and Non-intensive. Finally, this paper will apply the GCPT Framework to the system of Organizational Cost Estimation and provide examples of results that Raytheon has seen in its effort to improve the accuracy, affordability and accountability of the current bidding/estimation process | 2016 | Management, EVM & Scheduling |
| Beyond CSDRs: Collecting Contractor Cost Data for Detailed Cost Estimates | Tim Lawless | Quality cost estimates are based on actual costs, and for the Department of Defense, usually are acquired through Cost and Software Data Reports (CSDRs). However, CSDRs may not be available, complete or consistent for quality analytical purposes. This paper examines the collection and use of detailed contractor actual costs for estimates, such as with Priced Bills of Material and invoices, and their analytical benefits, perhaps supplanting CSDRs as the preferred data source for cost estimators. | 2016 | Methods & Models |
| Crew and Space Transportation Systems Cost Model | Richard Webb | As part of the MSFC Engineering Cost Office's new Program Cost Estimating Capability (PCEC) suite of cost estimating tools and capabilities, we are developing the Crew and Space Transportation Systems Cost Model (CASTS), a new, unique cost model for use in estimating crew and space transportation systems. This paper will provide an overview of the capabilities, estimating approach, historical database, and key features of CASTS as well as plans for future improvements. | 2016 | Methods & Models |
| The Space Situational Awareness | James Smirnoff, Brennen Woodruff | Today there are over a thousand operating satellites orbiting the earth providing a variety of critical functions such as communications, navigation, weather sensing, reconnaissance, and astronomy. As the space environment becomes more crowded, Space Situational Awareness (SSA) is increasingly important. This paper describes a new tool called the Space Situational Awareness (SSA) Cost Model which provides an objective and data-driven framework to estimate the cost of ground-based electro-optical components of future SSA architectures. | 2016 | Methods & Models |
| Estimating the Costs of Future Air-to-Ground Weapons | Tom Sanders | The weapons acquisition community has been acquiring air-launched weapons in a slowly-evolving manner for years. While technologies experience great leaps forward, acquisition strategies evolve at a slower pace. The AF Research Lab is evaluating revolutionary weapons designs that facilitate newly-emerging acquisition strategies...thereby presenting real challenges to cost estimators. This paper documents techniques for estimating costs of those future weapons. | 2016 | Methods & Models |
| Challenges in Estimating the Development Cost of Microcircuit Technology | Christopher Price, Gurney Thompson | Electronic Technology continues to change at a rapid pace. Significant changes in the level of integration of electronic components dictate that circuit designs using these parts be redesigned every three to five years. Estimating the development cost of these new designs can be challenging. This paper will discuss these challenges in more detail, and describe some methods and tools available to rapidly and accurately estimate the development, production and O&S cost of these new technologies. | 2016 | Methods & Models |
| Workload-Adjusted Labor Cost Model | Michael Mender | Government estimators typically use Forward Pricing Rate Agreements (FPRAs) to estimate future labor costs. FPRAs make assumptions about future workload, and can be inaccurate as realized workload often differs from projections. To enable understanding of this uncertainty, the Naval Center for Cost Analysis (NCCA) developed a lagged regression model that predicts future overhead costs as a function of workload, allowing NCCA to quickly and accurately assess the impact of Navy procurement decisions on acquisition programs. | 2016 | Methods & Models |
| Using Physics Based Reliability Methods for Should Cost Estimates | Patrick K. Malone | Value based should cost methodologies supporting Better Buying Power tenants can help realize significant MDAP cost savings over the life cycle. The use of Physics Based Reliability techniques in program development is described to meet affordability goals, identify cost drivers affecting reliability, availability and maintainability of weapon systems, and demonstrate PBR advantages over the classical reliability growth model with actionable plans to maintain technological superiority. | 2016 | Methods & Models |
| Delphi Methodology for Cost Estimating | Cole J. Kupec, II | The Delphi methodology can be a practical and valuable tool for cost estimators. This paper explores its proper implementation based on current research to include group selection, question formulation, statistical analysis, and the survey process. Special attention is paid to tailoring the Delphi method to cost estimating applications. A case study is presented for example purposes. The history of the Delphi method as well as modifications to the traditional Delphi method are also presented. | 2016 | Methods & Models |
| Beyond Anderlohr: An Alternate Approach To Estimating Production Breaks | Brent Johnstone | Estimating the cost impacts of production breaks has long been problematic. Use of the Anderlohr method is widely accepted, but requires a significant degree of estimating judgment and can produce substantially different answers based on the individual user's assumptions. This paper suggests an alternate empirical methodology based on recent research on organizational learning and forgetting. | 2016 | Methods & Models |
| Estimating the Cost of the Aegis Flight III Test Platform | Jeremy Goucher | The Flight III upgrade to the Aegis Destroyer program brings unparalleled new capabilities to the surface Navy fleet which will require in depth testing prior to deployment. For live fire testing, the Navy is considering four unique options to procure, install, and test the required systems on an unmanned vessel. This paper discusses the available data, methodology, and results of the cost estimate. The paper will also discuss some of the technical challenges associated with each option. | 2016 | Methods & Models |
| Footprints in the Sand: A Conversational Approach to Basis of Estimate | Frank R. Flett | The Basis of Estimate (BOE) is the mechanism by which the cost estimator or cost proposal preparer hopes to convince the evaluator or reviewer that the method he/she took in preparing the estimate was "reasonable" (not the same as "correct"). This paper will present a technique of writing BOEs as if they were a part of a conversation with the evaluator/reviewer, who is, after all, a human being who will early on develop a distinct impression of the "goodness" of the end result. | 2016 | Methods & Models |
| Department of Defense Commercial-Off-The-Shelf | Heather L. Brown | NCCA partnered with DoD Enterprise Software Initiative (ESI) to periodically receive commercial-off-the-shelf (COTS) software orders with the intent of improving software cost estimating. As of today, more than 250,000 orders from the Army, Navy, Marine Corps, and other federal entities have been recorded. Together, these represent over six billion dollars in sales. NCCA examines this data and categorizes various product descriptions as licensed products, maintenance and support, or services in order to present statistical trends. | 2016 | Methods & Models |
| Naval Sea Systems Ship Command Logistics Requirements Funding Summary Tool | Jeremy Eden | The United States Navy has focused on planning and estimation of operations and support costs for programs. A task force created by Secretary Stackley (Assistant Secretary Navy Research, Development & Acquisition) to identify actions for achieving this goal recommended creating a logistics tool similar to that developed by the United States Marine Corps (USMC). This paper demonstrates the tool, development approach, cost estimating methodologies, interface, and deployment to the next generation | 2016 | Methods & Models |
| How Predictive Analytics is Improving Parametric Cost Estimation | Anthony A. DeMarco | Predictive analytics is defined as the use of data, statistical algorithms and machine-learning techniques to identify the likelihood of future outcomes based on historical data. This paper will explore the history and parallels of parametric estimating and predictive analytics. It will highlight how cost management professionals can use current predictive analytics tools, methods and talent to improve their estimates and their success rates. | 2016 | Parametrics |
| Multiple Regression: Prediction & Confidence Intervals Demystified | Stacy M. Dean | We all know the value of prediction/confidence intervals as a measure of the certainty in the best fit line for a regression equation; however, once we get past bivariate regressions how to actually calculate these statistics gets a little murky. This paper attempts to de-mystify multivariate confidence/prediction interval calculation as well as provide some fun facts on the origin and history of these infamous intervals. | 2016 | Parametrics |
| The Collinearity Kill Chain: Detect, Classify, Localize, Neutralize | Brian Flynn, Adam James | This paper examines the issue of multicollinearity from the fresh perspective of a statistical warfighter. A kill-chain of steps is offered to defeat this illusive foe. Heuristics, chi-squared tests, t-tests, and eigenvalues are the Aegis, Predator, Bradley, and Ma Deuce 50-caliber statistical equivalents used to detect, classify, localize, and neutralize the problem. | 2016 | Parametrics |
| Dialing for Dollars: Improving Cost Estimating Accuracy Through Data Mining | Brittany Holmes, James Glenn, Christian Smart | This paper shows the benefits of leveraging obligation and expenditure data from DoD Enterprise Resource Planning (ERP) Systems to supplement cost data from CCDRs or IPMRs. One benefit is that cost trends and funding patterns can be analyzed with more precision because the ERP data is updated real-time vice monthly or annually. Another benefit is that more comprehensive CERs can be created because ERP systems capture all cost vice only contracts with cost reporting CDRLs. | 2016 | Parametrics |
| Develop PRESS for Nonlinear Equations | Shu-Ping Hu | Predicted residual sum of squares (PRESS) and Predicted R2 statistics are commonly used in regression analysis outside the cost analysis industry to determine (1) how well the model predicts new observations and (2) if the model includes more independent variables than necessary. Currently, these two leave-one-out statistics are only available for ordinary least squares. This paper develops these two statistics for nonlinear equations and demonstrates their value in practical applications. | 2016 | Parametrics |
| Including Escalation in Cost Estimating Relationships | Christopher Jarvis | Prior to developing a Cost Estimating Relationship (CER) the data should be normalized for effects not explained by the assumed CER model. This typically includes the effects of inflation and/or escalation which has recently generated considerable discussion within the DoD. In this paper we demonstrate two common CERs models and augment them to include an escalation term. Discussion will be made regarding the reliability of the solutions of the augmented models. | 2016 | Parametrics |
| Dangers of Parametrics | Andy Prince | What if our models are not solving our estimating problems, but instead are the source of our problems? The purpose of this paper is to address this question. We will look at what a cost model is, and what it isn't. We will examine how cost models appeal to our need for certainty and helps us create a good story for our cost estimate. We will take a look at a simple cost model to see what can go wrong when we trust the model over trusting the data. Finally, we will identify specific actions | 2016 | Parametrics |
| Development of an Additive Manufacturing Cost Model | F. Gurney Thompson, III, Grady Noll | This paper discusses our research and development of a cost model for additive manufacturing (AM), also known as 3D printing. Working with academia and multiple AM companies, we investigated cost drivers, collected cost and technical data, derived AM CERs, and developed an approach for including AM components within a larger parametric cost estimate. This paper discusses the research approach, data collection, results, cost model implementation, and future work. | 2016 | Parametrics |
| X-Planes to X-Wings ~ Developing a Parametric Cost Model | Steve Sterk | In today's cost-constrained environment, NASA needs an X-Plane database and parametric cost and schedule model that can quickly provide rough order of magnitude predictions of cost and schedule from initial concept to first flight of potential X-Plane aircraft. This paper takes a look at the steps taken in developing such a model and reports the results. The challenges encountered in the collection of historical data and recommendations for future database management are discussed. | 2016 | Parametrics |
| Integrating Cost Estimating and Data Science Methods in R | Josh Wilson, Laura Barker | Data science is a growing, interdisciplinary field about the extraction of knowledge from data. It emphasizes the utilization of statistics and advanced analysis techniques to answer questions in a well-documented and reproducible manner. R is a free, open source programming language that is popular among data scientists. This presentation discusses the benefits of using R to supplement traditional cost estimating and data analysis practices. | 2016 | Parametrics |
| AoAs the Right Way | Amanda Wilson, David Macdonald, Mariam Uzunyan | The follow-ons to two large space acquisition programs from the Space and Missile Systems Center completed the AOA processes last year. They were set apart from past AoAs and AoA-like studies with the leadership and involvement of OSD CAPE and the AFCAA teams, with broad community involvement strengthening the teams. This change resulted in benefits, challenges and lessons learned; various visual tools were created to highlight risks, opportunities, and investment strategies to senior leaders. | 2016 | Parametrics |
| Cost Estimating Relationship Development Handbook | John Fitch, Adam James, Alfred Smith | Developing appropriate and well-defined parametric relationships is a foundational requirement for Cost Estimators. Current training for analysts focuses on various ordinary least squares regression (OLS), techniques, but ignores logical decision sequences employed by good analysts applying OLS and omits methods beyond OLS. The CER Handbook includes these techniques in a flow-chart format and guides analysts along each step in developing a parametric relationship. | 2016 | Parametrics |
| Cost Risk for Firm Fixed-Price Contracts | David Biron, Christian Smart | The terms "cost risk" and "firm fixed-price contracts" seem contradictory. By design the contractor bears all risk under a firm fixed-price (FFP) contract. In spite of this, overruns often occur, and contractors have recourse to a Request for Equitable Adjustment (REA) when cost grows beyond the contract value. We present statistics on cost overruns for FFP contracts, some of which are significant, and show how to model risk for such contracts. | 2016 | Risk & Uncertainty |
| Modeling Prediction Intervals Using Monte Carlo Simulation Software | James Black, Qing Qing "Q" Wu, | The use of a prediction interval (PI) is a simple method of quantifying risk and uncertainty for a Cost Estimating Relationship (CER) derived from an Ordinary Least Squares (OLS) regression. Yet, few cost estimators implement PIs in their estimates despite their frequent use of CERs. This presentation will provide a step-by-step tutorial for modeling a PI for an example CER using Monte Carlo Simulation software and will identify the beneficial impact on the coefficient of variation (CV). | 2016 | Risk & Uncertainty |
| Introducing RIFT to Protect Your Uncertain Schedule | Nicholas DeTore, Peter Frederic | There are industry-accepted methods for allocating cost risk and uncertainty analysis results to detailed WBS elements; schedule results cannot be allocated the same way since duration behaves differently than cost. We present an innovative solution to this issue. The RIFT algorithm calculates a threshold date, for any task or milestone, that if exceeded puts the probabilistic project finish date in jeopardy. RIFT provides a new tangible metric to guide decision makers. | 2016 | Risk & Uncertainty |
| Risk vs. Uncertainty - What's the difference? | Melvin R. Etheridge, Jr. | After years of applying "risk", many in the profession still do not understand the difference between program risk and cost uncertainty in cost estimates. Determining confidence levels goes beyond simply applying the effects of programmatic risks. Simply put, risks have a probability of occurrence, while we know that quantitative inputs have some degree of uncertainty and must be treated as random variables. This paper explores analytical methods in Excel-based cost models using Monte Carlo s | 2016 | Risk & Uncertainty |
| A 'Common Risk Factor' Method to Estimate Correlations between Distributions | Marc Greenberg | A 'common risk factor' method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared to estimate common risk factor "intersections" that are, in turn, proxies of correlation. | 2016 | Risk & Uncertainty |
| How Regression Methods Impact Uncertainty Results | Boyan Jonov, Shu-Ping Hu, | Log transformation and weighted least squares are commonly used to develop multiplicative error cost estimating relationships. We objectively compare the two regression techniques and provide a sound defense of log-linear ordinary least squares (LOLS) to counter arguments against its use. We then demonstrate how uncertainty modeling varies substantially based on regression method. Lastly, we establish criteria to use so that LOLS leads to a justifiable uncertainty assignment. | 2016 | Risk & Uncertainty |
| The Zone System of Uncertainty Analysis | Jeff McDowell | This paper presents a methodology for selecting cost uncertainty distributions and their dispersion inspired by the photography Zone System. The method seamlessly addresses five groupings of uncertainty across a sliding complexity scale. The distribution shapes and their parameters are drawn from patterns developed from the AFCAA Cost Risk and Uncertainly Metrics Manual (CRUAMM) body of work. | 2016 | Risk & Uncertainty |
| Cloud Total Ownership Costing: Considering the Technologies, Costs and Benefits | Dan Galorath, Steven Woodward | Cloud abounds with promises of drastically reduced costs, which are all attractive. However, Cloud computing can be ineffective in terms of technology, affordability and total cost of ownership in many situations. The assumption that cloud technology reduces government costs is often flawed. The benefits from cloud computing relating to time-to-market, savings, innovation and agility should be analyzed, providing balanced perspectives for the cloud opportunities presented. | 2016 | Software & IT |
| Data Collection for Agile Projects | Blaze Smallwood | Agile software development projects produce a unique set of metrics, such as points and velocity, that can provide interesting insight into project progress. However, collecting this data requires specialized mechanisms, since no established standards exist, like SRDRs. This paper will discuss the types of data that are useful to collect for agile software development projects and mechanisms that have been used to collect them for several government projects. | 2016 | Software & IT |
| Apples and Oranges: a Presentation and Analysis of Results of Cloud Cost Calculators and Rate Cards | Daniel J. Harper | A recent effort for an Army customer examined over a dozen calculators and rate cards for estimating storage and hosting costs for cloud applications. This presentation will provide an overview of several calculators and tools, guidance for cost estimators on interpreting IT-centric inputs, and a discussion of similarities and variation in results. We will also present a cloud complexity plotter which provides a visual tool for explaining cloud cost and complexity drivers. | 2016 | Software & IT |
| A Primer on Agile Software Development for Cost Analysts | Eric Hawkes, Cole J. Kupec II | As Agile software development continues to become more common, cost estimators are compelled to adapt their methods to the Agile approach. This paper begins by explaining what Agile really is and how it differs from the common waterfall methodology. It also presents two case studies that detail how Agile was successfully implemented in a DoD acquisition program. Lastly, the paper proposes four innovative cost estimating methods for an Agile environment. | 2016 | Software & IT |
| The Cost of Cyber | Ann Hawpe, effrey Voth | A digital revolution is changing the nature of warfare and the cyber domain is now a top priority for all MDAP and MAIS programs. As the continued advancement of cyber warfare changes the future of land, air, sea, and space over the next decade, the cost community will need to consider the challenges facing defense programs and how to estimate the cyber security requirements within the construct of a reformed acquisition process. | 2016 | Software & IT |
| Maturing the Economic Aspects of Agile Development in the Federal Government | Jennifer Manring | Agile development has emerged in the commercial domain as a leading software development methodology, with growing adoption across federal agencies. This new approach for software development challenges traditional acquisition areas including cost estimation, economic analysis, and system engineering. This research provides a deeper understanding of the economic implications of agile development for the Federal Government and provides insights needed to understand the value proposition of an agile development environment compared with a traditional waterfall environment. | 2016 | Software & IT |
| Using Predictive Analytics for Cost Optimization Across Cloud Workloads | Zachary Jasnoff, David A. Cass | As organizations move to the cloud, estimating costs proves difficult, presenting many challenges. Once estimated, cloud costs must be optimized taking into account workloads. Organizations failing to estimate cost based on workloads will either underestimate or over spend on cloud services, often with disastrous results. A framework for optimizing costs (spanning IaaS, PaaS, and SaaS) and determining the cost/benefit between public clouds, private clouds and hybrid clouds will be presented. | 2016 | Software & IT |
| SURF Process Summary & Initial Findings: A Deeper Focus on Software Data Quality | Nicholas Lanham, Corinne Wallshein, Ranae P. Woods | As a result of the joint Software Resource Data Report Working Group (SRDRWG), several recommendations were generated by the supporting, multi-agency SRDRWG members to the Cost Leadership Forum. One of which led to the development of the first joint-agency SRDR Validation and Verification (V&V) guide as well as the SRDR User Review Function (SURF) subgroup. This study summarizes the results and data-quality improvement areas identified from 319 V&V comments generated by the SURF team. | 2016 | Software & IT |
| Cloud Solutions - Infrastructure, Platform or Software - Where should you go? | Arlene F. Minkiewicz, Ashley Hoenigke | Cloud computing technologies promise to create some new and interesting challenges for the cost estimating community. How does one determine the effort and costs associated with migration of existing applications to the cloud? What challenges do the various cloud solutions (IaaS vs. PaaS,vs. SaaS) present? This paper presents a case study outlining the migration activities and cost implications for migrating the same application on each of the three cloud solution models. | 2016 | Software & IT |
| Process-Related Effort and Schedule Estimating Relationships for Software Cost Estimating | Nichlas Lanham, Corinne Wallshein, Wilson Rosa | The Naval Center for Cost Analysis will present comprehensive, updated findings of software size growth factors, effort estimating relationships (EER), and schedule estimating relationships (SER) with subsets of Department of Defense Computer Software Configuration Item records. This presentation focuses on software size (new, modified, reused, auto-generated, and total code). Subsets include maturity, application and super domain, language, contract type, and operating environment. | 2016 | Software Sizing |
| Estimating Small Scale Software Integration Efforts with a Large Scale Code Base | Joe Bauer | Estimating the cost and effort for testing and integration is often one of the most challenging aspects of software estimation. Integration effort depends greatly on the components under evaluation. When relatively small capabilities are introduced into large existing code bases, the integration impact due to the existing code may surprise stakeholders even when requirements are reduced. This presentation highlights software integration modeling for a major defense acquisition program. | 2016 | Software Sizing |
| Alternatives to SLOC in the Modern Environment | Jeremy Goucher | Source Lines of Code (SLOC) based estimating relies heavily on subject matter expert input. However, viable alternatives to SLOC are few. By mapping deliverables to functionality and functionality to effort, the authors propose an alternate technique with reduced reliance on SLOC. Agile development and some recent innovations represent alternatives as well. The authors will present an overview of the topic with examples, required data inputs, and estimating methodology. | 2016 | Software Sizing |
| Agile and Function Points: A Winning Combination | Daniel B. French | Many users of Agile have an incorrect belief that projects do not need to adhere to proper project management practices. However, it doesn't advocate getting rid of metrics or PM processes. The most critical aspect of software development is software size. Function Points can be effectively used with Agile development and this presentation will demonstrate the benefits of using FP in many aspects of project management including estimation, schedule and product backlog. | 2016 | Software Sizing |
| Cost Element Structure for Cloud Migration Business Case Analysis | Lorena Aguilar | In 2010, the White House released its initial "cloud first" policy. Agencies have since struggled to identify all of the costs associated with cloud migration, and recent audits have criticized agencies for not properly assessing cloud-related costs and benefits. We have developed a cloud migration cost element structure (CES), which simplifies and standardizes cost data collection and estimating. We will present the structure and its applicability to business case and cost benefit analyses. | 2016 | Software Sizing |
| Collaborative Scoring: An Innovative Approach for Sizing Software Projects | Blaze Smallwood | A major challenge facing government organizations is developing defendable cost and schedule estimates for software development projects, driven by adequately estimating software size. This paper will describe a viable new method for sizing software requirements that utilizes a collaborative scoring methodology similar to those used during agile sprint planning. It has been successfully implemented on several government projects and is a viable option on all software projects. | 2016 | Software Sizing |
| Cost Estimate Credibility from a Government Perspective | Henry Apgar | This original paper describes a search for the definition of "Cost Estimate Credibility." The author documents his review of government-agency estimating handbooks, contemporary textbooks, previous ICEAA papers and journal articles, and the opinions of contemporary experts. The paper concludes with a concise definition which can be used by government and contractor estimators for their cost models and cost estimates. | 2016 Int'l | Government Perspectives |
| NASA's X-Plane Database and Parametric Cost Model V2.0 | Steve Sterk | In 2015 the idea of creating an Armstrong X-Plane Cost Model was conceived. Initially the data being used was leveraged from Dr. Joe Hamaker's Quick Cost v5.0 and verified and supplemented by Armstrong's summer intern Aaron McAtee (PhD). After several peer reviews, it was suggested to go back and take a hard look at the data collected. Our objective is to produce an X-plane database that ensures that all the data is as accurate, traceable and far-reaching as possible. | 2016 Int'l | Government Perspectives |
| NATO Agency Transformation: Creating a Centralized Cost Estimating Capability | Candace Mahala | A new NATO Agency is born, transforming multiple agencies into one, changing culture, and implementing a centralized robust cost estimating capability. This is the journey of how to grow and cultivate cost estimating from inception, integrating into projects as a value added asset, maturing over time and eventually becoming a fundamental part of the culture in the acquisition and execution processes. | 2016 Int'l | Government Perspectives |
| Master of Cost Estimating & Analysis Brief | Greg Mislick | Greg Mislick will provide an update on the all-distance learning Master's Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS). Entering its sixth year, the MCEA program has proven to be a success for both the students who take the program and the DoD Services and other US Government Executive Agencies in which the students work. Expert cost estimates are key underpinnings for critical government processes dependent on credible and reasonable cost estimates, from budgeting to cost-benefit and financial analyses. | 2016 Int'l | Government Perspectives |
| OSCAM - US Navy's Operating and Support Cost Estimating Tool | Stephen Curram | OSCAM is a simulation based tool for operating and support cost estimating that has been used by the US Navy for 20 years. It plays an important role in submissions for US DoD procurement milestones. An overview and brief history of OSCAM will be given, with examples of equipment programmes that have used OSCAM. It will conclude with lessons learned, including insights on the type of data required for operating and support cost estimating. | 2016 Int'l | Government Perspectives |
| NASA Project Cost Estimating Capability: New Analyses for Spacecraft Estimating | Brian Alford, Frank A. (Andy) Prince | The NASA Project Cost Estimating Capability (PCEC) is a free, publicly-available parametric tool for estimating the cost of spacecraft and transportation systems for scientific and human exploration of our solar system and beyond. In development since late 2013, PCEC replaces the NASA / Air Force Cost Model (NAFCOM). This presentation will discuss the new overall philosophy and data analysis underlying PCEC, the challenges encountered during development, and provide a high-level overview of the | 2016 Int'l | Government Perspectives |
| The 10+/-2 Factors for Estimate Success | Andy Nolan | A study of completed projects across Rolls-Royce revealed the factors that lead to accurate estimates. This has enabled us to develop an Estimate Readiness Level (ERL) assessment method, to determine the likely accuracy of an estimate. Success is no accident and estimates accuracy can be predicted from the outset of a project. This paper will cover the factors for estimate success and the development of the ERL assessment. | 2016 Int'l | Management |
| Is it Worth It? The Economics of Reusable Space Transportation | Richard L. Webb | Much has been invested in reusable space transportation systems in the belief that reusing system elements is cheaper than expending costly hardware each flight; a view not held by all industry stakeholders. This paper will define the metrics by which stakeholders measure financial "goodness," provide examples of application of the metrics to investment decisions, and offer observations on their potential influence on current and future decisions regarding space transportation system investment. | 2016 Int'l | Management |
| What Does A Mature Cost Engineering Organisation Look Like? | Dale Shermon | Cost Engineering Health Check; How good are those numbers? is Dale Shermon's second book and co-authored with Dr Mark Gilmour. The book is the paper and the presentation will consider what a mature cost engineering organisation looks like? It will cover the essential requirements for a credible and justified cost forecast. | 2016 Int'l | Management |
| Using Public Data for Validation and Winning Bids | Alexander King | To win bids, cost forecasting involves more than just creating a reasonable baseline. This presentation will show how public data can be combined and triangulated to ensure our predictions are competitive as well as reasonable. The presentation will show how parametric estimates, past contract data, financial statement data and statistical data can combine to make powerful metrics for productivity and competitiveness. | 2016 Int'l | Management |
| OSA: Cost and Schedule Saver or Driver? | Victoria Cuff, Brian D. Fersch | Open Systems Architecture employs a modular design and utilized consensus based standards for its key interfaces with full design disclosures [Office of the Deputy Secretary of Defense, Systems Engineering, Initiatives]. A challenge with OSA is the ability to continuously design modular systems with obsolete technology and pieces, evolving technological environment and emerging threats. Examining past successes and failures, this study will analyze means to reduce costs and increase success. | 2016 Int'l | Management |
| The Value of R&D - a Real Options Analysis Approach | John Shimell, Neil Davies | This study sought to apply Real Option Analysis (ROA) to answer the question "what is the value achieved from the MODs Chief Scientific Advisor's (CSA) "£400m annual budget". Typically this has been reviewed retrospectively via case studies and surveys. This study developed a tool based on an amended Black & Scholes Formula, combined with a Monte Carlo analysis of the "tree" of R&D to determine a predictive value for the R&D spend of the CSA. | 2016 Int'l | Management |
| Costing Aircraft Availability and 50-Shades of Grey Water | Andrew Langridge, Anna Foote | Data gathering is often one of a cost estimator's biggest nightmares for many reasons. Even in cases where data is available, the logistics of data collection can be daunting. This paper discusses a methodology that PRICE has developed to use open source web crawling software to regularly and repeatedly update guidance on commodity prices for Information Technology items with the push of a button. Uses of this methodology and further directions will be presented. | 2016 Int'l | Models & Methods |
| Successful Cost Estimating with T1-Equivalents | Grady Noll, F. Gurney Thompson III | This paper discusses our research and development of a cost model for additive manufacturing (AM), also known as 3D printing. Working with academia and multiple AM companies, we investigated cost drivers, collected cost and technical data, derived AM CERs, and developed an approach for including AM components within a larger parametric cost estimate. This paper discusses the research approach, data collection, results, cost model implementation, and future work. | 2016 Int'l | Models & Methods |
| Facilities Cost Estimates Drivers in the Oil and Gas Field Development | Linda Newnes, David Peacock, Ettore Settani, Jon Wright | In aerospace and defence research has focused on estimating the cost of delivering an outcome, e.g. available aircraft. Our research shows that focusing on the product alone, using product reliability/failures does not reflect the cost of delivering an outcome. In this paper we show why this is the case and how the use of input-output modelling and systems engineering can be used. We then demonstrate how the approach can be used in estimating the cost of waste water treatment. | 2016 Int'l | Models & Methods |
| The Signal and the Noise in Cost Estimating | Christian B. Smart | We seek to extract signal and eliminate noise when building models with historical data to predict future costs. One common problem is normalization, which is necessary when making comparisons but can inject noise when used in modeling. We discuss kernel smoothing and distribution fitting as ways to avoid overfitting peculiarities in historical data and the important issues of cross-validation and parsimony as ways to validate models and avoid the lure of overfitting. | 2016 Int'l | Models & Methods |
| CER Issues And Solutions | Kurt Brunner | Cost estimating Relationships (CERs) are widely accepted as an effective cost estimating tool. There are some issues with CER development and application that should be addressed. This briefing will explore those probable shortcomings and how they may be alleviated with the goal of refining our cost estimating capabilities. | 2016 Int'l | Risk Analysis |
| Should You Care About How Good An Estimation Process? | Alain Abran | This presentation is based on the author's 2015 book on Software Project Estimation (Wiley & IEEE Press) and illustrates how organizations (including in the car industry) having collected their own data using international standards have built their own estimation models and developed a key competitive advantage through improved software estimation capabilities. Examples will include data from a large European car manufacturer and a small software organization using Agile. | 2016 Int'l | Risk Analysis |
| A Parametric Model for the Cost Per Flight Hour | Michail Bozoudis | The parametric estimating technique provides timely cost estimates for "unknown" systems, utilizing cost estimating relationships deriving from historical datasets. This case study describes the development of a parametric model that estimates the cost per flight hour (CPFH). The cost derives as a function of the aircraft empty weight and the engine's specific fuel consumption. As an example, the F-35A CPFH is estimated under the hypothetical scenario that it is operated by the Hellenic Air Force. | 2016 Int'l | Risk Analysis |
| The Way From a Parametric Estimate to a CAD-Driven Calcualtion | Joachim Schoeffer, Herbert Spix | The industrial requirements for quick, rough and exact costings (e.g. for budgeting) and the detailed cost break down to the last screw or processor is as old as both methodologies exist. To satisfy both fractions we need different solutions that address both needs. Ideally both scenarios are integrated and interface between each other. In this presentation the author shows a way of integrate parametric and CAD-driven bottom-up. | 2016 Int'l | Risk Analysis |
| Outing the Outliers | Alan R. Jones | Inappropriate inclusion or exclusion of data that may or may not be regarded as outliers may make a project unachievable or uncompetitive. This paper looks at some of the issues involved and what tests are available to support an informed decision process rather than relying on a random guess on what is, or is not, an outlier. The paper will compare the performance of these tests against a number of sample data sets. | 2016 Int'l | Risk Analysis |
| Inherent Risk in Spreadsheets (IRiS) | Alan R. Jones | Many spreadsheets may be certified as being error-free on the day they are tested, but some are more prone to future errors due to the manner in which they have been designed or compiled. | 2016 Int'l | Risk Analysis |
| Software Estimating Model Using IFPUG Standard Sizing Methods | Christine Green | A major input for any cost model for software projects are the scope and effort for a software projects regardless of technology or type of project process. IFPUG Sizing Standards methods can be utilized for software estimating model and thereby establish a valuable input for the cost models produced that will include both scope control and effort estimates. | 2016 Int'l | Software |
| Cloud Solutions - Infrastructure, Platform or Software | Arlene F. Minkiewicz, Ashley Hoenigke | Migrating capability to the cloud comes with several planning and management challenges. How does an organization determine the right capability to migrate and the right platform for the migration? What challenges do the various cloud solutions (Infrastructure vs. Platform vs. Software as a Service) present? This paper presents a case study outlining the migration activities and other implications for migrating the same application on each of the three cloud solution models. | 2016 Int'l | Software |
| Measurement of Software Size: Contributions of COSMIC to Estimation Improvements | Alain Abran, Charles Symons, Frank Vogelezang | This talk presents 1) an outline of the design of the 2nd generation COSMIC method for measuring a functional size of software; 2) industry evidence of its practical value in software project performance measurement and its accuracy in estimating; 3) an account of the method's full automation, with very high accuracy, for real-time embedded software specified in Matlab-Simulink and developed through a world network of software contractors. | 2016 Int'l | Software |
| The Monitoring and Early Warning Indicators for a Software Project | Christine Green | This presentation will focus on controlling by monitoring and early warning indicators on a software project's scope, estimate and performance - all factors that highly influence the cost and budget. | 2016 Int'l | Software |
| Introduction to Software Obsolescence Cost Analysis Framework | Sanathanan Rajagopal | Almost every project in Defence sectors has got software with various degrees of complexity and dependencies. Whilst various research and studies have been conducted on system obsolescence and tools developed to cost systems and components obsolescence, no major research has been undertaken to develop a framework to estimate cost of software obsolescence. This paper will discuss the Software Obsolescence Cost Analysis Framework developed as part this research. | 2016 Int'l | Software |
| Applying Earned Value to Agile Development Programs | Bob Hunt, Michael Thompson | A general movement to Agile software development process is one method used to address and potentially resolve these problems. The movement to Agile Software Development introduces several unique problems for EVM assessments. This paper will discuss some key element associated with: Agile/Hybrid Agile software development; Establishing a process for developing the technical, cost and schedule baseline; Applying Earned Value Management to agile programs. | 2016 Int'l | Software |
| Hubble Space Telescope and Its Color Photographs | David Pine | The Hubble Space Telescope has taken thousands of "color" photographs that have excited people the world over. This presentation examines the "Mirror Problem" and then how scientists change the photons being collected by the telescope into the phenomenal photographs that have thrilled the public. Understanding the color photographs begins with understanding what color is, and how Hubble's cameras operate to eventually produce pictures. | 2016 Int'l | Space Systems |
| Satellite Mass Growth | Donald MacKenzie, Erik Burgess | A new study of satellite mass growth on 21 programs shows that equipment type and design maturity are drivers. Unlike current AIAA guidelines, acquisition strategy is shown to be a significant driver, with competitively awarded programs experiencing much higher growth than sole-source awards, even for units with the same design maturity. Overall, mass growth allowance recommended by AIAA for every equipment type is much lower than the actual growth experienced in this dataset. | 2016 Int'l | Space Systems |
| QuickCost 6.0 | Bob Hunt | QuickCost 6.0 has just been released by Galorath Federal and NASA. QuickCost 6.0 is an all new implementation QuickCost (QuickCost 1.0 was released in 2004). The philosophy behind the model is that accurate cost and schedule estimates of automated spacecraft missions can be obtained parametrically in a quick and easy fashion estimating at the top 11 NASA WBS elements and using only a few input variables. QuickCost 6.0 is based on data from the NASA ONCE CADRe database. | 2016 Int'l | Space Systems |
| Halfway to Anywhere | Fabian Eilingsfeld | A 'tariff zone map' is proposed for space transportation. Using parametric analysis, applying the paradigm of terrestrial public transport, it displays specific energy cost depending on total energy delivered to payloads. Every space transportation system leaves its own distinctive footprint when its performance data is plotted on the proposed map. It visualizes performance and cost trends with an emphasis on affordability, which helps rational discussion of project options and roadmaps. | 2016 Int'l | Space Systems |
| The SSCAG Legacy | Henry Apgar | Active partnership in this ICEAA Symposium marks the final milestone of a spectacular forty-year journey of the Space Systems Cost Analysis Group. SSCAG provided a unique international forum for space-systems cost analysts. Over those years, more than 300 skilled cost estimators, business analysts, program managers, space engineers, and logisticians filled the international venues, at more than 100 scheduled meetings, to deliberate major issues associated with government and commercial development. | 2016 Int'l | Space Systems |
| Using Budgeted Cost of Work Performed to Predict Estimates at Completion for Mid-Acquisition Space Programs | C. Grant Keaton | Conventional techniques to predict estimates at completion using earned value management data generally underpredict the end costs for space acquisition programs. This research assesses the accuracies of earned value management analysis methods to forecast estimates at completion for on-going space system acquisitions. | 2015 | Journal of Cost Analysis and Parametrics |
| An Economics Approach to Fixing the Fare of the Parking Lot Service in Bogota Using Price Cap Regulation | Jorge A. Perdomo | This article presents an application of the Ramsey Pricing approach to establish the price cap or pricing ceiling per minute for the parking lot service in Bogota (Colombia), using the microeconomic framework for price fixing from a costs analysis (total fixed costs, total variables costs, average total cost, and marginal cost) of the provision of parking lot service in Bogota. | 2015 | Journal of Cost Analysis and Parametrics |
| Predicting the Likelihood of Cost Overruns: An Empirical Examination of Major Department of Defense Acquisition Programs | Alan K. Gideon, James S. Wasek | This article provides a method for predicting the cost of a major acquisition program five years after program development approval. The author work extends the effort begun by Asher and Maggelet by adjusting for changes in median cost growth factors for each acquisition domain. Procurement average unit costs at program approval and five years afterwards were compared for 101 major United States Department of Defense acquisition programs. | 2015 | Journal of Cost Analysis and Parametrics |
| To Cost an Elephant: An Exploratory Survey on Cost Estimating Practice in the Light of Product-Service-Systems | Ettore Settanni, Nils E. Thenent, Linda B. Newnes, Glenn Parry, Yee Mey Goh | Businesses now contracting for availability are regarded as part of a paradigm shift away from the familiar product and support business model. The main difference being that such businesses eventually commit to provide a service outcome via product-service-system. The research presented in this article investigates how current cost estimating practice relates with the idea of having as the point of focus for the analysis a product-service-system delivering service outcomes, rather than a product. | 2015 | Journal of Cost Analysis and Parametrics |
| Parametric Scale-Up Cost Factors for Conventional and Micro-Scale Tools | Donald S. Remer, Kerry M. Chin | This article estimates cost scale-up factors for micro-scale tools, including high speed steel micro tools, micro thread hand taps, and various types of mesh and wires. Many of these tools initially have a positive scale-up factor where the material costs directly correlate with the cost of the tool; they then transition to a negative scale-up factor at the micro-scale where manufacturing is more difficult and becomes the determining factor for the prices of the products. | 2015 | Journal of Cost Analysis and Parametrics |
| Using Earned Value Data to Forecast the Duration of Department of Defense Space Acquisition Programs | Shedrick Bridgeforth, Jonathan Ritschel, Edward D. White, Grant Keaton | Traditional earned value management techniques to predict final costs of space acquisition programs are historically inaccurate. A 2015 study by the Air Force Cost Analysis Agency (Keaton 2015) sought to improve the accuracy of the cost estimate at completion for space system contracts through a linear relationship between budgeted cost for work performed and time. | 2015 | Journal of Cost Analysis and Parametrics |
| Bending the Cost Curve: Moving the Focus from Macro-level to Micro-level Cost Trends with Cluster Analysis | Bradley C. Boehmke, Alan W. Johnson, Edward D. White, Jeffery D. Weir, Mark A. Gallagher | Bending the cost curve has become the ambiguous jargon employed in recent years to emphasize the notion of changing unwanted cost trends. In response to the planned $1 trillion Department of Defense budget reduction over the next six years, the Air Force has launched its own Bending the Cost Curve initiative in an effort to reduce cost growth. | 2015 | Journal of Cost Analysis and Parametrics |
| A Literature Survey and Future Directions for Product Development: A Focus on Conceptual Design Stage | Satish Kumar Tyagi, Xianming Cai, Kai Yang | Product development process follows a sequence of activities, methods, and tools to design and develop a high quality product. An optimized strategy for a single generation product development may not be the best choice when multi-generation scenarios are considered. In this article an attempt has been made to study the nature of multi-generational product development and compare it with that of single generation product development to drive validating arguments. | 2015 | Journal of Cost Analysis and Parametrics |
| Pioneers of Parametrics: Origins and Evolution of Software Cost Estimation | Ricardo Valerdi | This article provides an historical account of the development of the field of software cost estimation, specifically the area of parametrics, through information obtained during interviews of 13 pioneers in the field. Cost model developers, users, and practitioners were interviewed with the intent to capture their views on the impact between cost estimation research and practice. | 2015 | Journal of Cost Analysis and Parametrics |
| Time Phasing Aircraft R&D Using the Weibull and Beta Distributions | Gregory E. Brown, Edward D. White, Jonathan D. Ritschel, Michael J. Seibel | Early research on time phasing primarily focuses on the theoretical foundation for applying the cumulative distribution function, or S-curve, to model the distribution of development expenditures. Minimal methodology prior to 2002 provides for estimating the S-curves parameter values. Brown et al. (2002) resolved this shortcoming through regression analysis, but their methodology is not specific to aircraft and does not consider aircraft-specific variables, such as first flight. | 2015 | Journal of Cost Analysis and Parametrics |
| Covered with Oil: Incorporating Realism in Cost Risk Analysis | Christian B. Smart | When Jimmy Buffett sang the words All of those tourists covered with oil in his song Margaritaville he probably never imagined that this phrase might apply to crude oil instead of suntan lotion. Both the cost and the environmental impact from the 2010 oil spill in the Gulf of Mexico were much worse than anyone had expected or could have predicted. It was, in the words of financial writer, Nassim Taleb, a black swan "an unexpected event with tremendous consequences. " | 2015 | Journal of Cost Analysis and Parametrics |
| Modifying Duration-Based Costing to Illustrate the Effect of Fixed Costs | Anne-Marie Teresa Lelkes | This study introduces Modified Duration-Based Costing (MDBC) as an alternative to Duration-Based Costing (DBC) developed in Lelkes and Deis (2013). Both DBC and ABC have a tendency to treat fixed costs as though they were variable. This study expands on the DBC model by showing an alternative way of dealing with fixed costs instead of treating them as variable costs. This study uses analytical methodology and simulations to analyze MDBC relative to an Activity-Based Costing (ABC) system. | 2015 | Journal of Cost Analysis and Parametrics |
| Trade Space, Product Optimization, and Parametric Analysis | Douglas K. Howarth | This article shows how to bound, build, and assemble trade spaces for product optimization. The advent of computerized tools that describe available trade spaces has changed not only the nature of optimized product design, but that of parametric cost studies as well. Because these tools allow broader analysis, engineers can produce many more potential designs. | 2014 | Journal of Cost Analysis and Parametrics |
| Applications of a Parsimonious Model of Development Programs Costs and Schedules | David Lee, Carol Dezwarte, Stephanie Sigalas-Markham, Jeremy Eckhause | A model of the cost and schedule of a development program, characterized by three non-dimensional parameters, gives means for estimating the cost and schedule impacts of constraining funding below planned levels, as well as for assessing the realism of the costs and schedules of planned programs. In contrast to models of the Norden-Rayleigh-Weibull class, the model explicitly considers specific components of cost, and captures the distinction between a development programs value (i.e., the things delivered) and its cost (i.e., the money paid to acquire the value). | 2014 | Journal of Cost Analysis and Parametrics |
| Software Industry Goals for the Years 2014 through 2018 | Capers Jones | This article discusses 20 quantitative targets for software engineering projects that are technically feasible to be achieved within a five year window. Some leading companies have already achieved many of these targets, but average and lagging companies have achieved only a few, if any. Software needs firm achievable goals expressed in quantitative fashion. | 2014 | Journal of Cost Analysis and Parametrics |
| COTECHMO: The Constructive Technology Development Cost Model | Mark B. Jones, Phil F. Webb, Mark D. Summers,Paul Baguley | A detailed analysis of the available literature and the aerospace manufacturing industry has identified a lack of cost estimation techniques to forecast advanced manufacturing technology development effort and hardware cost. To respond, this article presents two parametric Constructive Technology Development Cost Models (COTECHMO). The COTECHMO Resources model is the first and is capable of forecasting aerospace advanced manufacturing technology development effort in person-hours. | 2014 | Journal of Cost Analysis and Parametrics |
| Engineering Systems: Best-in-Class/Worst-in-Class | Donald M. Beckett | Measurements goal is to help assess performance to determine which methods are productive or counterproductive. Metrics are tools used to identify and implement practices that lower costs, reduce time to market, and improve product quality. But process improvement is not accomplished through measurement or metrics alone. | 2014 | Journal of Cost Analysis and Parametrics |
| Comparing Lifecycle Sustainment Strategies in an Electronic Component Obsolescence Environment | Kenneth D. Underwood, Jeffrey A. Ogden, Matthew T. McConville | Rapid advancements in technology and the diminishing lifecycle of electronic systems have complicated the sourcing and sustainment activities of many organizations as suppliers of original components go out of business or refuse to produce obsolete products. This article explores diminishing manufacturing sources and material shortages as well as the obsolescence costs and reliability issues associated with electronic components. | 2014 | Journal of Cost Analysis and Parametrics |
| Cost Risk Allocation Theory and Practice | Christian B. Smart | Risk allocation is the assignment of risk reserves from a total project or portfolio level to individual constituent elements. For example, cost risk at the total project level is allocated to individual work breakdown structure elements. This is a non-trivial exercise in most instances, because of issues related to the aggregation of risks, such as the fact that percentiles do not add. | 2014 | Journal of Cost Analysis and Parametrics |
| Activity-Based Parsimonious Cost Systems | Shannon Charles, Don Hansen | Accurate product costing information is instrumental in effective decision making, especially for product related decisions, such as product mix and product emphasis. A body of literature suggests that activity-based costing plays an important role in providing accurate product costing information. However, activity-based costing systems are also more complex due to the number of cost drivers identified, compared to traditional single cost driver systems. | 2014 | Journal of Cost Analysis and Parametrics |
| An Assumptions-Based Framework for TRL-Based Cost and Schedule Models | Bernard El-Khoury, C. Robert Kenley | The Technology Readiness Level scale has been used to assess progress and provide a framework for developing technology. Many Technology Readiness Level-based cost and schedule models have been developed to monitor technology maturation, mitigate program risk, characterize transition times, or model schedule and cost risk for individual technologies as well technology systems and portfolios. | 2014 | Journal of Cost Analysis and Parametrics |
| Development of a Product Life-Cycle Cost Estimation Model to Support Engineering Decision-Making in a Multi-Generational Product Development Environment | Xianming Cai, Satish Tyagi | The research under product development domain identifies and implements new concepts in design, planning, manufacturing, testing, and service sectors to keep up with the market demands. Many product developments are multi-generational in nature and may require redesigning the product at each generation because an optimized strategy for a single generation may not be the best option in the multi-generation scenarios. | 2014 | Journal of Cost Analysis and Parametrics |
| A Process for the Development and Evaluation of Preliminary Construction Material Quantity Estimation Models Using Backward Elimination Regression and Neural Networks | Borja Garcia de Soto, Bryan T. Adey, Dilum Fernando | During the early stages of a project, it is beneficial to have an accurate preliminary estimate of its cost. One way to make those estimates is by determining the amount of construction material quantities that are required and then multiplying the estimated construction material quantities by the corresponding unit cost. One advantage of making estimates in this way is that it allows for the segregation of quantities and costs. | 2014 | Journal of Cost Analysis and Parametrics |
| On the Shoulders of Giants: A Tribute to Prof. Barry W. Boehm | Jo Ann Lane, Daniel D. Galorath, Ricardo Valerdi | Prof. Barry Boehms life work is full of contributions to the software engineering and systems engineering disciplines. This article presents Prof. Barry Boehms work in the context of the giants on whose shoulders he stands as well as the people he has mentored to carry on his work. Much of Prof. Boehms work described in this article focuses on his key contributions to the software and system development industries, as well as from the perspective of the enduring legacy he has established with his industry affiliates and students. | 2014 | Journal of Cost Analysis and Parametrics |
| NPS and AFIT Masters Degree in Cost Estimating and Analysis | Daniel Nussbaum, Greg Mislick | This presentation provides an update on the Joint, all-Distance Learning Masters Degree in Cost Estimating and Analysis offered at the Naval Postgraduate School (NPS) and Air Force Institute of Technology (AFIT). | 2014 | Business and Art of Cost Estimating |
| A Comprehensive CES and BCA Approach for Lifelong Learning | Kevin Cincotta, Darcy Lilley | The Air Force Air Mobility Command (AMC) Enterprise Learning Office (ELO) mission is to transform AMC into a premier Air Force learning organization, achieve learning through optimum approaches and develop Mobility Airmen into life-long learners who demonstrate institutional Air Force competencies with a positive approach to managing their own learning. In this context, learning has three main components: training, education, and experience. ... | 2014 | Business and Art of Cost Estimating |
| BOE Development: Scope Evaluation and Criteria | Michael Butterworth, Demetrius Prado | Current basis-of-estimates (BOEs) do not identify succinctly, how the scope of the estimate was derived and how actuals used for build-up, analogous and parametric methodologies fit the technology and product under consideration. We believe that one of the many problems leading to poor BOE development revolves around the lack of identifying the scope of work and understanding the effort needed in particular skill mix to perform the task. ... | 2014 | Business and Art of Cost Estimating |
| Long Term Affordability through Knowledge Based Bid & Proposal | Zachary Jasnoff | According to a 2013 GAO report, "positive acquisition outcomes require the use of a knowledge-based approach to product development that demonstrates high levels of knowledge before significant commitments are made. In essence, knowledge supplants risk over time." Often times, acquisition proposals that are not knowledge-based introduce significant risk and while seeming reasonable in the short term, cannot sustain long term affordability. ... | 2014 | Business and Art of Cost Estimating |
| What Happens to a Cost Estimate When It's Done? | William Barfield, David Bach | What happens to a cost estimate when it is done, or "finished?" Cost estimates are used to support a wide variety of financial decisions. When estimates are done and the decision has been made, is the estimate still useful after the decision, or does it become "shelf-ware?" We surveyed the international cost community to determine how we develop, document, use, and archive our various kinds of cost estimates. ... | 2014 | Business and Art of Cost Estimating |
| Update On The Cost FACTS (Factors, Analogies, CER's & Tools/Studies) Group " Enhancing KM By Leveraging Enterprise Social Networking | Daniel Harper, Ruth Dorr | Cost FACTS is a community-driven initiative to bring together the cost estimating community across MITRE, Government agencies, the contractor community, and academia to share reusable, non-restricted cost information and to encourage dialogue throughout the worldwide cost community. Cost FACTS provides an easily accessible repository of reusable cost estimating FACTS, i.e., Factors, Analogies, Cost Estimating Relationships (CERs), and Tools/Studies. ... | 2014 | Business and Art of Cost Estimating |
| Space Shuttle Cost Analysis: A Success Story? | Humboldt Mandell | In the aftermath of the highly successful Apollo lunar program, NASA struggled for a few years to find a meaningful program which would satisfy long range national space strategies, as well as reflecting the realities of the rapidly changing political environment in the nation. The Space Shuttle emerged from the need to lower the costs of orbital cargo delivery for construction of a space station and enabling Mars exploration, but also was highly constrained by DOD requirements ... | 2014 | Business and Art of Cost Estimating |
| A Balanced Approach to Meeting Fiscal Constraints | Steve Green, Kevin Davis, Kurt Heppard | The effective and systematic use of cost and budgeting information is a critical component of strategic planning and decision making in most organizations. The Department of Defense's (DoD) current operational environment, scarce resources, and conflicting stakeholder expectations are resulting in extreme fiscal constraints. The result is the need to reconsider missions and goals, reassess priorities, entertain force structure alternatives, and ultimately reduce budgets. ... | 2014 | Cost Management |
| Cost Overruns and Cost Growth: A Three Decades Old Cost Performance Issue within DoD's Acquisition Environment | Leone Young | For the last three decades, the US Department of Defense (DoD) has been encountering program performance issues such as inaccurate and unrealistic estimations for its acquisition programs, and its effort of eliminating cost overruns and cost growth phenomenon has been deemed ineffective as well. This paper examines and consolidates multiple government reports and studies, which pertain to the cost performance of DoD that has been viewed as unsatisfactory and problematic. ... | 2014 | Cost Management |
| Supplier Cost/Price Analyses - Best Practices for evaluating supplier proposals and quotes | Mike Mardesich | During proposal planning, preparation, and review, an important but often over looked aspect is the evaluation of a supplier's proposal or quote. Requirements in the Federal Acquisition Regulations (FAR) provide for certain visibility into supplier's proposals and quotes, depending on the value of the proposal/quote. Additionally it is critical to note the FAR puts the onus of supplier cost (or price) analysis on the prime contractor. ... | 2014 | Cost Management |
| Innovative Business Agreements and Related Cost & Pricing Methods at NASA in Support of New Commercial Programs | Torrance Lambing, James Roberts | This paper and presentation, focusing on Kennedy Space Center, will discuss changes and new methods of pricing and estimating the costs of NASA facilities and services to be provided to outside entities for use in new Commercial Space endeavors. It will also give an overview of new NASA regulations and documents that establish policy and guidance for entering into Agreements and how they are priced under the various types of Agreement currently being used at NASA. ... | 2014 | Cost Management |
| The Other RCA: Restaurant Cost Analysis | Peter Braxton | The Weapon Systems Acquisition Reform Act (WSARA) of 2009 highlighted the importance of Root Cause Analysis, or RCA, but its conduct remains shrouded in mystery. In illustrating the central role of risk and uncertainty analysis in cost, Dick Coleman often made the provocative pronouncement to the effect of "You can't stand outside a restaurant with a menu, and the people you'll be dining with, and a calculator and get within 10% of the final bill, so what makes you think you can estimate a complex multi-billion-dollar acquisition program with that precision?!" ... | 2014 | Cost Management |
| Intelligence Mission Data Cost Methodology Guidebook | Eugene Cullen, III, Matthew Schumacher | In 2013, Booz Allen Hamilton developed and authored the Intelligence Mission Data Cost Methodology Guidebook (IMD CMGB) for the Defense Intelligence Agency's (DIA) Intelligence Mission Data Center (IMDC). This guidebook is the official costing manual for life-cycle mission data planning, required by Department of Defense Directive (DoDD) 5250.01 "Management of IMD within DoD Acquisitions," and defines costing methodologies, procedures, and processes that result in OSD Cost Assessment & Program Evaluation (CAPE) and Government Accountability Office (GAO) compliant cost estimates that are required by DoD Acquisition Systems. ... | 2014 | Cost Management |
| Achieving Affordable Programs NRO Cost Analysis Improvement Group (CAIG) Support of Cost Driver Identification | Linda Williams, Pat Kielb, Eileen DeVillier, Jay Miller | This paper will describe the approach taken over the last several months to identify cost drivers and summarize findings that senior leaders found helpful to the decision making process. In summary, as budgets reductions trigger asset affordability reviews, the NRO CAIG has been able to assist in identifying total organizational cost drivers and provide a framework for future cost reduction opportunities. | 2014 | Cost Management |
| Godel's Impact on Hilbert's Problems Or Cost Consistency and Completeness as an Impossible Exercise | David Peeler | In a previous set of papers, the idea of using Hilbert's Problems as a construct to propose and recently revisit the status of a list of Hilbert's Problems for Cost Estimating. This paper similarly employs Godel's enlightenment with respect to Hilbert's attempts onto the cost estimating community. What can we learn about ourselves as estimators and where can we exert the greatest impact with respect to the use of our estimates? Using Godel's two theorems of undecidability as catalyst, we will explore the effect and utility of exacting math and other motions on cost estimates specifically and programmaitics generally. | 2014 | Cost Management |
| A New Cost Management Policy for Your Organization: An Integrated Approach? | Tom Dauber, Woomi Chase, Ken Odom | Developing a robust Cost Management Policy is a key driver to the success of any organization, regardless of size or industry. The policy should ensure cost control measures that are valid and effective, risks are mitigated, solutions are delivered on time, and profits/ROIs are maximized. The Cost Management Policy should be a systematic approach to managing cost through-out the life cycle of a program through the application of cost engineering and cost management principles. ... | 2014 | Cost Management |
| Right Sizing Earned Value Management for Your Project | Gordon Kranz | Earned Value Management (EVM) is a program management tool that provides data indicators that can be used on all programs to enable proactive decision making throughout the program lifecycle and facilitate communication across the program team. Each program has unique attributes that should be considered when determining program management and reporting requirements, including, but not limited to, contract size and type, scope of work, complexity, risk, technology maturity, and resource requirements. ... | 2014 | Cost Management |
| Big Data Meets Earned Value Management | Glen Alleman, Thomas Coonce | When the result of an action is of consequence, but cannot be known in advance with precision, forecasting may reduce decision risk by supplying additional information about the possible outcomes. Data obtained from observations collected sequentially over time are common. Earned Value Management is an example where project performance data (BCWP) is collected from the status reports of planned work (BCWS) and a forecast of future performance is needed to manage the program. ... | 2014 | Earned Value Management |
| Don't Let EVM Data Mislead You | Steve Sheamer | EVM data is a popular data source for cost estimators and for good reason; in theory, it should provide most of the data needed to develop an estimate for a program. For completed programs, it provides historical costs by WBS and for programs that are in work it provides a measure of the work completed, work remaining, and forecast of the work remaining. But during a period of frequent cost overruns, estimates built using EVM data often fail to forecast the extent of program overruns... | 2014 | Earned Value Management |
| Trust but Verify - An Improved Estimating Technique Using the Integrated Master Schedule (IMS) | Eric Lofgren | It has long been the wonder of management why the Integrated Master Schedule (IMS) fails to give advanced warning of impending schedule delays. The oft-touted Government Accountability Office (GAO) 14-Point Check for Schedule Quality analyzes schedule health using key metrics, leading one to assume that such a test authenticates schedule realism. Why, then, do practitioners find themselves caught off guard to slips when their IMS appears in good health? ... | 2014 | Earned Value Management |
| A Cure For Unanticipated Cost and Schedule Growth | Glen Alleman, Thomas Coonce | Federal programs (DoD and civilian) often fail to deliver all that was promised and many times cost more than estimated and are often late. Delivering programs with less capability than promised while exceeding the cost and planned undermines the Federal government's credibility with taxpayers and contributes to the public's negative support for these programs. ... | 2014 | Earned Value Management |
| Unleashing the Power of MS Excel as an EVM Analysis Tool | Allen Gaudelli, Steve Sheamer | What do you do if you need to analyze or report on EVM data and you don't have access to (or can't afford) the latest industry software? Nearly everyone has a very powerful analysis and reporting tool on their desktop with the flexibility and capability to consolidate cost, schedule, and risk drivers into a single view. In this presentation, we will show you how to leverage and manipulate the inherent capabilities of Microsoft Excel to build interactive EVM dashboards ... | 2014 | Earned Value Management |
| Design to Cost: Misunderstood and misapplied | Erin Barkel, Tolga Yalkin | The Canadian Department of Defence maintains that concerns over cost overruns are overstated because it adopts a design to cost approach. According to the US Government, design to cost "embodies the early establishment of realistic but rigorous cost targets and a determined effort to achieve them." From the beginning of a project to its completion, "[c]ost is addressed on a continuing basis as part of a system's development and production process." ... | 2014 | Earned Value Management |
| Testing Benford's Law with Software Code Counts | Chuck Knight, Chris Kaldes | When analyzing a data set, common thinking may lead one to suspect that the leading digit of each data point would follow a uniform distribution, where each digit (1 through 9) has an equal probability of occurrence. Benford's law, to the contrary, states that the first digit of each data point will conform to a nonuniform distribution. More specifically, it states that a data point is more likely to begin with a one than a two, a two more likely than a three, a three more likely than a four, and so on. ... | 2014 | Information Technology |
| Improved Method for Predicting Software Effort and Schedule | Wilson Rosa, Barry Boehm, Ray Madachy, Brad Clark, Joseph P. Dean | This paper presents a set of effort and schedule estimating relationships for predicting software development using empirical data from 317 very recent US DoD programs. The first set predicts effort as a function of size and application type. The second predicts duration using size and staff level. The models are simpler and more viable to use for early estimates than traditional parametric cost models. Practical benchmarks are also provided to guide analysts in normalizing data. | 2014 | Information Technology |
| Costs of Migration and Operation in the Cloud | Arlene Minkiewicz | At one level cloud computing is just Internet enabled time sharing. Instead of organizations investing in all the Information Technology (IT) assets such as hardware, software and infrastructure they need to meet business needs; cloud computing technology makes these resources available through the Internet. Cloud computing allows an organization to adopt a different economic model for meeting IT needs by reducing capital investments and increasing operational investments. ... | 2014 | Information Technology |
| How I Continued to Stop Worrying and Love Software Resource Data Reports | Nicholas Lanham | This presentation highlights the trends and cost estimating relationships derived from detailed analysis of the August 2013 Office of the Secretary of Defense (OSD) Software Resource Data Report (SRDR) data. This analysis was conducted by Nicholas Lanham and Mike Popp and provides as follow-on analysis to the August 2012 SRDR brief developed and previously presented by Mike Popp, AIR 4.2. ... | 2014 | Information Technology |
| Mobile Applications, Functional Analysis and Cost Estimation | Tammy Preuss | This presentation will demonstrate how to derive cost estimates at different stages in a project's lifecycle by using function points and the advantages of using an FP based size estimate over a SLOC based estimate. The intended audience is software cost estimators, project managers, and anyone who is interested in software measurement. | 2014 | Information Technology |
| In Pursuit of the One True Software Resource Data Reporting (SRDR) Database | Zachary McGregor-Dorsey | For many years, Software Resource Data Reports, collected by the Defense Cost and Resource Center (DCARC) on Major Defense Acquisition Programs (MDAPs), have been widely acknowledged as an important source of software sizing, effort, cost, and schedule data to support estimating. However, using SRDRs presents a number of data collection, normalization, and analysis challenges, which would in large part be obviated by a single robust relational database. ... | 2014 | Information Technology |
| Optimizing Total Cost of Ownership for Best Value IT Solutions: A Case Study using Parametric Models for Estimates of Alternative IT Architectures and Operational Approaches | Denton Tarbet, Kevin Woodward, Reggie Cole | Because of a variety of architectures and deployment models, Information Technology (IT) has become more and more complex for organizations to manage and support. Current technology IT system architectures range from server based local systems to implementations of a Private Cloud to utilization of the Public Cloud. Determining a "best value architecture" for IT systems requires the ability to effectively understand not only the cost, but the relative performance, schedule and risk associated with alternative solutions. ... | 2014 | Information Technology |
| Estimating Hardware Storage Costs | Jenny Woolley, William Black | Estimating Commercial-off-the-Shelf (COTS) hardware storage volume and cost requirements can be challenging. Factors such as storage type, speed, configuration, and changing costs can potentially lead to estimating difficulties. This is especially true when a Redundant Array of Independent Disks (RAID) configuration is implemented. Due to the multiple attributes that can vary within each RAID level, as well as other factors that may influence the total storage volume needed, developing relationships for estimating long-term storage costs can become complicated. ... | 2014 | Information Technology |
| Relating Cost to Performance: The Performance-Based Cost Model | Michael Jeffers, Robert Nehring, Jean-Ali Tavassoli, Kelly Meyers, Robert Jones | For decades, in order to produce a cost estimate, estimators have been heavily reliant on the technical characteristics of a system, such as weight for hardware elements or source lines of code (SLOC) for software elements, as specified by designers and engineers. Quite often, a question will arise about the cost of adding additional performance requirements to a system design (or in a design-to-cost scenario, the savings to be achieved by removing requirements). Traditionally, the engineers will then have to undertake a design cycle to determine how the shift in requirements will change the system. ... | 2014 | Information Technology |
| Lessons Learned from the International Software Benchmark Standards Group (ISBSG) | Arlene Minkiewicz | This paper will introduce the ISBSG and the database that are available from the ISBSG. It then provides details of the data driven approach applied to develop these templates - discussing research approach, methodology, tools used, findings and outcomes. This is followed by a discussion of lessons learned including the strengths and weaknesses of the database and the strength and weaknesses of the solutions derived from it. While particularly relevant to software estimators, this paper should be valuable to any estimator who lacks data or has data they are not quite sure what they might do with it. | 2014 | Information Technology |
| Software Maintenance: Recommendations for Estimating and Data Collection | Shelley Dickson, Bruce Parker, Alex Thiel, Corinne Wallshein | The software maintenance study reported at ICEAA in 2012 and 2013 continued to progress in 2013 in spite of the high data variability. This presentation summarizes the past years' software maintenance data collection structure, categorizations, normalizations, and analyses. Software maintenance size, defect, cost, and effort data were collected from Fiscal Years (FY) 1992 - 2012. Parametric analyses were performed in depth on available variables included in or derived from this U.S. Department of Defense software maintenance data set. ... | 2014 | Information Technology |
| An Update to the Use of Function Points in Earned Value Management for Software Development | Michael Thompson, Daniel French | This presentation describes the opportunity that was presented to the team and how the recently completed pilot program was developed and implemented to address it. The authors will address how effective the pilot program was as far as identifying and resolving issues, measuring earned value, as well as the challenges and lessons learned with the development, implementation, and sustainment of the FP based EVM process. | 2014 | Information Technology |
| The Federal IT Dashboard: Potential Application for IT Cost & Schedule Analysis | Daniel Harper | Federal agencies have experienced a growing demand for rapid turnaround cost and schedule estimates. This need is increasing as the pressure to deploy systems rapidly mounts. The push for Agile SW development compounds this problem. A critical component in cost estimating is the data collection of costs for the various elements within the estimate. Analogous programs constitute a robust source for credible estimates. The problem is how to find analogous programs and how to capture the cost of elements within those programs at a sufficiently detailed level to use in a cost estimate and in a timely manner so that the cost data is still relevant. ... | 2014 | Information Technology |
| Trends in Enterprise Software Pricing from 2002 to 2011 | Ian Anderson, Dara Logan | One of the biggest challenges in the cost estimating community is data collection. In the Information Technology (IT) cost community, technology is always evolving, while the data capturing it tend to be scarce and more difficult to use in building solid cost models. Fortunately, NCCA learned the Department of the Navy (DON) Chief Information Officer (CIO) has been collecting benchmarking measures, including pricing, since 2002 under the Enterprise Software Initiative (ESI) Blanket Purchasing Agreements (BPAs). ... | 2014 | Information Technology |
| Estimating Cloud Computing Costs: Practical Questions for Programs | Kathryn Connor | Cloud computing has garnered the attention of the Department of Defense (DoD) as data and computer processing needs grow and budgets shrink. In the meantime, reliable literature on the costs of cloud computing in the government is still limited, but programs are interested in any solution that has potential to control growing data management costs. ... | 2014 | Information Technology |
| The Agile PM Tool: The Trifecta for Managing Cost, Schedule, and Scope | Blaze Smallwood, Omar Mahmoud | The growing number of DoD software projects that are adopting an "Agile" development philosophy requires cost estimators to not only adapt the methodologies and metrics they use to estimate software development costs, but also re-think their models to give PMs the information they need to effectively manage these programs. The Agile PM Tool is one manifestation of this trend as it provides a logical, dynamic approach for helping the government effectively manage the cost, schedule, and scope of their "Agile" projects. | 2014 | Information Technology |
| Which Escalation Rate Should I Use? | Nathan Honsowetz | Conducting life cycle cost estimates requires time frames of 10, 20, even 30 years, and with such long time frames it's important to use appropriate escalation indices. Escalation can have a significant impact on cost estimates, especially estimates with longer time frames. However, often cost estimators insert a "standard" escalation index into their models without considering whether that index is appropriate for their estimate. ... | 2014 | Life Cycle Costing |
| Ground Vehicle Reliability Analysis Using the Mean Cumulative Function | Caleb Fleming | This paper surveys and outlines the fundamental MCF methodologies and explanations detailed in-depth in Wayne Nelson's Recurrent Events Data Analysis for Product Repairs, Disease Recurrences, and Other Applications. Applying the MCF to vehicle maintenance data reveals recurring component failure behaviors, develops new guidelines for interpretation, and assists in data normalization and validation. | 2014 | Life Cycle Costing |
| Cost Overruns and Their Precursors: An Empirical Examination of Major Department of Defense Acquisition Programs | Alan Gideon, Enrique Campos-Nanez, Pavel Fomin, James Wasek | This paper proposes a model of acquisition program future cost for two specific acquisition domains - aircraft and ships - that takes into account the non-recurring developmental costs defined at program approval and each domain's historic tendencies to exceed planned program cost. Technical and non-technical reasons for these observations are discussed. ... | 2014 | Life Cycle Costing |
| System Utilization: An In-depth Method of Modeling and Measuring Military Manpower Costs | Omar Mahmoud | Establishing defendable cost estimating methodologies for capturing military manpower costs is a key component in any Program Life Cycle Cost Estimate. With a proven and systematic approach to estimating military manpower costs, a program can be confident in selecting a proper course of action among competing alternatives when conducting an EA, appropriately designate their ACAT level, avoid the pitfalls that lead to over/under estimating or double counting costs, and above all obtain a high level of confidence from their resource sponsor and Milestone Decision Authority. | 2014 | Life Cycle Costing |
| Integrating Sustainability into Weapon System Acquisition within the Department of Defense | Walt Cooper, Remmie Arnold | DoD acquisition and logistics professionals use the term sustainment to describe the support needed to operate and maintain a system over its lifetime. In the context of the DoD acquisition process, sustainability involves using resources to minimize mission, human health, and environmental impacts and associated costs during the life cycle. This paper will present a draft version of "DoD Guidance - Integrating Sustainability into DoD Acquisitions," initial findings from pilot studies, and the challenges and road ahead. ... | 2014 | Life Cycle Costing |
| Cost Analysis & Optimization of Repair Concepts Using Marginal Analysis | Justin Woulfe | OPRAL is an analytical model for determining the optimal repair locations and spares allocations in a multi-level hierarchical support organization to optimize Life Cycle Cost. With this model, the analyst can either treat repair decisions as fixed and given as input parameters, or using the OPRAL algorithm, evaluate several different repair strategies in order to find the optimal one, considering all aspects of life cycle cost. ... | 2014 | Life Cycle Costing |
Military Construction Cost Estimating | Nicole Barmettler (Sullivan) | An informative presentation on construction cost estimating specifically dealing with military facilities. Within this topic, the author will identify, define, and explain the cost methodologies and cost adjustment factors considered when developing construction cost estimates for general military facilities. Project costs will be exemplified by illustrating a breakdown and walkthrough of the process. The author will specifically discuss the process involved in a five year facility acquisition timeline that is usually required for a typical major military construction effort, which is defined by a project cost exceeding $750,000. | 2014 | Methods and Models |
| Cost and Performance Trades and Cost-Benefits Analysis | Steven Ikeler | This paper will discuss the basics of Trades Analysis and the C-BA. It will include what the cost analyst should do to prepare and what information to collect during the Trades Analysis. One observation is that non-traditional Work-Breakdown Structures need to be considered and the analyst should model potential second and third order effects beforehand. ... | 2014 | Methods and Models |
| Lessons Learned from Estimating Non-Standard Equipment Returning from Overseas Operations | Michael Metcalf | This paper explores challenges in estimating the cost of retaining this equipment. Items that present unique challenges include: the type classification and full materiel release processes; repair, reset, and upgrade; short- and long-term storage; and knowledge retention and loss of wartime experience. We will also explore funding challenges; moving targets in number and configuration of retained systems; the transition of O&S from wartime contractor-based to peacetime organic; training; and system disposal and divestiture. | 2014 | Methods and Models |
| Weibull Analysis Method | Erik Burgess, James Smirnoff, Brianne Wong | The NRO Cost and Acquisition Assessment Group (CAAG) develops time-phased estimates for space systems in support of milestone decisions, budget formulation, and other program assessment tasks. CAAG relies on parametric budget-spreading models based on historical data to provide objective analysis and credible information before contract award or early in a program when there is little or no execution history available. ... | 2014 | Methods and Models |
| Study of Cost Estimating Methodology of Engineering Development | Myung-Yul Lee | The purpose of this paper is to study the actual engineering labor hours for FSED Program and PE/PI project and to create a parametric estimating model in a weapon system, especially cargo aircraft. The C-17 final assembly facility in Long Beach, California closes in 2015. It is worthwhile to review historical data for the C-17 FSED program and PE/PI project, both programs have engineering development program in view of estimating methodology research. | 2014 | Methods and Models |
| Validation and Improvement of the Rayleigh Curve Method | Michael Mender | The presentation begins with a brief overview of the Rayleigh method, followed by an explanation of R-squared and the issues with non-linear functions. We then discuss the method for linearizing the Rayleigh function and then using it for linear regression. We then provide a brief demo of the tool we developed and then move on to a discussion of our results pertaining to the performance of the Rayleigh method. ... | 2014 | Methods and Models |
| Rotorcraft Cost Model Enhancements for Future Concepts | F. Gurney Thompson III | This paper will discuss the ongoing research efforts to improve upon existing rotorcraft cost estimation capabilities, from both a cost estimating relationship (CER) update and a model development perspective. We will share our approach, many of our findings, and any lessons learned. Efforts currently underway include data collection, updates to existing cost models and their CERs, adding support for new aircraft types and technologies, and the addition of new analysis capabilities to better understand total ownership cost. | 2014 | Methods and Models |
| Kill Vehicle Work Breakdown Structure | nnifer Tarin, Christian Smart, Paul Tetrault | This paper provides an alternative to Appendix C: Missile Systems for inclusion in MIL-STD-881C, the Department of Defense standard for Work Breakdown Structures (WBSs). The Missile Defense Agency (MDA) produces interceptors that are similar to missiles with the exception of the payload element. Where Appendix C defines the payload element with a limited set of WBS elements, the MDA interceptor payload, referred to as a kill vehicle, includes a large collection of significant WBS elements. ... | 2014 | Methods and Models |
| Meet the Overlapping Coefficient: A Measure for Elevator Speeches | Brent Larson | You've seen this picture before. . . a plot of two overlapping probability distributions. You may have created one with an overlay chart. Typically this graphic contrasts two cost distributions so as to illustrate similarity, difference or change. However, seldom seen is a number that quantifies the overlap or area shared by both distributions. ... | 2014 | Methods and Models |
| Excel Based Cost Roll Up Method | Matthew Leezer | This paper will show the user how to create a custom function (sumlowerlevel) using Visual Basic and apply this function in Excel to generate a report that will save time and increase the accuracy of estimates. The method uses the lookup function to capture the cost of purchased components and subsystems and uses the custom function for all make assemblies to roll up the cost of the purchased parts. ... | 2014 | Methods and Models |
| The Role of Cost Estimating in Source Selection | Annette Barlia | The analysis will focus on the process of developing an IGCE and then utilizing it to evaluate vendor proposals for the acquisition of new technology. It will demonstrate that a strong IGCE facilitates source selection. If the cost estimate is developed the right way, organizations will have more leverage during contract negotiations with vendors, and the acquisition will run smoothly and meet program goals. ... | 2014 | Methods and Models |
| Automated Costing to Identify Opportunities to Reduce Direct Material Spend | Julie Driscoll, Dale Martin | This session will cover how technology is automating the costing process through integrating costing solutions with CAD, PLM and ERP; pulling information about components directly from CAD files; and using an intelligent cost engine that evaluates manufacturing routings for feasibility and cost effectiveness. We will look how these solutions enable automated batch costing of components and are used by manufacturers to support cost reduction projects. ... | 2014 | Methods and Models |
| Moving Beyond Technical Parameters in our CERs | Eric Druker, Charles Hunt | One of the frequent criticisms of JCL analysis (integrated cost and schedule risk analysis) has been that the results typically exhibit coefficients of variation (CV) that are orders of magnitude less than those seen in parametric estimates of similar scope. During a recent NASA research task examining how parametrics estimates can be linked to program management artifacts, the research team stumbled upon a characteristic of our Cost Estimating Relationships (CERs) that almost certainly leads our parametric estimates to have higher than necessary CVs. ... | 2014 | Parametrics |
| Using Dummy Variables in CER Development | Shu-Ping Hu, Alfred Smith | This paper explains the reasons for using dummy variables in regression analysis and how to use them effectively when deriving CERs. Specific guidelines are proposed to help analysts determine if the application of dummy variables is appropriate for their data set. This paper also demonstrates some common errors in applying dummy variables to real examples. An application using dummy variables in splines (to derive the fitted equation as well as the intersection) is also discussed. | 2014 | Parametrics |
| Bayesian Parametrics: Developing a CER with Limited Data and Even Without Data | Christian Smart | This paper discusses Bayes' Theorem, and applies it to linear and nonlinear CERs, including ordinary least squares and log-transformed ordinary least squares. | 2014 | Parametrics |
| Tactical Vehicle Cons & Reps Cost Estimating Relationship Tool | Cassandra Capots, Jeffery Cherwonik, Adam James, Leonard Ogborn | When estimating Operating and Support (O&S), it is reasonable to assume that as reliability increases, consumable and reparable parts ("cons and reps") cost should decrease (less frequent repairs), while as vehicle price increases, parts cost should increase (more expensive parts). Developing a dataset to support cost estimating relationships (CERs) for the Army's Tactical Vehicle fleet is a significant challenge. ... | 2014 | Parametrics |
| Unmanned Aerial Vehicle Systems Database and Parametric Model Research | Bruce Parker, Rachel Cosgray, Anna Irvine, Brian Welsh, Patrick Staley, Praful Patel | This handbook documents the first two years of research sponsored by NCCA and ODASA-CE. With the inclusion of UAS in the United States' (U.S.) military arsenal, the government has a desire to understand the components of a UAS including the air vehicle, GCS and payloads, the development and production process, and the O&S implications of these systems. ... | 2014 | Parametrics |
| Building a Complex Hardware Cost Model for Antennas | David Bloom, Danny Polidi | This paper discusses the development of a Complex Antenna Cost Model based on quantifiable sizing mechanisms which are designed to quickly and accurately calculate the "top-down" cost for all engineering and operations disciplines and functions required for antenna development and test. ... | 2014 | Parametrics |
| ESA Project Office Cost Model | Herve Joumier | This paper describes the definition and the implementation of a Project Office parametric cost model aimed at defining a reference manpower allocation based on fair judgement and rational fair modelling. It has been developed to improve the cost estimations capability of ESA providing outputs that are used by agencies for comparison with contractors' proposals. ... | 2014 | Parametrics |
| Improving the Accuracy of Cost Estimating Relationship for Software Systems | David Wang | In this paper, we leverage recently published results on the statistical characterization of schedule and cost risks to analyze the prediction accuracy of CER for software systems. Our analytical analysis and empirical statistical analysis of actual code size growth data suggest that the statistics of code size estimate can also be characterized by fat-tail distributions.... | 2014 | Parametrics |
| Hybrid Parametric Estimation for Greater Accuracy | William Roetzheim | This talk will discuss hybrid parametric estimation based on HLO catalogs, and give examples of the application and accuracy of this technique within organizations including the State of California, Halliburton, IBM, Procter and Gamble, and multiple top 25 financial institutions. | 2014 | Parametrics |
| Linking Parametric Estimates to Program Management Artifacts (LPEPM) | Mike Smith, Ted Mills, John Swaren | A common fate of parametric cost and schedule estimates is that they fall into disuse as a Project's own artifacts (e.g. Work Breakdown Structure (WBS), budget, schedule, risk lists, etc.) are created and mature. Parametric estimates typically do not map cleanly to WBS or schedule-derived artifacts, allowing a sense among Project Managers (PMs) ? rightly or wrongly ? that "parametric estimates are fine, but they don't reflect my project." ... | 2014 | Parametrics |
| Impact of Full Funding on Cost Improvement Rate: A Parametric Assessment | Brianne Wong, Erik Burgess | The NRO Cost and Acquisition Assessment Group (CAAG) currently houses data collected from various U.S. Government organizations, including the Department of Defense and NASA. These data points are pooled with NRO data and used in Cost Estimating Relationships for space hardware, which underpin CAAG estimates for major system acquisition programs, aiding in the development of programs and budgets. ... | 2014 | Parametrics |
| Developing R&D and Mass Production Cost Estimating Methodologies for Korean Maneuver Weapon System | Doo Hyun Lee, Sung-Jin Kang, Suhwan Kim | In this research we have attempted to establish a CER development process to meet the current need, and found certain cost drivers for the Korean historical maneuver weapons system data, using Forward selection, Stepwise Regression and R square selection. We have also developed a CER model for production labor costs, using Learning rate which has been generally applied to estimate valid production labor costs. ... | 2014 | Parametrics |
| Affordability Engineering for Better Alternative Selection and Risk Reduction | Marlena McWilliams, Bob Koury | This paper will outline the process and steps to how to implement affordability into your estimating environment to understand system requirements vs. system costs and affordability; and provide best value identifying and accepting the most affordable, feasible, and effective system or alternative. The need to evaluate and assign a best value is essential to both the government (DoD) and the contractors supplying systems / alternatives to the government. | 2014 | Risk |
| Critique of Cost-Risk Analysis and Frankenstein Spacecraft Designs: A Proposed Solution | Mohamed Elghefari, Eric Plumer | In this paper, we present a historical data driven probabilistic cost growth model for adjusting spacecraft cost Current Best Estimate (CBE), for both earth orbiting and deep space missions. The model is sensitive to when, in the mission development life cycle, the spacecraft cost CBE is generated. The model is based on historical spacecraft data obtained from the NASA Cost Analysis Data Requirements (CADRe) database. ... | 2014 | Risk |
| Risk Adjusted Inflation Indices | James Black | It is often observed that Office of the Secretary of Defense (OSD) inflation rates are different than prime contractor specific inflation rates seen in Forward Pricing Rate Agreements/Proposals (FPRAs/FPRPs) and in commodity group composite rates (e.g. Global Insight indices). Yet, it is a standard practice in many cost estimating organizations to use OSD inflation rates for escalating costs in estimates without giving consideration to a range of different possible inflation rates. ... | 2014 | Risk |
| Improved Decision Making with Sensitivity Analysis | Blake Boswell | In this study, we review common applications of SA methods to project estimation including a description of each method as well as its advantages and disadvantages. Additionally, we explore the topic of Global Sensitivity Analysis (GSA), which is a process for measuring the overall contribution of uncertain model inputs to variation in model outputs and is a popular technique for model validation in engineering and life sciences. ... | 2014 | Risk |
| Excel Based Schedule Risk and Cost Estimates | William Evans | The emphasis on robust cost and schedule estimating solutions has resulted in the creation of multiple solutions for analysts and clients. Excel based integrated cost and schedule risk is only one methodology for solving client problems. Incorporating cost and schedule risk in Excel leads to an increased ability to audit and trace the schedule and cost risk methodology throughout an Excel based PLCCE, improving the confidence and robustness of the estimate. While there are hurdles to the implementation of an Excel based schedule risk solution, when combined with form controls, the benefits to PLCCE auditability and usability are immense. | 2014 | Risk |
| Using Bayesian Belief Networks with Monte Carlo Simulation Modeling | Marina Dombrovskaya | One of the main aspects of creating a Monte Carlo simulation cost estimate is the accuracy in defining uncertainty and risk parameters associated with the cost components of the model. It is equally important to assess and accurately represent inter-dependencies between uncertain variables and risks, which are measured via correlation. Since oftentimes historical data is insufficient for a rigorous statistical analysis, both probability distribution and correlation are commonly estimated via a subject matter opinion. ... | 2014 | Risk |
| Expert Elicitation of a Maximum Duration Using Risk Scenarios | Marc Greenberg | This paper provides with a brief review on a current method to elicit a most-likely commute time, a "practical maximum" commute time and risk factors that contribute to commute delays. This paper continues by showing how these risk factors can be organized into an objective hierarchy of risk factors, leading to the creation of a customized risk work breakdown structure (WBS). ... | 2014 | Risk |
| Quantifying the Necessity of Risk Mitigation Strategies | James Northington, Christopher Schmidt, Chuck Knight | This paper will begin by highlighting flaws with the current risk management process, walk through the new proposed methodology for risk mitigation, and provide a quantitative example of the process in action using raw data. In the end, the proposed methodology will provide a greater understanding of program risks, a measurement of importance of implementing a risk mitigation strategy, a measurement of the mitigation strategy's subsequent impact, and a quantitative measurement of benefit for Program Mangers to defend their risk mitigation strategies. | 2014 | Risk |
| NASA JCL: Process and Lessons | Steve Wilson, Mike Stelly | Our paper will describe JCL implementation and address the creation, implementation, evolution, inherent benefits, inherent issues, its ultimate place among program management's decision-making toolset, and hard recommendations for organizations hoping to wage successful JCL campaigns. Real-world examples will be referenced, including those from the Constellation, Commercial Crew, and Orion spacecraft development programs. ... | 2014 | Space |
| NASA's Phasing Estimating Relationships | Chad Krause, Erik Burgess, Darren Elliott | Cost and schedule estimating in support of budget formulation is limited when cost phasing is not considered. As a result, NASA'a Office of Evaluation (OE) Cost Analysis Division (CAD) initiated a review of historic mission funding profiles for the purpose of corroborating current phasing profiles and optimizing future budgeting performance. Actual expenditures by year, technical parameters, and programmatic information were compiled and normalized from NASA's extensive library of CADRe (Cost Analysis Data Requirment) documents for programs since 1990. ... | 2014 | Space |
| Developing Space Vehicle Hardware Nonrecurring Cost Estimating Relationships at the NRO CAAG | Ryan Timm, Jan Sterbutzel | This paper builds on our 2012 SCEA conference briefing that described the NRO CAAG approach to developing Space Vehicle (SV) hardware Cost Estimating Relationships (CERs) for Nonrecurring (NR) engineering. These CERs are developed from the NRO CAAG's cost database of more than 2300 space hardware boxes, and can stand as alternatives to other popular parametric tools, like the nonrecurring CERs in USCM or NAFCOM. ... | 2014 | Space |
| A Next Generation Software Cost Model | Jairus Hihn, Tim Menzies, James Johnson | In this paper we will summarize our findings on effort/cost model estimation and model development based on ten years of software effort estimation research using data mining and machine learning methods. We will then describe the methodology being used in the development of a NASA Software Cost Model that provides an integrated effort, schedule, risk estimate, as well as identifying the changes in the project characteristics that are most likely to improve a given projects cost-schedule performance and risk exposure. | 2014 | Space |
| The NASA Project Cost Estimating Capability | Andy Prince, Brian Alford, Blake Boswell, Matt Pitlyk | The paper begins with a detailed description of the capabilities and shortcomings of the NAFCOM architecture. The criteria behind the decision to develop the PCEC are outlined. Then the requirements for the PCEC are discussed, followed by a description of the PCEC architecture. Finally, the paper provides a vision for the future of NASA cost estimating capabilities. | 2014 | Space |
| NASA Instrument Cost Model (NICM) | Hamid Habib-Agahi, Joseph Mrozinski, George Fox | The NASA Instrument Cost Model (NICM) includes several parametric cost estimating relationships (CERs) used to estimate NASA's future spacecraft's instrument development cost. This presentation will cover the challenges associated with creating cost models in an environment where data on previously built instruments is 1) sparse, 2) heterogeneous and 3) book-kept differently by the various NASA centers and support institutions. ... | 2014 | Space |
| New Air Force Integrated Baseline Review (IBR) Process - A Quick Reaction Capability (QRC) Perspective | James Ross, Martin Levitan, Christine Bolton, Barbara Meyers | Since the early 1990s, the U.S. Air Force has been using Integrated Baseline Reviews (IBR) as a key technique for early assessment of the efficiency and effectiveness of baseline plans, resource allocations, scheduling and costs. Much of the earlier IBR doctrine does an excellent job describing what is required to complete these assessments, but little detail had been written about the specific processes, procedures, steps and techniques required to undertake these assessments. The January 2012 "Air Force Integrated Baseline Review (IBR) Process Guide", developed by SAF/AQXC personnel, is one of the best documents to date to address this concern. A drawback of this document is that the described IBR Process is based primarily on experiences with, and lessons learned from, the Acquisition Category (ACAT) I KC-46 Tanker Program... | 2013 | Earned Value Management |
| The Use of Function Points in Earned Value Management for Software Development | Ben Netherland, Mike Thompson, Dan French | In this presentation, the authors detail their efforts in the development of an EVM methodology for a government software development project utilizing the International Function Point User Group (IPFUG) function point software sizing metric. Traditionally it has been difficult to apply Earned Value Management (EVM) criteria to software development projects, as no tangible value is earned until the software is delivered to production... | 2013 | Earned Value Management |
| Measuring and Managing Organizational Performance in the Government | Kellie Scarbrough (Wutzke) | In this presentation, the author details the challenges that government agencies face when attempting to measure organizational performance and presents a general methodology for successfully implementing a robust performance measurement program. The majority of government agencies do not operate like private industry and face unique challenges when defining meaningful metrics and collecting performance data... | 2013 | Earned Value Management |
| Understanding Requirements for Subcontract EV Flow Down and Management | Mark Infanti | Do you use Earned Value Management (EVM)? Do you have a government customer that is requiring you to use EVM and report the data monthly? In either of these situations, if you have a subcontractor (SC), you need to know how they will report to you. You need to have a plan on how you will incorporate their data into your data for a complete program understanding. If your SC reports to you at cost level how do you add the fee to their report? How do you add your companies general and administrative (G&A) overhead to their costs? Do you share your Integrated Master Schedule (IMS) with them or just the key milestones?... | 2013 | Earned Value Management |
| Cost Estimation and Earned Value Integration | Sissy Gregg, Michelle Ehlinger | Traditionally, cost estimators and earned value analysts work separately from one another, focusing on their own individual discipline. We have found synergies by merging these disciplines together resulting in stronger analysis through improved methods, more robust data sets, and new tools. The benefits of cost estimating (CE) and earned value (EV) management integration are realized throughout the acquisition lifecycle and enhance senior leadership decision making and improve acquisition outcomes. This paper presents the proven benefits of CE/EV integration, including more accurate cost estimates, cost estimating relationships, improved program management support, and cross program assessments... | 2013 | Earned Value Management |
| Time is Money: The Importance and Desired Attributes of Schedule Basis of Estimates | Justin Hornback | "Time is Money" is a maxim made popular by Benjamin Franklin. It reflects the long understood importance of schedule requirements and their impact on cost. While the importance of cost basis of estimates (BoE) have gained in popularity among industry communities, schedule BoEs are, at least equally, if not more important as cost, but have yet to reach the same level of understanding of importance within the industry... | 2013 | Earned Value Management |
| Estimating Cost To-Go Without Stable EVM Data | Peter Frederic, Ronald K. Larson | Initial independent estimates for development programs are typically created using parametric methods. As a program progresses, it becomes important to separate sunk cost from cost to-go. In the very early stages of a program, it is acceptable to estimate cost to-go by simply subtracting the reported sunk cost from the parametrically-estimated total cost. However, as the program continues to progress and more significant progress accumulates, we can no longer assume that dollars spent equate directly to technical progress. It is then necessary to understand progress to-date in order to estimate cost to-go... | 2013 | Earned Value Management |
| The [Whole] Truth about ANSI-compliant EVMS | Michael Nosbisch | Over the past several years, there have been multiple presentations by government agency representatives and consultants alike that have characterized an EVMS that complies with the ANSI/EIA 748(B) Standard as simply "good project management," and as such should not be that difficult for most successful government contractors to implement. It is true that the basic concepts and principles of EVM can absolutely be likened to sound project management. However, the costly, time-intensive process of implementing, validating, and then maintaining an "ANSI-compliant" EVM system is far and above what any contractor would do if it were not a contractual requirement that is also fully reimbursable. ... | 2013 | Earned Value Management |
| Cloud Computing Starter Kit: Cost and Business Case Considerations | Jennifer Manring, Raj Agrawal | Cloud computing is a major trend in the information technology (IT) industry that is affecting both commercial companies and Federal Government agencies due to its promise of agility, scalability, and cost savings. Prior to investing in cloud computing, decision makers are asking "what is my return on investment (ROI)?' However, there is no one size fits all answer to this question and caution should be exercised when applying generic rules of thumb and quick assessment cost and economic models. Costs, benefits, and risks must be carefully assessed for each situation... | 2013 | eTrack |
| Got a Defense Business System? The effect of the new defense business system policy on cost estimating, tools, and funding | Charles Gu | The Department of Defense (DoD) budgets over $7 billion a year for business system investments and has mandated that all information technology systems must be assessed and certified through a formal process before funding can be obligated in FY 2013. This new law, Section 901 of the Fiscal Year 2012 National Defense Authorization Act (FY2012 NDAA), significantly expands the scope of systems requiring certification to include any business system with a total cost in excess of $1M over the period of the current future-years defense program, regardless of type of funding or whether any development or modernization is planned... | 2013 | eTrack |
| How to Estimate and Measure "the Cloud" and Make COCOMO Cloud Enabled | David Seaver, John Rosbrugh | Cloud and Big Data have become the new buzzwords in the commercial, federal and DoD technology arenas. This presentation will: 1. Define Cloud & Big Data 2. Identify what's new and different with the Cloud and Big data. 3. Identify the cost drivers for cost estimates... | 2013 | eTrack |
| Utilization of Visual Basic in Cost Estimating and Analysis Tools - Anyone Can Cook | Jeremy Eden | As collaborative environments become more prevalent in all industries, the cost estimating and analysis industry is no exception to this movement. Increasing amounts of pressure are put upon cost estimators and analysts to develop tools that are low cost, fast, robust in design, have long standing methodologies, do not require proprietary software or licenses, and are and easy to use by both the advanced cost estimator looking for maximum control and the novice simply trying to diligently support the early stages of a development program... | 2013 | eTrack |
| Methods to Analyze Services Portfolio Cost Drivers and Efficiencies | Valerie Reinert, Virginia Wydler | This presentation provides a statistical methodology used to evaluate the "health" of a services portfolio through performance data analysis and cost analysis, with the goal to identify cost drivers and efficiencies for performance improvement. Managing a Federal government portfolio is a complex endeavor. Portfolio Managers have a major responsibility to evaluate whether their portfolio is operating efficiently and effectively. Thus, they need to use analytical procedures that, when executed, will evaluate the "health" of the portfolio and identify areas for increased efficiency and effectiveness. If the analysis is not data-driven, this leads to a lack of proper business case analysis and cost benefit analysis to weed out bad actors in the portfolio... | 2013 | eTrack |
| Touch Labor Estimate Modeling | Michael Yeager, Lyle Davis | To support its mission for the F-35, the production cost estimating team developed touch labor estimate models which include the flexibility to run production rate effects, loss of learning, commonality adjustments, affordability initiative and outsourcing impacts, multiple learning curve break point analyzes, and estimates for touch labor in hours or by realization. Given the scrutiny to the F-35 Lightning II program by the Department of Defense and Congress detailed and accurate cost modeling allows for better budgeting and more credibility within the services themselves and with the American public... | 2013 | eTrack |
| Innovative Business Agreements and Resulting Cost & Pricing Methods at NASA in Support of New Commercial Programs | James Roberts, Torrance Lambing | On April 15, 2010 President Obama delivered a speech at Kennedy Space Center in which he outlined his new vision for the U.S. space program. Emphasis was placed on enabling the exploration of Space by Commercial entities instead of by the Government. Since that time, NASA's role has changed in many instances from being a program manager-overseeing development of space launch hardware and conducting space exploration missions-to one of support and a provider of space-related facilities and infrastructure... | 2013 | eTrack |
| Capacity Cost Model: Balancing the Demand for Software Changes Against the Supply of Resources | Chris Kaldes | A Program Executive Office (PEO) within the Department of Defense (DoD) is responsible for multiple information systems that support the Services (Army, Navy, and Air Force). This PEO is not only responsible for Operations and Maintenance (O&M) of the information systems, but also for implementing system change requests that are made by the Services' Senior Service Representatives (SSRs). The PEO was relatively satisfied with the status quo, but the executives within the PEO recognized that there were three problems with doing business today. (1) Currently, there was no way to scientifically indicate/prove to the SSRs what software changes they could promise... | 2013 | Information Technology |
| Cloud Computing and Big Data - What's the Big Deal? | Arlene Minkiewicz | At one level, cloud computing is just Internet enabled time sharing. Instead of Information Technology (IT) organizations investing in all of the hardware, software and infrastructure necessary to meet their business needs, cloud computing makes access to them available through the Internet. On the one hand, cloud computing allows an organization to adopt a different economic model for meeting IT needs by reducing capital investments and increasing operational investments. On the other hand, cloud computing enables the capture, storage, sharing, analysis and visualization of huge amounts of data in multiple formats from multiple media type: Big Data analysis... | 2013 | Information Technology |
| ODASA-CE Software Growth Research | Kevin Cincotta, Lauren Nolte, Eric Lofgren, Remmie Arnold | For several years, the Office of the Deputy Assistant Secretary of the Army for Cost and Economics (ODASA-CE) has used a single growth factor to account for size growth in weapon system software development estimates. This factor is invariant to program characteristics and may, for example, lead to excessive growth estimates for large programs, whereas experience suggests that these grow less in percentage terms than their smaller counterparts. Over the past year, ODASA-CE worked with Technomics, Inc. to research improved methodologies for incorporating growth in software estimates... | 2013 | Information Technology |
| Developing A Business Case for Cloud | Cynthia O'Brien, John Bell | Commercial organizations and their customers have embraced cloud-based data management solutions with astonishing speed. Government agencies are not far behind, spurred on by the Office of Management and Budget's "Cloud First" Federal IT policy urging agencies to look to cloud-based solutions whenever possible. Government decision makers are moving to cloud solutions not only because of pressure from OMB and end users; they are also intrigued by the promise of dramatic cost savings. But can a decision to pursue a cloud transition be based on the promise of cost savings alone? There have been numerous examples where transitions to cloud have not yielded the 40% or more expected cost savings... | 2013 | Information Technology |
| Estimating Real-Time Software Projects with the COSMIC Functional Size Measurement Method and the ISBSG Repository | H.S. van Heeringen | Nowadays, software project estimation of administrative software systems has evolved into quite a mature stage. Standardized functional size measurement methods, like IFPUG and NESMA, are used to measure the functionality that the software application is going to offer to the user and this functional size is considered to be the main cost driver for administrative software projects. These methods are suitable for project estimation, because they measure the functional user requirements (which should be described before the project starts), instead of the actual delivered product (for instance source lines of code, which can only be measured after the project finished)... | 2013 | Information Technology |
| The COSMIC Functional Size Measurement Method: An Introduction | H.S. van Heeringen | The COSMIC functional size measurement method is a relatively new and not very well known method to measure the functional size of pieces of software. The main advantages of this method over the existing methods (like IFPUG and NESMA function point analysis) is that COSMIC is designed to also measure other types of software than the traditional administrative software. COSMIC can therefore be used to measure real-time, embedded, infrastructure software, as well as administrative software. Another advantage is the continuous scale that is used in the COSMIC method, which makes it possible to accurately measure the difference in size between different functional processes... | 2013 | Information Technology |
| Software Maintenance Cost Estimating Relationship Development for Space Systems | Ryan Timm | Methods development for Space Vehicle (SV) cost estimating has mainly focused on acquisition cost, as acquisition makes up the majority of a SV's lifecycle costs and has high interest from oversight agencies and congress. Tightening budgets and increased space vehicle life expectancy has bolstered the need to further refine methods for estimating SV maintenance costs... | 2013 | Information Technology |
| Estimating Software Development Costs for Agile Projects | Blaze Smallwood, Omar Mahmoud | Addressing the need to more rapidly develop and field capabilities for the warfighter, more and more software-centric DoD programs are transitioning towards an industry trend called "Agile" software development. While "Agile" software development is geared towards producing usable software products more rapidly than traditional waterfall or incremental methods, it also requires more flexibility with managing requirements. The main challenge this has created for program managers is figuring out how to effectively manage scope, cost, schedule, and performance in this flexible, fast-paced development environment in which requirements are more fluid. In turn, cost estimators have been challenged to develop new data collection approaches and estimating methodologies to more effectively estimate software costs for these "Agile" programs... | 2013 | Information Technology |
| Domain-Driven Software Cost, Schedule, and Phase Distribution Models: Using Software Resource Data Reports | Wilson Rosa, Barry Boehm, Brad Clark, Joe Dean, Ray Madachy | Instead of developing cost and schedule estimation models with many parameters, this paper describes an analysis approach based on grouping similar software applications together called Productivity Types. Productivity types are groups of application domains that are environment independent, technology driven, and are characterized by 13 product attributes. Also consideration is given to the operating environment that the software operates within. Over 196 actual software projects from DoD's Software Resource Data Reports (SRDRs) were fully inspected and analyzed to produce a comprehensive set of Cost Estimation Relationships, Schedule Estimation Relationships, Software Productivity Benchmarks, and Best Practice Data Normalization Guide. Analysis results will be discussed in this presentation... | 2013 | Information Technology |
| Software Maintenance: Recommendations for Estimating and Data Collection | Corinne Wallshein, Bruce Parker, Vanessa Welker, Thomas Harless, Peter Braxton | Entering a period of fiscal austerity, it becomes more important than ever to estimate and consider operating and support (O&S) costs, which represent the lion's share of life cycle cost (LCC) for most platforms, during acquisition. Given the ubiquity of software in today's complex programs, a key component of O&S is software maintenance. This paper will discuss our work to understand the process of software maintenance, data collected and normalized to date, and resultant benchmarks for use in developing and cross-checking software maintenance estimates. It represents an update to "Software Maintenance Data Collection and Estimating Challenges" (Welker, et al., SCEA/ISPA, 2012)... | 2013 | Information Technology |
| Estimates of Unit Cost Reductions of the F-16 Fighter as a Result of U.S. Arms Export Production | Andrew A. Yeung, Keenan D. Yoho, Jeremy Arkes | Arms exports have increasingly become an attractive option for reducing escalating unit costs of new weapon systems to the United States Department of Defense. However, while there is no lack of conjecture, there is little data that show weapon system costs to the United States actually decrease when the same weapon is sold to a foreign buyer. | 2013 | Journal of Cost Analysis and Parametrics |
| Accuracy Matters: Selecting a Lot-Based Cost Improvement Curve | Shu-Ping Hu, Alfred Smith | There are two commonly used cost improvement curve theories: unit cost theory and cumulative average cost theory. Ideally, analysts develop the cost improvement curve by analyzing unit cost data. However, it is common that instead of unit costs, analysts must develop the cost improvement curve from lot cost data. An essential step in this process is to estimate the theoretical lot midpoints for each lot, to proceed with the curve-fitting process. | 2013 | Journal of Cost Analysis and Parametrics |
| A Macro-Stochastic Model for Improving the Accuracy of Department of Defense Life Cycle Cost Estimates | Erin Ryan, Christine Schubert Kabban, David Jacques, Jonathan D. Ritschel | The authors present a prognostic cost model that is shown to provide significantly more accurate estimates of life cycle costs for Department of Defense programs. Unlike current cost estimation approaches, this model does not rely on the assumption of a fixed program baseline. Instead, the model presented here adopts a stochastic approach to program uncertainty, seeking to identify and incorporate top-level (i.e., macro) drivers of estimating error to produce a cost estimate that is likely to be more accurate in the real world of shifting program baselines. | 2013 | Journal of Cost Analysis and Parametrics |
| Comparison between the Mix-Based Costing and the Activity-Based Costing Methods in the Costing of Construction Projects | Leandro Torres Di Gregorio, Carlos Alberto Pereira Soares | After bibliographic research on costing methods in civil construction, and a presentation of the mix-based costing method, as well as an application of the activity-based costing method in the costing of civil construction projects based on other authors, a possible application of the mix-based costing was sought. This method allows the distribution of costs and indirect expenses to products without the subjectivities and uncertainties typical of traditional apportionment, by means of analyses of different production scenarios. | 2013 | Journal of Cost Analysis and Parametrics |
| Galaxy Charts: The 1,000-Light-Year View of the Data | Robert Nehring, Katharine Mann, Robert Jones | This article presents a new kind of chart, called a Galaxy chart, which combines the strengths of other chart types. A Galaxy chart displays an entire Cost Element Structure on a single sheet of paper, showing all of the elements, their relationships, and their costs in a visually appealing way. Each child cost element is in orbit around its parent, with its children in orbit around their parent. The size of each cost element is directly proportional to its magnitude. | 2013 | Journal of Cost Analysis and Parametrics |
| Feasibility of Budget for Acquisition of Two Joint Support Ships | Erin K. Barkel, Tolga R. Yalkin | The mandate of the Parliamentary Budget Officer is to provide independent analysis to Parliament on the state of the nations finances, the governments estimates, and trends in the Canadian economy, and, upon request from a committee or parliamentarian, to estimate the financial cost of any proposal for matters over which Parliament has jurisdiction. | 2013 | Journal of Cost Analysis and Parametrics |
| Can DoD Inflation Indices and Discounting Processes Be Improved? | Kathryn Connor, James Dryden | Currently the DoD is facing an uncertain budget environment. This will have an impact on what the DoD can spend for acquisition programs and sustainment of major weapons systems. Current practices for inflation and discounting skew program affordability, especially during operations and sustainment. In this presentation, we look at how well current inflation indices and discount rates serve programs today and whether there are strategies to improve the accuracy of these estimates. After examining the experience of several major weapons systems we have identified potential policy changes and strategies for cost estimators to employ on inflation and discounting. We believe that these can improve a program's understanding of long run affordability and potential risks associated with inflation and discounting. | 2013 | Life Cycle Cost |
| Lessons Learned from the Joint STARS Analysis of Alternatives for Cost and Risk Analyses | Daniel Mask, David Stem | The Joint Capabilities Integration and Development System (JCIDS) plays a key role in identifying the capabilities required by the warfighters to support the National Security, Defense, and Military Strategies. The Analysis of Alternatives (AoA) is a documented evaluation of the performance, operational effectiveness, operational suitability, and estimated costs of alternative systems to meet a capability need that has been identified through the JCIDS process. As weapon system complexities and their interdependencies between other systems increase in the collaborative warfighter space, this evaluation of mission systems and technologies has become more and more challenging... | 2013 | Life Cycle Cost |
| Analysis of Large O&S Proposal: Lessons Learned! | James Linick | Recently, a $20M contract was awarded for weapon system O&S. The contractor's proposal was subjected to technical evaluation and extensive negotiations ensued. Over 100 BOEs were evaluated for multiple CLINs. Proposal cost components were labor, materials and ODCs. This presentation will focus on how a final agreement was reached, focusing on: 1) Proposal construction, what are CLINs, BOE's and how sub-contractor work was credited, 2) BOE technical evaluations, the delineations and weighting of on-site and offsite work, 3) Forward price rate agreements, what they are, and the difference between them and final negotiated labor rates, 4) Cost to price calculations, how they are performed and what are Fringe, G&A, Overhead, Fee and Cost of Money and 5) How final contract price and weapon system CONOPS cost estimate compared... | 2013 | Life Cycle Cost |
| The Dynamic Economic Model: A Flexible Approach to Investment Analysis with Analytica | Kevin King, Jason Mehrtens | Given the current economic crises facing the United States, government spending is being placed under more intense scrutiny. As a result, major investments are being more closely analyzed and pressure to be cost effective is of the utmost importance. In the world of cost estimation, the scrutiny is translating to increased demand for cost metrics prior to the establishment of clear scope requirements. For MCR, uncovering new tools and strategies to adapt to this type of environment has become a top priority. One way MCR improved versatility while maintaining our valued reliability has been to create economic cost models using Analytica®, a modeling software designed to enhance the capabilities of the modern spreadsheet tool... | 2013 | Life Cycle Cost |
| The Forgotten Costs and Uncertainty Analysis within the DoD Acquisition Environment | Roberto Perez, Elizabeth Evans | Despite the best efforts of the Department of Defense (DoD) to capture all costs associated with the life cycle of acquisition programs, it continues to be an elusive. A primary reason for this shortcoming in cost estimating is the lack of time spent on estimating operations & sustainment (O&S) costs compared to the time and effort estimating research, development, and procurement costs. The implications for this imbalance are significant given the fact that 60 - 70 percent of an acquisition program's life cycle cost is comprised of operations and support... | 2013 | Life Cycle Cost |
| Life Cycle Cost Estimate (LCCE) Assessments | Casey Trail, David Brown, Colleen Craig | This presentation summarizes a scorecard tool and criteria that are used to conduct assessments. The criteria, which are based on the GAO Cost Estimating and Assessment Guide, include four characteristics, which are further broken in to 13 criteria and 75 sub-criteria. Lessons learned, and key findings are presented as examples of LCCE characteristics associated with both positive and negative evaluation outcomes. The presentation is of interest to government and contractor cost estimators, as well as anyone from the cost community wishing to leverage best practices and tools related to life cycle cost assessment. Participants with cost assessment experience are encouraged to share their experiences and lessons learned... | 2013 | Life Cycle Cost |
| Construction vs. Systems Acquisition Cost Estimating: A Comparative Analysis | Tom Sanders, Steve Essig, Tim Anderson | The continued push to improve DOD's acquisition processes includes several thrusts-regulatory streamlining, improving use of incentives, and workforce optimization. These are not new. To deal with these concerns, the acquisition community has a unique opportunity. DOD Cost estimators have recently received a strong "vote of confidence" in the acquisition community; new positions have been created and existing positions have been protected. That is not necessarily so in the construction industry, another large employer of cost estimators. The construction industry has undergone a transformation over the past six years, beginning with the housing collapse in 2006 and enduring through the years of the Great Recession... | 2013 | Management |
| The Joint Integrated Analysis Tool (JIAT)- Making Data Sharing Easier | Niatika Griffin, Melissa Cyrulik, John McGahan | The silver lining in the current DoD budget turmoil is that it provides us with an opportunity to reexamine how we operate and look for ways to be more efficient. In times of belt tightening, improving access and distribution of established resources is imperative. We are challenged to ultimately improve how we share cost resources. For cost professionals, these resources include databases of cost and technical information, libraries of resources, and approved models that can be used rather than re-invented. The Office of the Deputy Assistant Secretary of the Army - Cost & Economics (ODASA-CE) has invested in developing a tool that facilitates secure access to data needed by the cost analyst... | 2013 | Management |
| Strategic Value of the Business Case Analysis (BCA) | Eric Buller | The impact of sequestration has become the main topic of discussion in almost all circles in the Department of Defense since the Budget Control Act passed into law in 2011. In response to reduced budget sequestration, the DoD has been interested in exploring potential cost reductions in weapon system operations. In nearly all lifecycle costs, sustainment generally makes up 60% to 80% of the total program costs. In a defense world seeking increased efficiencies and budget savings it is imperative to examine effective sustainment strategies... | 2013 | Management |
| Back to the Big Easy: Revisiting Hilbert's Problems for Cost Estimating | Peter Braxton, Richard Coleman | At the International Congress of Mathematicians at the Sorbonne in Paris in 1900, German mathematician David Hilbert boldly put forth a list of 23 theretofore unsolved problems in mathematics, which subsequently became quite influential in 20th-century research efforts. At the Joint SCEA/ISPA Conference in New Orleans in June, 2007, the authors audaciously emulated Hilbert with a list of 21 problems for cost estimating and risk analysis... | 2013 | Management |
| Net Present Value (NPV): The Basics and the Pitfalls | Kevin Schutt, Nathan Honsowetz | One of the essential metrics of an economic analysis is the Net Present Value (NPV) measure. While it is a fairly simple calculation, made even more routine and efficient by modern technology, be wary of pitfalls in its application. This presentation will cover the basic components and methodology of NPV and discuss the pitfalls and easily-made mistakes in its application. These include incorrect discount rate formulas, inconsistent treatment of inflation, incorrect understanding of the discount rate, incorrect application of Excel's "=NPV()" formula (hint: it's not really "Net"!), and including irrelevant cash flows. Come learn about Net Present Value and avoid these potentially career-limiting mistakes! | 2013 | Management |
| Fifteen Undeniable Truths About Project Cost Estimates, or Why You Need an Independent Cost Estimate | Tim Anderson | All project cost estimates are wrong, some more so than others. It is exceedingly difficult to accurately estimate the cost of a large project (e.g., major defense acquisition project). Reasons for this are manifold, but the primary causes seem to be general uncertainty, insufficient technical detail, unanticipated design/technical changes, funding and schedule perturbations over the acquisition life cycle, other programmatic "difficulties," and otherwise attempting to predict the future with imperfect information. The end result is that project cost estimates tend to be too low, with early estimates being the most likely to be understated... | 2013 | Management |
| Cost Engineering Heath Check ... How Good are your Numbers? | Dale Shermon, Mark Gilmour | High quality cost estimating gives a business leader confidence to make rational financial decisions. Whether you are a business leader or a cost estimating manager, you have a vested interest in understanding whether you can depend on your organisations ability to generate accurate cost forecasts and estimates. But how can business leaders be confident that the cost information that they are being provided is of high quality? How can a cost estimating manager be sure that their team is providing high quality cost information?... | 2013 | Management |
| Sailing Blind: Data-Driven Estimating at the Concept Phase, a Case Study of Canada's Joint Support Ship Project | Tolga Yalkin, Erin Barkel | Canada's lack of recent experience in naval shipbuilding and the unique nature of the National Shipbuilding Procurement Strategy (NSPS) present unique estimating challenges. A methodology was needed that was at once data-driven and defendable while at the same time able to address shipbuilding realities at a very high level conceptual stage. This paper outlines a case study where PRICE Systems and the Canadian Parliamentary Budget Officer (CPBO) developed a methodology and model to address these challenges for the Joint Support Ship (JSS) project... | 2013 | Management |
| Supplier Cost/Price Analyses | David Eck, Todd Bishop | Under United States Federal Government contracts, an important but overlooked aspect is the audit and evaluation of a subcontractor's cost proposal. Requirements in the Federal Acquisition Regulations (FAR) provide for certain visibility into subcontractor's cost proposal depending on the value of the proposal. Additionally it is critical to note the FAR puts the onus of subcontractor cost (or price) analysis on the prime contractor. The expectation is that the same level of detail that the prime includes in its proposal should be included as support to a subcontractor's cost proposal. Effectively the prime must perform subcontractor cost analysis in accordance with the FAR... | 2013 | Management |
| National Reconnaissance Office (NRO) Experiences with the Implementation of Will Cost/Should Cost Management | Sissy Gregg, Greg Lochbaum, Erik Burgess | The objective of this presentation is to address how to assess and incorporate contract performance in an Estimate to Complete, for the purpose of determining a reasonable amount of margin and how that margin should be managed and controlled. The discussion will go beyond cost and schedule variance to forecast total program cost including future requirements, other government costs, etc... | 2013 | Management |
| Avoiding Pitfalls When Applying Learning to your Estimate | Andrew Busick | Most cost estimators understand the basics of learning curves, but we sometimes fail to recognize the implications of the way that we apply learning. This presentation points out some major pitfalls and mistakes that can be made when dealing with learning. We address both deriving learning curves from historical data and applying learning to a cost estimate. Additionally, we recommend best practices to use when dealing with these issues... | 2013 | Methods and Models I |
| Cost Effective Analysis: The Role of Discounting in Government Investing | Brandon Shepelak | In order for the government to best fulfill its obligation to efficiently spend tax dollars, it uses various analysis methods to determine which investment to choose. Cost effectiveness analysis is often the preferred analysis method when an investment is not being judged as to the value added but to determine the least expensive spending alternative. According to the Office of Management and Budget (OMB) in circular A-94, an investment is cost effective when, comparing competing alternatives, its cost are the lowest as expressed in present value terms for a given amount of benefits (OMB, 2011)... | 2013 | Methods and Models I |
| Learning Curve Analysis of Small Data Sets - Spacecraft Bus Cost Improvement Analysis | Brian Welsh, James York | Recent initiatives to implement cost savings across DoD have produced a number of potential opportunities. Within the Space Portfolio one such opportunity is to acquire satellites more efficiently through block buys and limiting production breaks. To quantify these opportunities, we can analyze historical cost experience. This paper describes three approaches used to pool cost improvement data from multiple programs, allowing for regression of all data simultaneously. The approaches are (1) scaling based on weight, (2) normalizing to a common theoretical first unit, and (3) the use of program binary ("dummy") variables... | 2013 | Methods and Models I |
| Building an Analysis Schedule " Lessons Learned from the SGSS Program | Matt Blocker, David Jacintho | Constructing an integrated cost and schedule uncertainty model involves bringing together important program elements: cost, risk, and schedule. The model's core is the schedule; the linking of costs and risks to the schedule is what allows the implementation of time-dependent costs and, ultimately, an analysis that jointly considers cost and schedule. It follows, then, that one of the first, and often largest, difficulties in constructing an integrated cost and schedule model is the development of the schedule backbone... | 2013 | Methods and Models I |
| A Novel Non-Recurring Production CER Methodology | Lisa Hackbath, Raymond Covert | Production costs are generally categorized as either non-recurring or recurring. Typically non-recurring costs include tooling and pre-production activities, among others. Cost estimating relationships (CERs) are generally developed first for recurring hardware costs, and then non-recurring CERs are developed as a function of recurring hardware costs... | 2013 | Methods and Models I |
| Data Collection and Analysis Supporting Defendable Cost Estimates | Arlene Minkiewicz | Cost modeling and estimation has a long and interesting history in the Aerospace and Defense industry starting around the time of World War II. All sorts of mathematical and experiential models have been proposed and used over the years to help with bidding, planning, proposing and executing contracts. While general purpose models are useful, more and more industry and government cost professionals are asking for models built with data very specific to their industry and their organization. Unfortunately many organizations do not have the infrastructure, processes or tools for collecting project data efficiently... | 2013 | Methods and Models I |
| Getting (and sharing!) the FACTS: Factors, Analogies, CERs and Tools/Studies | Daniel Harper, Ruth Dorr | One of MITRE's corporate values is "People in partnership." MITRE values [...] partnership with the government, collaboration within and without..." We at MITRE have been charged via our corporate goals to "Apply technical and engineering excellence," by bringing to the customer the best thinking possible by "[...] tapping into a deep technical base, both within MITRE and globally, across the breadth of industry and academia." ... | 2013 | Methods and Models I |
| Aircraft Operation - Cost Models; Lessons Learned! | James Linick | In FY10 the Air Force Cost Analysis Agency (AFCAA) asked one of its cost support contractors to study the processes in place for producing specific aircraft cost products and implement improvements, specifically concentrating on format consistency, maintainability and automation. The periodically produced products to be studied were: 1) Aircraft Operation and Sustainment Cost Estimates 2) Cost per Flying Hour Risk Models 3) Jet Engine Maintenance Procedures versus Time on the Wing... | 2013 | Methods and Models I |
| Implementing a Data Collection Tool to Estimate and Analyze Spending | Thomas Brooks, Daniel Mask | As budgets and resources shrink, the effort to better understand spending has become a priority to many decision makers within the Marine Corps. Since Operation Iraqi Freedom (OIF) began in 2003, military organizations have been able to request and receive immediate Overseas Contingency Operations (OCO) funding without following traditional procurement processes. This readily available funding has led to the emphasis of developing and maintaining operational capability in lieu of program estimates and resource management. The lack of quality budget estimates and use of financial tracking mechanisms has made it difficult for Marines to strategically allocate budget cuts based on efficiencies or priorities; more often than not, budget reductions are distributed uniformly across the organization... | 2013 | Methods and Models I |
| The Dashboard Tool: Taking Cost Analysis to the Next Step By Combining Costs with Capabilities to Evaluate COAs | John Ko | Although life cycle cost estimates can provide a comprehensive view of the costs for a program, cost estimates do not provide the entire picture needed to make sound business decisions because they only focus on the costs for a program. To determine how programs are actually performing, the system's capabilities must be linked with the program's costs to properly quantify how programs meet their requirements with respect to cost. Booz Allen has developed an Excel Based tool called Dashboard that estimates both the program's costs and the capabilities achieved for a program... | 2013 | Methods and Models I |
| Realizing the True Cost of Energy - Keeping the DoD Green | John Ko | In recent years, the demand for energy as well as volatility in costs of energy for the Department of Defense has prompted individual programs to monitor these costs closely in order to identify new ways to control these costs. In response to spikes in energy costs in recent years, major programs are now evaluating different alternatives to improve fuel efficiency and ultimately reduce the military's dependence on fuel... | 2013 | Methods and Models I |
| Cost Estimating Tips: Learn Tricks in Excel and Best Practices | Eric Hong | This presentation for the 2013 International Cost Estimating and Analysis Association (ICEAA) Conference is to present Microsoft Excel tricks and best practices for cost estimating. Many of the presentations at the conference are insightful and show the amazing cost estimating products that are being developed, but the knowledge gained from those presentations usually cannot be applied on a daily basis. The goal for this presentation is to provide the attendees efficient and more effective ways of modeling... | 2013 | Methods and Models I |
| Galaxy Charts: Depict and Color Your WBS in a Meaningful Way | Robert Nehring, Katharine Mann, Robert Jones | For centuries, we have searched for new ways to display our thoughts and ideas in ways that will allow the viewer to easily digest and understand our point of view. Displaying quantitative data in interesting yet meaningful ways is no different. In fact, discerning how to display your entire Work Breakdown Structure (WBS) succinctly and clearly has proven to be very difficult. One answer to this challenge is called the Galaxy Chart, which shows both relationships and magnitudes on a single chart... | 2013 | Methods and Models II |
| Computing Fully Burdened Costs of Energy - Fuel (FBCE-F) | Walt Cooper, Richard Lee | The Defense Science Board, in studies conducted in the early years of the past decade, concluded that the DoD was not according sufficient emphasis to the consumption of energy in the battle space. Based on those studies, DoD leadership has elevated the importance of seeking opportunities to deliver more capable forces that consume less energy. The Joint Capabilities Integration and Development System now requires an energy Key Performance Parameter. Pilot studies have been conducted to establish a framework for the computation of the fully burdened costs of energy (FBCE) that, by statute, must now be used to inform cost, schedule and performance trade decisions in Analyses of Alternatives (AoAs). Based on those pilot efforts, the Assistant Secretary of Defense (Operational Energy Plans and Programs) has recently published a framework for computing FBCE. A detailed and thorough understanding of this framework is essential for cost analysts supporting AoAs... | 2013 | Methods and Models II |
| Estimating Alternatives for Joint Future Theater Lift (JFTL) | Robert Georgi | US military operations in Afghanistan and Iraq are making military transport aircraft work at rates not foreseen just a decade ago. At the same time the existing inventory of transport aircraft are limited in their ability to transport heavy cargo to austere unimproved landing zones where military forces routinely operate. The US Air Force (USAF) with considerable support from the US Army conducted the Joint Future Theater Lift (JFTL) Technology Study (JTS) to consider how best to equip its theater lift fleets beyond 2020 to address this need. The JTS was established by merging the US Army's Joint Heavy Lift (JHL) and the USAF's Advanced Joint Air Combat Systems (AJACS). The JTS analyzes the cost, risks, and operational effectiveness of theater lift technology alternatives that address capability shortfalls identified in the JFTL Initial Capabilities Document (ICD); an essential capability shortfall being the delivery of a combat-configured medium weight armored vehicle (up to 36 tons) into austere, short, unimproved landing areas without ground handling equipment. One of the alternatives included a hybrid airship design... | 2013 | Methods and Models II |
| Ground Radar Expenditure Phasing Analysis | Rick Garcia | Realistic budget submissions are critical to a program's success. Budget realism relies on a full understanding of the system's requirements so that a comprehensive cost and schedule estimate can be developed that can inform a realistic budget position. In addition to developing a realistic estimate, the budget position must also reflect a realistic phasing of budget dollars across an appropriate schedule; so that sufficient budget dollars are available at the correct time in a program's acquisition lifecycle to assist the program achieve success. Most existing expenditure phasing models are Rayleigh or Weibull-based distributions and were developed on a variety of DoD space and ground programs and are not commodity specific to terrestrial radar systems... | 2013 | Methods and Models II |
| Improving Cost Estimates in a Medical Acquisition Environment | Mark Russo, Kimberly Wallace | Estimating costs for drug development has proven difficult over the years. Difficulties stem from the overlapping of the Department of Defense (DoD) acquisition process and the Food and Drug Administration's (FDA's) review process, small data pool, industry close hold of cost data, the inherent difficulties in the drug development process, and the lack of using best practices in cost estimating. Over the past couple of years the medical cost community has been working towards improving their cost estimates. These efforts included the establishment of a standard drug development work breakdown structure to ensure that data could be collected and compared in a standard way. A medical cost model was then created in order to use the available data to develop program estimates... | 2013 | Methods and Models II |
| Incorporation of Program Priorities in Funding-Constrained Environments | Leon Halstead, Michael Land, Derrick Kuhn, Bryan Shrader | The recent era of declining Federal (and organizational) budgets has given new impetus to the ability to prioritize funding in an efficient and effective manner. While there are multiple methodologies to pursue these budget reductions, there is a tendency for both allocations and reductions of funding to be top-down, i.e. Organization-driven with a less defensible framework to support Program priorities. This approach heightens the risk that Program goals and objectives will be negatively impacted due to underfunding in this manner due to lessened consideration of Program-specific environmental factors... | 2013 | Methods and Models II |
| Quantifying Uncertainty in Early Lifecycle Cost Estimation (QUELCE) | Robert Stoddard, Robert Ferguson, Dennis Goldenson, Jim McCurley, Dave Zubrow | Early lifecycle cost estimation continues to grow in importance within both industry and the U.S. Government. In the commercial arena, shrinking business and product development cycles demand more rapid and early cost estimates. Within the U.S. Defense Department, cost estimates are now mandated prior to Milestone A approval within the DoD Acquisition Lifecycle. One of the primary challenges with early lifecycle cost estimation remains the incongruence of input data required by existing cost estimation models and the data available early in the lifecycle. Recent research into capability-based cost estimation offers improvements but ignores much of what may be viewed as program execution change drivers. These program execution change drivers embody the root causes of changes in program performance leading to changes in the cost of the program. As a result, our team developed the QUELCE method as an innovative approach to elicit domain expert knowledge about the possible future scenarios of change driver behavior for a given program... | 2013 | Methods and Models II |
| Sensitivity Analysis in Cost-Benefits Analysis | Steven Ikeler | The Army has been using Cost-Benefits Analysis (C-BA) for all new requirements with cost impact for two years. As with the Analysis of Alternatives (AoA), a sensitivity analysis of the C-BA results is required. The C-BAs frequently use weighted decision matrices to combine the benefits into a single Cost-Benefit Index. This greatly simplifies the sensitivity analysis and facilitates allocation of cost to the individual benefits. The author reviewed the sensitivity analysis in 80 C-BAs. This session will address some of the techniques for presenting sensitivity analysis, the basic math behind the sensitivity analysis and allocating cost to benefits and generalizing the results to more complicated problems. | 2013 | Methods and Models II |
| Probabilistic Technology Investment Ranking System | Peter Frederic | This paper describes the Probabilistic Technology Investment Ranking System (PTIRS), a multi-tool cost estimating system. PTIRS is a new cost estimating tool that has been developed for the NASA Environmentally Responsible Aviation (ERA) Project at Langley Research Center. The ERA Project is charged with developing new technologies for commercial transport aircraft that will reduce fuel consumption, emissions, and noise. The PTIRS tool assesses the integrated, end-to-end benefits of a new technology and allows the benefits to be weighed against the cost of maturing, certifying, and ultimately implementing the technology in a generation-after-next (N+2) subsonic transport system. The tool is both deterministic and probabilistic: it allows inputs with ranges of uncertainty described as statistical distributions and it uses Monte Carlo simulation to produce results with statistically described ranges of uncertainty. The tool produces a two part answer: 1) the life cycle cost impact and noise, emission, and performance benefit of implementing and operating the new technology, and 2) the cost and time required to mature and certify the technology to a readiness level appropriate for full-scale development... | 2013 | Methods and Models II |
| Rapid Generation and Optimisation of Ship Compartment Configuration based on Life Cycle Cost and Operational Effectiveness | Aidan Depetro, Rhyan Hoey | It has been well established that the majority of Life Cycle Cost (LCC) is incurred during the in-service period. Among other factors, this is strongly linked to the design of the ship and the decisions made during the early design phase. In particular, compartment configuration can have a significant effect on LCC. Poorly considered compartment configuration and hull selection can result in hydrodynamic efficiencies which significantly increase energy consumption and hence fuel costs. Associated space limitations, inadequate or non-existent removal routes and other accessibility problems may result in expensive equipment overhaul and replacement procedures, invasive removal methods, longer maintenance availabilities and increased maintenance costs. Current design methods and decision analysis techniques focus mainly on the trade-off between operational effectiveness and acquisition cost rather than LCC... | 2013 | Methods and Models II |
| Data-Driven Estimating - Quantifying Electronics Complexity Using Public Data | F. Gurney Thompson | The parametric cost estimation industry is being challenged to increase the defendability of their estimates, mainly through comparison to analogous data. However, companies are more protective than ever of their high quality cost data. While this data is often obtainable, it usually comes with strings attached (non-disclosure agreements, etc.) that preclude its usage in defending your estimate. At the same time, government and military databases with cost and technical data are being exposed to the public, and while the noise introduced by granularity issues and the mapping process of this data presents its own set of challenges, this trend also contains the key to a solution... | 2013 | Methods and Models II |
| Affordability Analysis: The Role of Process, Cost and ROI Modeling In Improved Program Performance | Dan Galorath | Affordability analysis as part of decision making may be the biggest edge of the decade for both commercial organizations and DoD / government organizations. Affordability has been addressed in the past but never to the level it is today. In an IT context companies struggle to increase profits and often view IT as a necessary evil: one that consumes resources rather than contributes to the bottom line. However, IT can be a significant contributor when IT decisions are made after modeling affordability in multiple dimensions. | 2013 | Methods and Models II |
| Applying Cost Analysis to the DoDAF | Michael Butterworth | Cost analysts in the aerospace community today are being challenged to accurately estimate new and on-going program efforts that will be integrated into the Department of Defense Architecture Framework (DoDAF). The DoDAF provides a foundational framework for developing and representing architecture descriptions that ensure a common denominator for understanding, comparing, and integrating architectures across organizational, Joint, and multinational boundaries. It establishes data element definitions, rules, and relationships and a baseline set of products for consistent development of systems, integrated, or federated architectures... | 2013 | Methods and Models II |
| The Role of Value Engineering In Affordability Analysis | Quentin Redman, Robert Koury, Joseph Bobinis, Paul Tuttle, Kevin Woodward, Hein B.A. de Jong | The purpose of this white paper is to describe the role Value Engineering plays within the affordability process. The paper is not a step by step "How To Conduct or Execute" Value Engineering (VE) but is a discussion of the context, input, setup, execution hints, and output of Value Engineering in support of conducting affordability analysis and management. As such it is important to understand the concept of affordability within the Systems Engineering paradigm. This paper is designed to provide insights, lessons learned, and suggestions for using Value Engineering in the affordability process... | 2013 | Methods and Models II |
| Adding Process Tailoring to Product Size for Better Cost Estimation | David Bloom, Chris Fudge | Generally, when one is looking to create a parametric model to predict costs of an activity, then one looks to size relationships associated with that activity. For example, if one were to develop a parametric model for estimating the development of a Field Programmable Gate Array (FPGA), then, because most of the work to complete an FPGA revolves around software code development, building a parametric model around new code, modified code and reused code (with other size relationships) would probably be a good place to start... | 2013 | Parametrics |
| Evaluating Cost Relationships with Nonparametric Statistics | Caleb Fleming | Parametric statistics are often used to analyze data, evaluate hypotheses, and determine the significance of a given set of inputs. Most commonly, cost estimators use the following parametric testing measures for significance: Pearson's product-moment correlation, z-tests, t-tests, and Analysis of Variance (ANOVA) tests. While each of these tests is subjectively appropriate to evaluate varying scenarios, each test also objectively depends on the validity of a fixed set of assumptions. Though the assumptions differ by test, all root themselves in the general idea that the data is contained within an adequately large sample and follows a particular probability distribution with estimable parameters... | 2013 | Parametrics |
| Fit, Rather Than Assume, a CER Error Distribution | Shu-Ping Hu | Analysts usually assume a distribution (e.g., normal, log-normal, or triangular) to model the errors of a cost estimating relationship (CER) for cost uncertainly analysis. However, this hypothetical assumption may not be suitable to model the underlying distribution of CER errors. A distribution fitting tool is often used to hypothesize an appropriate distribution for a given set of data. It can also be applied to fit a distribution to... | 2013 | Parametrics |
| Developing a Military Aircraft Cost Estimating Model in Korea | Sung Jin Kang, Dong Kyu Kim | A parametric cost estimating model is very useful for the weapon acquisition process. But it requires validated and normalized historic cost data to develop the Cost Estimating Relationships (CERs). Statisticians seek hundreds of data points to assure a good statistical fit between CERs and the supporting data points comprised of cost, technical, and program information... | 2013 | Parametrics |
| Improving Program Affordability Through The Application of Data Analytics | David Wang, Austin Lee | In the current budgetary environment, program affordability is a key concern. Improving program execution and minimizing schedule delays and cost growth are keys to improving program affordability. However, there is a lack of quantitative analysis of schedule risks, and a lack of understanding of the root causes for schedule delays and cost growth. Consequently, most of the common affordability improvement suggestions centered on relaxation of test and verification requirements, and streamlining of engineering processes may not address the true root causes. The effectiveness of these affordability improvement suggestions is uncertain... | 2013 | Parametrics |
| Portfolio Management: A Parameter Approach | Andy Nicholls | My paper outlines one high level method of managing a portfolio of future projects from pre-concept into acquisition phases and examines current practices and tool set requirements at the same time; examines the limitations of the current toolset and the possibilities of inclusion of probabilistic risk calculations via use of individual program risk registers... | 2013 | Parametrics |
| On General Purpose Model Credibility | Wendy Lee, Evin Stump | In the aero/defense community and elsewhere, cost estimation of proposed large scale hardware projects by prospective bidders or sponsors is often done using general purpose parametric models, i.e., models based on statistical analysis of a collection of historical data. Of interest in this paper are the parametric models that provide estimates of costs of hardware development and production for a wide variety of hardware, especially hardware that may be used in many situations and environments... | 2013 | Parametrics |
| Benefits of Integrating Schedule and Cost Risk Analysis | Rafael Hartke | Project managers have the arduous task of running their projects within budget and deadlines, all while facing a wide range of risks like prices and labor costs uncertainty, suppliers performance and reliability, technological challenges, weather, currency exchange rates, etc. Thus, assessing the likelihood and impact of risks is of utmost importance to manage not only the project itself, but also the expectations of the various stakeholders... | 2013 | Risk I |
| Modeling the Risk and Uncertainty of Inflation Rate Projections | Brian Flynn, Peter Braxton | "Experience in controversies such as these brings out the impossibility of learning anything from facts till they are examined and interpreted by reason; and teaches that the most reckless and treacherous of all theorists is he who professes to let facts and figures speak for themselves, who keeps in the background the part he has played, perhaps unconsciously, in selecting and grouping them, and in suggesting the argument post hoc ergo propter hoc." Alfred Marshall... | 2013 | Risk I |
| Probabilistic Mass Growth Uncertainties | Darren Elliott, Eric Plumer | Mass has been widely used as a variable input parameter for Cost Estimating Relationships (CER) for space systems. As these space systems progress from early concept studies and drawing boards to the launch pad, their masses tend to grow substantially hence adversely affecting a primary input to most modeling CERs. Modeling and predicting mass uncertainty, based on historical and analogous data, is therefore critical and is an integral part of modeling cost risk. This paper presents the results of an ongoing NASA effort to publish mass growth datasheet for adjusting single-point Technical Baseline Estimates (TBE) of masses of space instruments as well as spacecraft, for both earth orbiting and deep space missions at various stages of a project's lifecycle... | 2013 | Risk I |
| Deciphering JCL: How to use the JCL Scatterplot and Isocurves | Eric Druker | As the use of integrated cost and schedule risk analysis (ICSRA) methodologies rapidly expands across U.S. government programs there has been increasing confusion on how to interpret its seminal result: the Joint Confidence Level scatterplot and its associated isocurve. Over the past few years, the following statements have been made about JCL:... | 2013 | Risk I |
| Understanding the Results of an Integrated Cost/Schedule Risk Analysis | James Johnson, Darren Elliott | The recent rise of integrated risk analyses methods has created a new opportunity for complex projects to understand the dynamic inter-relationship of cost, schedule, and risk. NASA has been implementing Joint Confidence Level (JCL) analysis of cost, schedule, and risk for all major projects since 2009 in an attempt to proceduralize and codify the requirements for an integrated risk analysis product... | 2013 | Risk I |
| Top-level Schedule Distribution Models | Peter Frederic | In the initial stages of a development project, it is sometimes necessary to build a summary-level schedule for planning and budgeting purposes before the day-by-day details of the project are fully defined or understood. However, when uncertainty assessments are performed on schedule networks containing few activities, the distribution forms chosen for individual activity durations can have a significant impact on the overall results. It is therefore important to choose uncertainty distribution forms that accurately represent the behavior of the sub-network of activities represented by each summary activity... | 2013 | Risk I |
| Analysis Schedules: Danger at a Higher Level | Justin Hornback | A government organization charged with evaluating the adequacy of schedule and funding of large projects using project provided schedules and cost estimates receives a detailed (~2000 line) schedule capable of analysis with limited modification or summation. Analysis conducted using detailed schedule revealed interesting results when compared with high level schedule used in previous evaluations of the same project when a detailed analysis was not available... | 2013 | Risk I |
| Cost of Mission Assurance for Space Programs | Erik Burgess, Joe Frisbie, Michelle Jones, Chad Krause | Standards and strategies for assuring successful space missions have evolved in recent years, with some programs moving toward traditional government standards and others moving to industry's commercial standards. This has prompted questions from several government leaders about the cost of mission assurance. Although there are decades of engineering studies, test data, and on-orbit failure data to support detailed evaluation of the technical merits of various approaches, there is very little on the cost side of the equation. The National Reconnaissance Office Cost Analysis Improvement Group (NRO CAIG) has begun to address this by focusing on two of the more costly areas of mission assurance-environmental testing and high-reliability electronic parts... | 2013 | Risk I |
| Joint Cost Schedule Risk and Uncertainty Handbook | Duncan Thomas, John Fitch, Alf Smith, Jeff McDowell | The Naval Center for Cost Analysis (NCCA) has created a Cost Schedule Risk and Uncertainty Handbook (CSRUH) that clearly presents simple, well-defined cost risk and uncertainty analysis processes that are repeatable, defendable and easily understood. The CSRUH is based upon the Air Force Cost Analysis Agency (AFCAA) Cost Risk Uncertainty Handbook published in 2007. It updates key processes to address cost risk and uncertainty methods with an emphasis on providing guidance to capture the impact of schedule uncertainty and the risk register while avoiding "double counting"... | 2013 | Risk II |
| A Step-Wise Approach to Elicit Triangular Distributions | Marc Greenberg | As the federal government acquires less mature, more advanced and more complex systems, there is an ever-increasing burden on the cost analyst to employ methods of eliciting requirements, schedule and cost uncertainties from one of more subject matter experts (SMEs). Arguably, the most common technique a cost analyst uses today to elicit such data is to ask each SME for the lowest, most likely and highest value which, consequently, produces a triangular distribution... | 2013 | Risk II |
| Robust Default Correlation for Cost Risk Analysis | Christian Smart | Correlation is an important consideration in cost risk analysis. Exclusion of correlation from cost risk analysis results in the de facto assumption that all risks are independent. The assumption of independence leads to significant underestimation of total risk. However, figuring out the correct correlation values between work breakdown structures elements can be challenging. For instance, it is difficult to estimate the exact correlation value between the structures and thermal protection subsystems in a cost risk estimate... | 2013 | Risk II |
| Use of the Risk Driver Method in Monte Carlo Simulation of a Project Schedule | David Hulett | Identifying the root causes of project schedule and cost risk requires that the risk to the project schedule is clearly and directly driven by identified and quantified risks. In the Risk Driver Method the risks from the Risk Register drive the simulation. (As a side note, we find that the Risk Registers are not complete-during the interviews to collect risk data the interviewees introduce important risks that are, surprisingly, missing from the Risk Register.) The Risk Driver Method differs from older, more traditional approaches in which 3-point (low, most likely and high) estimates of the activity durations are applied directly to activity durations... | 2013 | Risk II |
| Base Realignment and Closure (BRAC) Savings and Acquisition Risk | Peter Braxton, Kevin Cincotta, Richard Lee | The Government Accountability Office (GAO) recently released the study Military Base Realignments and Closures: Updated Costs and Savings Estimates from BRAC 2005 (GAO-12-709R, June 29, 2012). Its appendices contain a wealth of risk data, with initial estimates (2005 BRAC commission) and final costs (Fiscal year 2011 DOD budget) for 175 distinct BRAC initiatives. Applying an innovative method for modeling within-program risk and uncertainty using cross-program data, this paper derives cost growth factors (CGFs) and coefficients of variation (CVs) for BRAC initiatives. Furthermore, the pattern in these data is astoundingly similar to that found in major defense acquisition program (MDAP) data, a strong confirmation of this modeling approach (meta analysis)... | 2013 | Risk II |
| Cost Contingency Analysis using Polytopes | Bohdan Kaluzny | Contingency is defined as a possibility that must be prepared for-an event that may occur but is not likely or intended. In the realm of cost estimation, contingency refers to a reserve above and beyond a baseline estimate that would be tapped if one or more unexpected events led to higher program costs. Ideally, the selection of a contingency amount should be based on achieving a particular level of confidence derived from cost uncertainty and risk analyses, but often a mere rule-of-thumb percentage (e.g. 15% of the baseline estimate) is applied... | 2013 | Risk II |
| Correlation Matrices Revisited | Steven Ikeler | In a cost risk analysis, risk distributions are sometimes assigned at a low level and then combined, most commonly using Monte Carlo simulation. When using this technique, it is important to use a realistic correlation matrix. It is usually impractical to know all the pairwise correlations. This session will address some of the techniques for completing a credible correlation matrix at differing levels of input information and certainty. Additionally, the session will show examples and review the mathematics behind legitimate correlation matrices. Past sessions have discussed these topics and typically deal with correction techniques for expert-based correlation matrices. This session will be different since it deals less with correcting expert based correlation matrices and more with dealing with uncertainty in the correlations or incomplete correlation matrices. It will also address an extension of the problem to a parametric analysis problem. | 2013 | Risk II |
| Optimization of Cost and Performance in Complex Systems Using System Dynamics Modeling | Michael Polly, Benjamin Harris | Analysis of trade spaces between cost and performance is critical to win new contract pursuits in today's highly competitive environment. The overall goal in a program is to design and produce a system that maximizes both performance and affordability. In attempting to balance cost and performance, traditional cost models only allow for experimentation with one variable at a time. This can be very cumbersome on large systems, and inhibits how much of the trade space can be explored under time constraints. Altering one variable at a time also limits the number of cost estimating relationships that can be incorporated into the analysis... | 2012 | Earned Value Management Track |
| Joint Analysis of Cost and Schedule (JACS) - A New Tool for JCL Analysis | Antonio Rippe, Rey Carpio | NASA has recently implemented a policy that requires a program/project to be approved at a 70% Joint Budget and Schedule Confidence Level (JCL) which are to be generated from models that integrate cost and schedule. To comply with the policy, a tool framework is needed that can conduct a risk and uncertainty analysis on schedule logic as well as integrate cost uncertainty analysis to account for phasing and costs that are dependent on overall duration. NASA is currently using several platforms, but has seen some issues in extensibility, cost integration, and overall performance... | 2012 | Earned Value Management Track |
| "The Answer is 5": Observations on Cost & Schedule in Small Defense Programs | Brian Fersch, Wesley Tate, Colleen Leonard | Cost & schedule overruns have been an enduring issue in government acquisitions. Our largest programs are typically dissected and their flaws illuminated with 20/20 hindsight. DoDs smaller programs, though comprising the majority of the budget and quantity of programs, do not get this same attention. Over the course of the past year this study team has collected historical & current cost and schedule data on numerous programs that has been collected at similar points in each programs acquisition lifecycle. The end result is a database that allowed the team to examine how cost estimates and schedule expectations changed from year to year for the same events... | 2012 | Earned Value Management Track |
| WBS Development: Rules, Aberrations and MIL-STD-881C Implementation Challenges | Michael Metcalf | Issued on October 3, 2011, Military Standard 881C (MIL-STD-881C) provides the framework for developing Work Breakdown Structures (WBSs) for use throughout the acquisition process, as well as a standard set of structures that fit eleven commodity types. These WBSs are used for management, system engineering, and cost estimation by a variety of stakeholders. Though comprehensive, the MIL-STD does not address some of the complex details encountered when developing a weapon system WBS, and there are many special cases and exceptions that could not be captured in the document... | 2012 | Earned Value Management Track |
| Integrated Project Management (IPM) - Transforming Data into Information | Andrea Mozzo, Bruce Koontz | Earned Value Management (EVM) integrates the cost, technical and schedule performance. EVM is not just tracking Schedule Performance Index (SPI) and Cost Performance Index (CPI), but has been transformed into Integrated Project Management (IPM). IPM involves three key areas: Managing cost, schedule, and technical performance within constraints; initiating effective cost, schedule, and performance tradeoffs when constraints are not achievable; and continually evaluating progress to predict and mitigate problems... | 2012 | Earned Value Management Track |
| Critical Chain Applied to an MRO - Surface Repair Facility | Howard Rainer | The project management methodology of Theory of Constraints, called Critical Chain, is examined for application to Scheduling, Planning and Control (SP&C) for a defense-related Maintenance Repair Operation (MRO). The opportunities for Critical Chain, as an extension of Theory of Constraints (TOC), have been reported for well over a decade. With this proven record comes a proposal for similar application in the SP&C at the Surface Repair Facility (SRF) at Cecil Field, Florida. The SRF has established reporting systems such as Earned Value Methodology (EVM) - mentioned here to distinguish the differences between execution and reporting in SP&C. | 2012 | Earned Value Management Track |
| Successful Implementation of an Over Target Baseline/Schedule from a Government Perspective | Joseph C. Annunziato, Daniel W. Donaldson | In recent years, Over Target Baseline/Schedule implementations have been accomplished with varying degrees of success. Using the Over Target Baseline (OTB) and Over Target Schedule (OTS) Handbook as a guide, this presentation will address the proper way to implement an Over Target Baseline/Schedule from a Government perspective. The purpose is to provide both government and industry professionals a general understanding of the processes and decisions that must be considered when implementing an OTB and to assist the earned value management (EVM) community in the understanding and implementation of this important management tool. It will also incorporate examples and lessons learned from recent OTB/OTS implementations... | 2012 | Earned Value Management Track |
| Accepted Standards and Emerging Trends in Over Target Baseline (OTB) Contracts | Simon Dekker | Over Target Baseline (OTB) projects or programs are those that have run significantly over cost and require formal reprogramming - essentially a complete replanning of the project - in order to help the contractor regain management control over the effort. The OTB process has been well documented and become an established part of Earned Value Management practice. Much of the literature-to-date has focused on OTBs from the contractor perspective, including the steps to take in order to propose and implement an OTB, and the proper channels and occasions for engaging the customer in the process... | 2012 | Earned Value Management Track |
| EVM for the Rest of Us | Javier Sloninsky | Earned Value Management (EVM) is the practice of evaluating cost and schedule data to measure productivity and progress throughout the lifecyle of a project. EVM, however, often has the stigma of being overly complex and burdensome to implement in a useful way. This session will discuss best practices in implementing a simple yet robust earned value management solution for organizations that benefit from progress measurement and the ability to forecast cost performance, but don't need to comply with ANSI 748, the EVM standard often used by government agencies and contractors. We'll discuss topics including the integration of required data, developing key performance indicators (KPI) and metrics, application of rules of credit, updating of percent complete, and automated EV reporting. | 2012 | Earned Value Management Track |
| A New EVM Performance Index: The MRPI | Michael Nosbisch | It has long been understood that the accuracy of the cost performance index (CPI) is closely tied to how management reserve (MR) has been historically utilized on a specific project/program. However, a formal assessment of MR usage is still not consistently performed in conjunction with CPI calculation and subsequent analysis. This presentation will offer a potential solution to this issue, by proposing a "Management Reserve Performance Index," or "MRPI." Once calculated, the author will demonstrate how the MRPI can be then used to develop a "risk-adjusted IEAC" that provides a more accurate assessment of a project's or program's overall cost performance. | 2012 | Earned Value Management Track |
| From Bid Package to Detailed EVM Baseline in One Easy Step | Karen Stiff | Your Company has put together a cost estimate and basic schedule along with the technical elements as a bid package for a Request for Proposal for a Government contract requiring Earned Value Management (EVM). The proposals have been evaluated, discussions have been held and now it's time for the award. Congratulations your company's won the contract! Now the manager assigned to the program has the task of setting up an EVM baseline for the program. Why is it so hard to do a detailed schedule and EVM baseline, when you've got the schedule and cost that the company bid? Why doesn't this baseline look in any way similar to the bid package cost and schedule? Should it look similar?... | 2012 | Earned Value Management Track |
| Touchpoints between the SCEA Cost Estimating Body of Knowledge (CEBoK®) and the Program Management Institute (PMI) Body of Knowledge (PMBoK) | Leon Halstead | The General Accounting Office (GAO), in its publication of the "GAO Cost Estimating and Assessment Guide" in May of 2009, focuses extensively on the importance of Cost Estimation and its contribution to overall project success. While this publication offers specific technical examples of cost estimation and analysis capability, e.g. pre-Program approval cost estimates/baselines, Affordability Analysis, et al, there is less focus on how these technical examples "fit" into the larger Project Management context... | 2012 | Informational Sessions |
| A Structured Approach to CCEA® Certification | Michael Mahoney, Rick Beavers (Presenter) | The structured approach to SCEA certification details the methods used by Lockheed Martin Chelmsford to prepare employees for the CCEA examination. In 2010, three of us set out to earn certification at the CCEA level and all three of us passed on the first attempt. The session will introduce a generic schedule for studying the CEBoK modules including the time allotted for each, an approach to taking sample tests, and lessons learned. We'll discuss what worked and what didn't and share tips and tricks. | 2012 | Informational Sessions |
| The EVP Certification: A Bridge Between SCEA and AACE | Michael Nosbisch | In the 1990s, SCEA and AACE International, formerly known as the Association for the Advancement of Cost Engineering, not only were both members of the now defunct Joint Cost Management Association, but they also maintained a cooperative agreement that intended for members of the two associations to "benefit from increased knowledge of the activities and services available from the other organization." While that document agreement unfortunately did not extend beyond 1995, there has recently been a joint effort to renew a similar type of agreement... | 2012 | Informational Sessions |
| The Fractal Nature of Cost Risk: The Portfolio Effect, Power Laws, and Risk and Uncertainty Properties of Lognormal Distributions | Christian B. Smart | Cost risk can be added to the list of the many phenomena in nature that follow a power-law probability distribution. Both the normal and lognormal, neither of which is a power-law distribution, underestimate the probability of extreme cost growth, as shown by comparison with empirical data. This situation puts the widely debated portfolio effect into further dispute. | 2012 | Journal of Cost Analysis and Parametrics |
| Prediction Bounds for General-Error-Regression Cost-Estimating Relationships | Dr. Stephen A. Book | Estimating the cost of a system under development is essentially trying to predict the future, which means that any such estimate contains uncertainty. When estimating using a cost-estimating relationship (CER), a portion of this uncertainty arises from the possibility that the cost-estimating form to which regression analysis is applied may be the incorrect one. | 2012 | Journal of Cost Analysis and Parametrics |
| Comparison of Cumulative Average to Unit Learning Curves: A Monte Carlo Approach | Trevor P. Miller, Austin W. Dowling, David J. Youd, Eric Unger, Edward D. White | Cumulative average and unit cost learning curve methodologies dominate current learning curve theory. Both models mathematically estimate the structure of costs over time and under particular conditions. While cost estimators and industries have shown preferences for particular models, this article evaluates model performance under varying program characteristics. | 2012 | Journal of Cost Analysis and Parametrics |
| Here, There Be Dragons: Considering the Right Tail in Risk Management | Christian B. Smart | The portfolio effect is a common designation of a supposed reduction of cost risk achieved by funding multiple projects (the portfolio) that are not perfectly correlated with one another. It is often relied upon in setting confidence-level policy for program or organization budgets that are intended to fund multiple projects. The idea of a portfolio effect has its roots in modern finance, as pioneered by 1990 Nobel Memorial Prize in Economic Sciences recipient Harry Markowitz (1959). | 2012 | Journal of Cost Analysis and Parametrics |
| A Case Study on Target Cost Estimation Using Back-Propagation and Genetic Algorithm Trained Neural Networks | Adil Salam, Fantahun M. Defersha, Nadia F. Bhuiyan, M. Chen | Cost estimation of new products has always been difficult as only few attributes will be known. In these situations, parametric methods are commonly used using a priori determined cost function where parameters are evaluated from historical data. Neural networks, in contrast, are nonparametric, i.e., they attempt to fit curves without being provided a predetermined function. | 2012 | Journal of Cost Analysis and Parametrics |
| Enhanced Scenario-Based Method for Cost Risk Analysis: Theory, Application, and Implementation | Paul R. Garvey, Brian J. Flynn, Peter J. Braxton, Richard C. Lee | In 2006, the scenario-based method was introduced as an alternative to advanced statistical methods for generating measures of cost risk. Since then, enhancements to the scenario-based method have been made. These include integrating historical cost performance data into the scenario-based methods algorithms and providing a context for applying the scenario-based method from the perspective of the 2009 Weapon Systems Acquisition Reform Act. | 2012 | Journal of Cost Analysis and Parametrics |
| Preliminary Study of Palm-Oil Biodiesel Life Cycle Cost Analysis in Indonesia | Sidhartha Sahirman, Dr. Saparso, Agus Sarjito, Mokhtar Awang, Lr. Shaharin A. Sulaiman | Recently, Biodiesel have been gaining popularity and getting accepted as an alternative fuel for diesel engines. In the last few years, Indonesian government has a special interest in the development of this renewable energy, the use of which can help reduce oil imports. Biodiesel has a bright future in this country, as Indonesia is the world's largest producer of crude palm oil (CPO) - a desirable feedstock for biodiesel production. Biodiesel has the potential to become a significant industry sector in Indonesia... | 2012 | Life Cycle Cost |
| Life Cycle Costs in CBA | Niatika Griffin, Ronako Carson | Cost Benefit Analysis (CBAs) are of growing importance to the acquisition process. The Vice Chief of Staff for the Army signed a memo dated December 20, 2009 stating that, in this era of constrained resources, Army leaders need to be responsible stewards of government funds. Any new requirement that is unfunded and/or newly expanded program proposals will be accompanied by a thorough CBA. The CBA must identify the total cost proposal to the Army in the form of life cycle cost, benefits that will result, and the bill-payers that would be used to pay for the unfunded or new requirement... | 2012 | Life Cycle Cost |
| Designing a Conceptual Framework for Estimation and Analysis of Total Ownership Cost | F. Gurney Thompson III, Robert Koury | In recent years, the push for greater efficiency and productivity in Defense spending has yielded an increased focus on affordability analysis. Understanding and estimating Total Ownership Costs (TOC) is key in assessing affordability, and the cost community must adapt to support TOC estimation. This paper discusses the development of a conceptual framework for estimating TOC in support of a broader audience, from the acquisition community to program managers and even as a decision support tool for entities such as Congress, DoD Financial / budgetary community, and G-8 Program Analysis & Evaluation... | 2012 | Life Cycle Cost |
| DCARC Contractor Sustainment Cost Data Collection | Sandra B. Enser | In the past, Contractor Sustainment (including Contractor Logistics Support (CLS), Contractor Performance Based Logistics (PBL), and other contractor sustainment efforts) were omitted from DoD centralized O&S databases, or were captured as an annual total with little or no supporting detail. For many programs, Contractor Sustainment costs are a major cost driver. They represent an increasing percentage of operating costs. As DoD faces major budget reductions, valid and timely Contractor Sustainment cost data supports analysis which enables Services to use declining operations funds effectively... | 2012 | Life Cycle Cost |
| Tactical Wheeled Vehicle (TWV) Fuel Economy Improvement Breakeven Analysis | Raymond Kleinberg | With the rising costs of and attention towards fuel usage in the Department of Defense (DoD), efforts are being made to procure more fuel efficient vehicles and modernize the Army's existing Tactical Wheeled Vehicle (TWV) fleet. This research is part of an ongoing effort to help program managers make decisions regarding how much investment can be made into new technologies in order to see benefits within the estimated useful life of individual systems and the fleet as a whole. | 2012 | Life Cycle Cost |
| Discrepancy Report Prioritization and Software Maintenance Impacts | Jennifer Woolley, Kyle Thomas | In the current budget-constrained environment, programs and organizations throughout the Intelligence Community (IC) are searching for new strategies to reduce costs. Rather than sacrificing new capabilities, maintenance is often one of the first areas of reduction. Many organizations perform developed software maintenance based on the submission of Discrepancy Reports (DRs), which are ranked according to their level of severity/importance. One approach to cutting maintenance costs involves only focusing on higher levels of DRs while leaving lower level DRs postponed or unaddressed. This strategy allows for a reduction in personnel because fewer DRs must be completed... | 2012 | Life Cycle Cost |
| Software Sustainment: Pay Now or Pay Later | Arlene Minkiewicz | With today's budget constraints the Department of Defense (DoD) is likely to build less new systems. In light of this, the systems that we have today will need to be stretched further. This means more time and effort will be going into maintenance. And since many systems fielded today rely on software to deliver much of their mission critical capability; this means more time and money will be devoted to software sustainment... | 2012 | Life Cycle Cost |
| How VAMOSC VIEWS Can Help You! | Brian Welsh, Walt Cooper, Paul Hardin, Elizabeth Koza, Robert Nehring | Several recent developments underscore the importance of identifying and analyzing operating and support (O&S) costs for weapon systems. This brief will present the VAMOSC VIEWS, innovative graphical displays of O&S cost data that serve as an invaluable tool for the cost analyst. In response to current economic conditions, we are entering an environment of reduced budgets within the Department of Defense (DoD). Legislative initiatives, starting with the Weapon Systems Acquisition Reform Act (WSARA) of 2009, are aimed squarely at increasing the visibility and use of O&S costs in weapon systems decision-making... | 2012 | Life Cycle Cost |
| Estimating Software Maintenance Costs for U.S. Army Weapons Systems | Cheryl Jones, James Judy, Brad Clark | The emerging defense economic environment is characterized by scarce resources, extended system life cycles, and evolving mission capability requirements. As a result, the Army's investment in software maintenance, sustaining engineering, and operational support efforts is coming under increased scrutiny by decision makers at all levels. Of particular interest is the validity and defensibility of the estimates used to determine, allocate, and evaluate the value of software maintenance funds... | 2012 | Life Cycle Cost |
| Software Maintenance Data Collection and Estimating Challenges | Vanessa Welker, Peter Braxton, Wilson Rosa, Corinne Wallshein, Joe Dean | Entering a period of fiscal austerity, it becomes more important than ever to estimate and consider operating and support (O&S) costs, which represent the lion's share of life cycle cost (LCC) for most platforms, during acquisition. Given the ubiquity of software in today's complex programs, a key component of O&S is software maintenance. This paper presents the results of a research study co-sponsored by the Air Force Cost Analysis Agency (AFCAA) and the Naval Center for Cost Analysis (NCCA) to collect software maintenance data from government support activities and development contractors to enable high-fidelity cost estimates for software maintenance... | 2012 | Life Cycle Cost |
| Job Satisfaction: The Link to Retention and the Correlation to Age, Gender and Organizational Position | Darrin L. DeReus | In an attempt to balance resource availability and workload, leaders in the United States Air Force have attempted to manage the budgets by fluctuating manpower levels. There is minimal research of multiple affiliations (active duty military, government civilians and contractors) and the effects of manpower reductions on the organization. This study collected data on career anchors and satisfaction levels to find the similarities and differences of multiple affiliations in the United States Air Force. The results of this study showed that there were correlations between career anchors and satisfaction scores... | 2012 | Management |
| Using Treasury Securities to Develop Inflation Indices | C. Tyler Cunningham, Joseph Parisi | This paper examines inflation assumptions in Department of Defense cost estimates. Professional guidance allows and encourages the development of custom inflation indices, but there are few resources detailing the methodologies to develop those indices. We address that need by laying out detailed steps to develop defensible, custom, risk-adjusted inflation indices using techniques that leverage analysis of the spread and volatility between traditional US Treasury Securities and Treasury Inflation-Protected Securities. | 2012 | Management |
| A Holistic Approach to Multiyear Procurements | Ann Hawpe, Travis Winstead | Multi-year contracts have the potential to save the Government a substantial amount of money if they are executed on stable programs in an environment with stable requirements and funding. With the current economic pressure for defense programs to do more with less, multi-year contracts are a good option for programs that meet these criteria. The largest amount of savings is the result of procuring a larger quantity of parts than would be procured with a year-over-year contract structure, thus realizing economies of scale. Savings are also captured when the labor hours to develop and submit year-over-year proposals are factored into the calculations... | 2012 | Management |
| Social Media Applications | Marc Wear, Michelle Ehlinger, David Pearson, Sara Wise | This paper explores the impact of adopting social media tools in the work environment as a means to improve communication, collaboration, and productivity for cost and earned value analysts. Public and private entities have taken steps to adopt social media platforms to benefit their organization; this paper demonstrates how cost analysts can implement these tools to improve methods and processes... | 2012 | Management |
| Building a DHS Cost Estimating & Analysis Center of Excellence | Katie Geier, Colleen Craig, David Brown, Kevin Cincotta | The Department of Homeland Security (DHS) Cost Estimating & Analysis (CE&A) Center of Excellence (COE) serves as the Department's primary advocate and resource for cost estimating and analysis. Formed in October 2011, the CE&A COE's primary goal is to take a proactive, streamlined approach to improving the Department's resources and skills in cost estimating and analysis. This presentation highlights the roles and responsibilities of CE&A COE and summarizes the work accomplished to date and planned future efforts for accomplishing the following objectives:... | 2012 | Management |
| Cloud Computing: Federal Mandates and the DoD | Heather Nayhouse, Eric Lumsden | The Federal CIO has laid out a plan to implement cloud computing in all federal agencies and consolidate 800 data centers by 2015. The Office of Management and Budget has required agencies to default to cloud computing solutions wherever those solutions are secure, reliable, and cost-effective. In response, the Department of Defense has launched a number of pilot programs as well as its own strategic plan to foster their deployment of cloud computing systems. Several case studies of cloud solutions within the Department of Defense are already available. The DoD is analyzing these cases for guidance into a larger push into cloud computing systems... | 2012 | Management |
| A Case Study in Broad System Analysis: DoD Spectrum Reallocation Feasibility Study, 1755-1850 MHz | Brian Wilkerson, Guenever Aldrich | With the increasing demand placed on the electromagnetic spectrum by the proliferation of web enabled cell phones, the President has begun an initiative to sell 500 MHz of dedicated bandwidth from the federal government to the wireless industry. The 1755-1850 MHz band was selected as the first band under this initiative for a study of the cost and technical feasibility of relocating DoD assets that use these frequencies. The study was incredibly broad as all services were involved, and the systems affected include Precision Guided Munitions, Point to Point Microwave, High Resolution Video, Multiple Radio Systems, Electronic Warfare, Air Combat Training, Telemetry, Satellite, and Small Unmanned Aerial Systems... | 2012 | Management |
| Using the Tools of Persuasion to "Sell" Your Estimate | Jennifer Kirchhoffer | What is the difference between a good cost estimator and a great cost estimator? Both have robust estimating skills and knowledge, but only great cost estimators are able to garner acceptance and usage of their estimates from even their harshest critics. The ability to "sell" the estimate is key to the utilization of the estimate in the proper context by the decision makers. Our jobs often entail delivering news that our customers and leaders are not anxious to hear. Many factors work against us as we try to garner support and acceptance of the estimates... | 2012 | Management |
| Capabilities Based Portfolio Analysis (CBPA) | Paul Gvoth, Brad Ellis | Total cost is no longer the singular driver of defense acquisition program economic decision making. Defense programs and projects must be evaluated according to their contributions to overall monetary and non-monetary Return on Investment (ROI) relative to the capabilities of a Family of Systems (FoS), System of Systems (SoS) or as a part of a portfolio of products that together meet documented operational requirements. This is needed because future defense strategies, including Smart Defence are being driven by budget cuts... | 2012 | Management |
| NRO Program Assessments - Best Practices and Lessons Learned | Greg Lochbaum, Linda Williams, Lisa Keller, Ken Odom | The National Reconnaissance Office (NRO) performs program assessments for a variety of purposes: the management of acquisition programs; portfolio analysis for resource allocation; and accountability to NRO's oversight organizations - Execution to Oversight. The NRO Cost Analysis Improvement Group (CAIG) recently stood up a Program Assessment group to continuously improve the NRO's program assessment process to become more efficient, consistent and standardized. This presentation provides the NRO's approach to each level of the Execution to Oversight pyramid... | 2012 | Management |
| How does a $20B Program Become a $30B Program? Lessons Learned in Production Cost Management | Steve Sheamer, Allen Gaudelli | How does a $100M ship become a $200M ship or a $75M fighter jet turn into a $125M fighter jet? As we have seen recently with several high profile acquisition programs, systems integrators and cost estimators continue to struggle when estimating the production costs of development systems. This presentation focuses on production-related cost drivers and why contract incentive-fee structures are often part of the problem, then details steps for better management of these production costs... | 2012 | Management |
| Case Study of Army Cost Management: Managing Variability in Analytical Cost Products for an ACAT ID Program | Tomeka Williams | Army Cost Management requires an understanding of the level of commitment the Department of the Army has made to cost management that goes beyond the Program Management Office (PMO) through, TRADOC, and HQDA levels. By requiring any program that is a new start, or major upgrade of an existing program to submit a Cost-Benefit Analysis based on guidance from the VCSA, is one of the unique cost management products the Army has instituted. This is now the first step to securing a viable and executable acquisition program within the Department of the Army... | 2012 | Management |
| Trying To Do Too Much with Too Little: How Poor Portfolio Management Can Lead to Schedule Delays and Cost Overruns | Christian Smart | Government organizations often try to accomplish too much with the resources with which they are provided. The genesis of new projects in government organizations often begins with the search to spend idle funds. An organization's leadership may discover that it has a relatively small amount of money that can be used to research a new technology or to begin initial development of a new project. New ideas to improve systems, develop better, newer ones, or to advance scientific understanding abound. In order to sell leadership, project management provides leadership with success-oriented, optimistic projections... | 2012 | Management |
| IT Service-Based Costing: Standardizing Provisioning and Servicing IT Resources | Emily Jessen, Paul Brown | In today's current economic environment, there is a strong focus on constraining spending of appropriated funds to meet the mission particularly within the Intelligence Community and Department of Defense (DoD). Part of the DIA's mission includes the delivery of IT products and services. In the past, IT services have been provided to non-Core customers for free, but in the new fiscal environment, the DIA does not have sufficient resources to continue to operate in this manner. In compliance with the Economy Act of 1932 and DoD Financial Management Regulations, the DIA has been taking steps to quantify the IT services it deploys in order to recover costs of those services... | 2012 | Management |
| Modeling R&D Budget Profiles | Erik Burgess | Prior research sponsored by the National Reconnaissance Office (NRO) and published in the Journal of Parametrics (Summer 2006) has become the principal method for developing space and ground segment budget profiles for NRO programs. Since then, the practice of budgeting to independent cost estimates has been signed into law for the intelligence community. The resulting scrutiny and reliance on budget-phasing models has motivated several improvements that should be relevant to the estimating process for other DoD commodities. The presentation first reviews the mathematical formulation of expenditure profiles that are based on historical data, and how some of the resulting accuracy metrics have been useful in defending annual budgets. New models for space and ground systems are presented. Since the annual budget authority required to support an expenditure stream is heavily dependent on a program's execution rates (i.e., outlay rates), we also present a new approach for estimating what those outlay rates will be. This is a new area of responsibility for most cost estimating organizations, yet its impact on annual budget requests can be significant. | 2012 | Management |
| Evaluating the Life Cycle Cost and Effort of Project Management for Complex Systems Development Projects | Leone Young | Across industries, systems development effort has become more costly and complex in nature over the decades, and the costs of project management that are associated with systems development effort have shown an upward growth as well. The estimation of systems development cost and effort has been studied by many scholars and researchers in several different areas such as hardware, software, integration and systems engineering, but the area of project management cost and effort has had very limited exploration, and the literature is void of project management costing methodologies as of today... | 2012 | Management |
| Defense Cost & Resource Center (DCARC) Executive Overview Brief | Mike Augustus | The OSD stood up an independent office to manage the CCDR (Contractor Cost Data Report) requirement in the late 1990s. Today, that office is known as the Defense Cost and Resource Center (DCARC), which is part of the Office of the Secretary of Defense (OSD) Cost Assessment and Program Evaluation (CAPE). The original charter has been expanded to accommodate the collection of not only cost data but software metrics data as well, from industry on ACAT 1 Major Defense Acquisition Programs and Major Automated Information Systems (MAIS) programs. Today the Cost and Software Data Report (CSDR) requirement spans over 140 MDAP ACAT 1C/1D programs and a large number of MAIS 1AM and 1AC Programs... | 2012 | Methods and Models I |
| One NASA Cost Engineering (ONCE) | Eric Plumer, Mike Blandford | This presentation is the first roll out of NASA's ONCE (One NASA Cost Engineering) database to the SCEA/ISPA community. Concept and early prototyping of the One NASA Cost Engineering (ONCE) database began several years ago and it has been developed to fully automate the CADRe data in a GUI database allowing for easy search and retrieval of data for the cost community. By automating the retrieval of CADRe data this tool is greatly improving the space cost community's ability quickly acquire the data needed to improve the quality and credibility of these cost estimates... | 2012 | Methods and Models I |
| Galaxy Charts: The 1,000-Light-Year View of the Data | Robert Nehring, Katharine Mann, Robert Jones | The old adage "A picture is worth a thousand words" is one that we have all heard and often try to use to our advantage. This paper applies the principles of visual display of information as advocated by Edward Tufte and others to develop an innovative graphic that will prove invaluable to cost estimators and consumers of their estimates everywhere... | 2012 | Methods and Models I |
| Building an Agile, Collaborative Environment for Capturing Productivity Based Cost Model Data | David Bloom, Wanda Grant, Mason Wexler | Understanding and analyzing productivity within an Aerospace electronics development organization requires continuous oversight and adjustment. For example, in digital electronics, a corollary to Moore's Law says that the capability of digital processor components doubles every 18 months. This would imply that the cost to design a set of digital electronics products to deliver a specific state-of-the-art capability would change (become less) as a function of the ability to incorporate more features on any one component... | 2012 | Methods and Models I |
| United States Marine Corps Logistics Requirements Funding Summary (LRFS) Cost Estimating Tool (CET) - A Quick Cost Estimator for Logisticians Part II | Charles Gu | Similar to a Life Cycle Cost Estimate (LCCE), the Logistics Requirements Funding Summary (LRFS) captures Integrated Logistics Support (ILS) related costs for a program throughout its life cycle. The LRFS provides visibility into the logistics requirements of a program which is needed during Program Objectives Memorandum (POM) and budget submissions. Additionally, it helps to plan and quantify requirements, identify and defend funding, and serves as the ILS input to the LCCE... | 2012 | Methods and Models I |
| Modeling Potential Cost Savings by Synchronizing Commercial Derivative Acquisition & Lifecycle Programs | Brad Boehmke | Synergy is defined as "the combined or cooperative action of two or more stimuli for an enhanced effect" (Razzetti, 2009). As Department of Defense (DoD) system acquisition costs continue to grow and commercial derivative programs becoming more accepted, it is critical that the Air Force look for ways to create cost savings in every possible way. With commercial derivative programs becoming more popular in defense acquisition, there may be acquisition programs running concurrently that have system commonalities and could benefit from synchronizing acquisition efforts. By leveraging commonalities in requirements, the Air Force and the Department of Defense may be able to "obtain cost savings through acquisition and logistics planning" (Poulin, 2010)... | 2012 | Methods and Models I |
| Zero Based Review Methodology | Martha Wells, Peter Meszaros | The days of incrementally funding activities based on past performance are over at DIA. Increased budget pressure throughout the federal government has created an environment for improvements to the way activities are funded. DIA is approaching this increasingly challenging budget environment with a new programming methodology - Zero Based Reviews. The Zero Based Review (ZBR) process incorporates programmatic analysis and activity-based costing to approach budgeting from a right sizing methodology. Rather than reviewing the enterprise based on current structure, the ZBR takes a topical approach to program review to determine if the right resources are being used in the right way for the right mission... | 2012 | Methods and Models I |
| Cost Benefit Analysis Overview | Darrell Hamilton | There are many different requirements to do some type of cost and benefit tradeoff analysis as part of the acquisition process, IT investment decisions, performance based logistics decisions and budget change proposals. Most of them have nuance definition differences and are known by their acronyms as CBA, EA, BCA, AoA, COA and several other terms. Every one of those analysis requirements is aimed at demonstrating or helping others to decide on what is the most reasonable and cost effective decision that needs to be made... | 2012 | Methods and Models I |
| Accuracy Matters: Selecting a Lot-Based Cost Improvement Curve | Shu-Ping Hu, Alfred Smith | There are two commonly used cost improvement curve (CIC) theories: unit cost (UC) theory and cumulative average cost (CAC) theory. Ideally, analysts develop the CIC by analyzing unit cost data. However, it is common that instead of unit costs, analysts must develop the CIC from lot cost data. An essential step in this process is to estimate the theoretical lot midpoints (LMP) for each lot, to proceed with the curve-fitting process. LMP is generally associated with UC theory, where the midpoint is always within the lot. The more general lot plot point (LPP) term is used in the context of both the UC and CAC theories... | 2012 | Methods and Models I |
| Developing Standardized Cost Element Structures for the United States Marine Corps | Jeremy Eden | This presentation explains the development of a Cost Element Structure (CES) for use in United States Marine Corps (USMC) Logistics Requirements Funding Summaries (LRFS). Industry and the Assistant Commander for Life Cycle Logistics formed an Integrated Product Team (IPT) to create a new CES. The IPT comprised logisticians and cost estimators who worked together over the course of one year to create the new USMC LRFS CES... | 2012 | Methods and Models I |
| Enhancing Excel-Based Cost Models with PivotTable Reporting | Blaze Smallwood, Omar Mahmoud | A major challenge facing DoD cost estimators is developing Program Life Cycle Cost Estimates (PLCCEs) that serve as more than just an acquisition reporting "check-in-the-box" document. The ideal is to simultaneously create a dynamic, flexible tool that the program office can use to support their day-to-day costing needs, from answering a wide-range of cost-related data calls to integrating PLCCE outputs into their Program, Planning, Budgeting, and Execution (PPBE) processes. For DoD cost estimators that utilize Microsoft (MS) Excel to create their PLCCE models, a valuable tool that can be leveraged to achieve these goals is Excel's PivotTable reporting capability... | 2012 | Methods and Models I |
| An Approach to Improving Cost Estimating and Budget Integration in Federal Programs | Michael Noonan, Kristin Jackman, Eric Hong | Over the past few years, political standoffs in Washington, D.C. have delayed or prevented passage of Federal Appropriations Bills and significantly reduced Federal spending. As Agency budgets continue to shrink, programs must increasingly compete with each other for a slice of dwindling resources. Program and Departmental managers responsible for resource decisions are demanding more detailed and agile resource justifications which can accommodate multiple budget scenarios. In response, program offices require tools that efficiently translate estimated program costs into flexible and defensible budgets... | 2012 | Methods and Models I |
| A Canadian F-35A Joint Strike Fighter Cost Estimation Model | Bohdan L. Kaluzny | The F-35 Joint Strike Fighter (JSF) is a single-engine, stealthy (radar-evading), supersonic multi-role fighter. Canada intends to purchase 65 F-35A-conventional takeoff and landing variant (CTOL) - jets for delivery between 2016 and 2022. In order to secure production, Canada will likely have to commit to procurement as early as 2012. Until recently, Canada relied on the United States (U.S.) JSF Program Office (JPO) for projected costs. In early 2011, the Department of National Defence (DND) F-35A cost estimate came under public scrutiny as a result of a Parliamentary Budget Officer report (Canada's equivalent to the U.S... | 2012 | Methods and Models I |
| Will-Cost and Should-Cost Management: It's Not Business As Usual | Zachary Jasnoff | In April , 2011 Under Secretary of Defense for Acquisition, Technology & Logistics Ashton Carter issued the Memorandum: "Implementation of Will-Cost and Should-Cost Management". The memo defines implementation of Should-Cost and Will management for all ACAT I, II and III programs and lists "Selected Ingredients of Should Cost Management". Thus, each organization involved with these programs must successfully deal with the challenges or planning, coordinating and managing Should Cost/Will cost programs and have the necessary tools to quantitatively manage them through their life cycle... | 2012 | Methods and Models II |
| DoD Contracts Database and Interactive Tool | Brian Octeau | Proposal Summary Good cost estimates, analyses and assessments should consider all sources of data available. One of those sources - contract data - is often overlooked, and sometimes misused. For many, the collection, compilation, and interpretation of contract data can be an unwieldy and time consuming venture. However, once compiled, the data can be an invaluable resource for cost analysts, providing insight relative to cost, schedule, and technical growth, highlighting data completeness gaps inherent in other data sources, and revealing data trends and/or patterns... | 2012 | Methods and Models II |
| Rapid Business Case Development Using Macro-Based Excel Tool | Andrew Hutchinson | Purpose: To present a tool that creates business cases based on user-entered customizations such as the start and end year of the investment. With the increasing scrutiny of federal government budgets the development of business cases for future investments will take on added significance. A best practice within industry is the development of business cases to analyze the net present value of potential investment opportunities. The question this analysis answers is how much value will potential initiatives create within your organization and allows for an "apples to apples" comparison of multiple investment opportunities... | 2012 | Methods and Models II |
| An Intuitive Application of Cost Risk Analysis to a LRFS | Blake Boswell | This presentation explains the implementation of an Uncertainty Modeling Capability (UMC) into a USMC LRFS. Similar to Life Cycle Cost Estimate (LCCE), the Logistics Requirements Funding Summary (LRFS) captures Integrated Logistics Support (ILS) related costs for a program throughout its life cycle. The LRFS provides visibility of logistics requirements for Program Objectives Memorandum and budget submissions. Additionally, it helps to plan and quantify requirements, identify and defend funding, and serve as the ILS input to the LCCE... | 2012 | Methods and Models II |
| Creating a Framework for Cost Analysis: The Cost Analysis Support Tool (CAST) | Eric Lumsden | There are many redundancies encountered when creating new cost models, and many operations that can be automated. CAST is a tool created by TASC that provides a framework to accelerate the process of building a cost model. Estimators are able to execute processes at the click of a button that would otherwise be very time consuming to create from scratch or complete by rote. They are able to transfer over work from other projects to eliminate redundant operations... | 2012 | Methods and Models II |
| Identifying the Cost Capabilities of the DoDAF Architecture Framework | Holly A. H. Handley, Resit Unal, Andreas Tolk | The Department of Defense Architecture Framework (DoDAF) is a key tool used by the United States engineering and acquisition communities to describe the overall structure for designing, developing, and implementing systems. In 2009, version 2.0 of DoDAF was released. This version represented a major shift in focus from output products, or documentation, to the collection of the underlying data that represents the architecture. This version describes a new data model, the DoDAF Meta Model (DM2); the DM2 provides the mechanism needed to collect, organize, and store data. While many pre-defined models reside in the DoDAF, none of them are considered cost models of the system... | 2012 | Methods and Models II |
| EDGARS CB - A New Memory Tool for Costers | William Barfield | Are you befuddled by what it takes to do a cost estimate? What kind of information is used and when is it needed so that a defendable cost and benefits estimate can be provided to the investment decision authority? Since beginners and professionals alike rely on tricks-of-the-trade to help get their job done, a new and easy memory aid is presented... | 2012 | Methods and Models II |
| Significant Reasons to Eschew Log-Log OLS Regression when Deriving Estimating Relationships | Stephen A. Book | Log-Log Ordinary Least Squares (LLOLS) regression, considered in the 18th and early 19th Centuries as the best (and, in fact, the only) method for fitting nonlinear algebraic relationships of the form y = axb to data sets of (x,y) pairs, has a number of serious defects that make it far from adequate for CER development in the 21st. No other option was available 200 years ago, but the advances in computing power and techniques of statistical optimization available to us today leave no reason to stick with an obsolete method. In the 21st Century, we insist that 21st Century engineering technologies be applied, so why would we continue to develop CERs to estimate them using 18th Century statistical methods?... | 2012 | Methods and Models II |
| Valuation in Cost Estimating: Taking a Page from the Investment Banker's Playbook | Kevin Schutt | Valuation is "the process of estimating what something is worth". It is commonly undertaken in investment banking to support activities such as initial public offerings of company stock and mergers & acquisitions. The need for valuation analysis arose in a cost estimation setting in order to satisfy Office of Management and Budget budgetary treatment criteria, yet valuation techniques in general may be a useful addition to the cost estimator's toolkit, facilitating price analysis and cross-checks of cost estimates. Likewise, cost estimation techniques may inform valuation analysis. While cost estimation generally results in market-independent estimates, valuation provides the viewpoint of what the market will bear. This presentation will provide an overview of four major valuation techniques: Price multiple; Replacement cost; Comparable transaction, and Discounted Cash Flow. A case study in valuation will be presented. | 2012 | Methods and Models II |
| Cost Estimating Training for Non-Cost Estimators | Jeremy Eden | The effort to improve the accuracy and fidelity of cost estimates is critical given budget restraints. As this effort continues, non-cost estimators (program managers, engineers, logisticians, etc.) are more frequently facing the demand of preliminary cost estimates prior to the development of formal cost estimates. These preliminary estimates are done in order to streamline the cost estimating process and guard against risk for a program in development. While most non-cost estimators have the information and tools available to address early program cost estimating needs, many do not have the skills and experience necessary to make good use of the resources at their disposal... | 2012 | Methods and Models II |
| Just-In-Time Cost Analysis | John Ko | Booz Allen developed and maintains a robust Life Cycle Cost Estimate (LCCE) for the acquisition of the Urgent Universal Need Joint Mine Resistant Ambush Protected (MRAP) vehicle program, the Defense Secretary's top acquisition priority. Building and maintaining the LCCE was challenging because the MRAP Program is unique from other urgent needs Acquisition Category I-D programs in two ways. First, to accommodate the aggressive and accelerated schedule, the MRAP Program conducted an open competition, which required a review of several designs from various contractors in a short period... | 2012 | Methods and Models II |
| Estimating Alternatives for Joint Future Theater Lift (JFTL) | Robert Georgi, Bruce Pursel | US military operations in Afghanistan and Iraq are making military transport aircraft work at rates not foreseen just a decade ago. At the same time the existing inventory of transport aircraft are limited in their ability to transport heavy cargo to austere unimproved landing zones where military forces routinely operate. The US Air Force (USAF) with considerable support from the US Army conducted the Joint Future Theater Lift (JFTL) Technology Study (JTS) to consider how best to equip its theater lift fleets beyond 2020 to address this need... | 2012 | Methods and Models II |
| Exploring Methods of Conflating Data from Various Data Sources | Ashley Moses | When creating a cost estimate, it is not out of the realm of possibilities that one might be required to combine data from multiple sources to get an accurate estimate. Coleman, Braxton, Druker, et al. have presented research on which methods might be the best to combine opinions given by subject matter experts (SMEs). Additionally, they have also discussed what adjustments are needed to make the estimates provided by SMEs more accurate as studies suggest that SMEs will tend to underestimate the range of possible data values when appropriate feedback is not given to them... | 2012 | Methods and Models II |
| Volatility and Cost Estimating | Jennifer Leotta | This paper will examine the properties and uses of implied volatility, stochastic volatility, and historic realized volatility. Further discussion will focus on what applications an assessment of market volatility brings to the field of cost estimating through the application of a volatility range derived for fuel prices at varying intervals over the course of a generic program's life cycle. | 2012 | Methods and Models II |
| Introducing Reliability and Availability Requirements into TOC Models | Evin J. Stump, Wendy W. Lee | Two important system requirements are reliability and availability. Reliability is the probability of no disabling failures over a certain span of time, while availability is the ratio of the time the system will actually fulfill its operational expectations, to the total time it is expected to fulfill them. Both of these are often included in system specifications... | 2012 | Parametrics |
| Estimating Relationship Development Spreadsheet and Unit-as-an-Independent Variable Regressions | Raymond P. Covert, Noah L. Wright | MCR has constructed an estimating relationship (ER) development spreadsheet based on the zero percent bias, minimum percent error (ZMPE) regression technique to help with more credible and efficient development of cost improvement curves and cost estimating relationships (CERs). The CER-development method accommodates linear, power and triad functions with single and multiple technical and dummy independent variables. Furthermore, the ER development spreadsheet may be modified to accommodate other functional forms that may be of value in particular contexts... | 2012 | Parametrics |
| Are Parametric Techniques Relevant for Agile Development Projects? | Arlene Minkiewicz | Agile software development practices are predicated on the following tenets as introduced in 2001 in the Agile Manifesto [1] - Individuals and interactions over processes and tools - Working software over comprehensive documentation - Customer collaboration over contract negotiation - Responding to change over following a plan | 2012 | Parametrics |
| Comparing Bottom-Up and Top-Down Estimating Approaches in a Custom Cost Model | Donald L. Trapp, Noah L. Wright, Lisa Hackbarth | Developing a cost model involves several steps such as data collection, normalization, cost estimating relationship (CER) development, model building and documentation. Choosing an appropriate level of detail for each CER is an important consideration. This is influenced by several factors such as user needs, data availability as well as the statistical behavior of the data. The cost modeler must sometimes choose to estimate costs for particular work breakdown structure (WBS) elements with either a bottom-up or top-down approach... | 2012 | Parametrics |
| Error Analysis of a Custom Cost Model | Raymond P. Covert, Donald L. Trapp, Noah Wright | How do you know how well a custom cost model performs? If it is based on historical data, then the simplest performance measurement is the model's percent standard error (PSE). The PSE measures a historically-derived model's ability to re-predict its underlying database. In this presentation we will demonstrate an analytic error analysis of a custom cost model using historically-derived cost estimating relationships (CERs) and factors. We will provide a sample model and show how we calculated the PSE of each of the CERs from the data as well as developing the error calculations for each step in the model. We will show how we used the concepts of effective correlation coefficients, propagation of errors, and the calculated variances of the sums and differences of two random variables. | 2012 | Parametrics |
| The Function Point Based Pricing Model: The Price is Right! | Daniel B French | Many IT organizations face a difficult challenge managing contractual engagements with vendors for application development and maintenance services. Regardless of the pricing model or acquisition strategy, projects come in late and over-budget more often than not. Be it a time and materials, cost plus, fixed price or other contractual arrangement, both the vendor and client face the challenge of providing the functionality desired, minimizing risk, and ensuring that the vendor is able to generate a reasonable profit from the engagement... | 2012 | Parametrics |
| Tactical Missile Bluebook Cost Model Development | Donald L. Trapp, Noah L. Wright, Lisa Hackbarth | As part of a quality review process, MCR constructed a cost model based on the cost estimating relationships (CERs) and cost factors contained in the Tactical Missile Bluebook. This effort proved to be valuable in refining the existing CERs and factors and in improving the Bluebook. While building the model, the CER developers found instances where CER input definitions in the documentation needed to be more specific to better serve CER users. When comparing candidate CERs to each other in the regression development process, CER developers relied on error statistics and goodness-of-fit metrics, but these metrics provided an incomplete comparison... | 2012 | Parametrics |
| Tactical Missile Bluebook and Cost Model Overview | Raymond P. Covert, Donald Trapp, Noah Wright, Andrew Drennon | MCR recently developed the Tactical Missile Bluebook together with a cost model based on the information contained in it. The Bluebook contains cost estimating relationships (CERs) for tactical missile development and production based on 13 programs and 153 recurring production lots. In this presentation, we show how the Bluebook is organized and how the CERs and factors were developed, including the filtering methods used to determine data points to be included and excluded from CER development, the challenges of building a useful estimating method from this compilation of data, the methods used to develop the estimating relationships, the results obtained, and significant general findings... | 2012 | Parametrics |
| Understanding and Measuring the Impact of Design and Systems Engineering Decisions on Affordability | Zachary Jasnoff | Methodologies for understanding and measuring the impact of design and systems engineering decisions on affordability In today's data driven cost estimating environment, it is critical to understand the impact of design and systems engineering decisions on cost. It is also important to leverage actual cost data in developing data-driven Cost Estimating Relationships (CERs) that relate engineering design and performance parameters to cost. In addition, the ability to conduct sensitivity analysis on not only the CER, but on the entire system estimated is needed to understand the full impact on Measures of Effectiveness and Measures of Performance... | 2012 | Parametrics |
| Applying Parametric Cost Models as a Predictive Parameter for CMMI Compliance | David Bloom, Wanda Grant, Robert Wright, Gary Bosworth | The Electronic Engineering Center (EEC) of Raytheon Space and Airborne Systems (SAS) recently breezed through a Capability Maturity Model Integration (CMMI) Level 5 assessment, jumping from an existing CMMI level 3 capability and without needing a separate CMMI Level 4 assessment in the process. Although this success was the result of a lot of hard work by many contributors, the bottom line is that there were very few changes in terms of business processes that were required to obtain a CMMI Level 5 organizational assessment, the hallmark of a learning and optimizing organization... | 2012 | Parametrics |
| Using Method of Moments in Schedule Risk Analysis | Raymond P. Covert | A program schedule is a critical tool of program management. Program schedules based on discrete estimates of time lack the necessary information to provide a robust determination of schedule uncertainty and therefore the risk that the proposed schedule will be completed on time. To determine the risk that a proposed, discrete schedule will meet a particular schedule and to find the probable critical paths (i.e., the criticality index), a probabilistic schedule risk analysis (SRA) is performed. SRA is a process by which probability density functions (PDFs), usually triangular, are defined at the task level in an effort to quantify the uncertainty of each task element... | 2012 | Risk |
| Joining Effort and Duration in a Probabilistic Method for Predicting Software Cost and Schedule | Michael A. Ross | This paper describes a data-driven method for estimating the cost and schedule of software development projects. This method correlates the estimates of cost and schedule such that constraining (perhaps reducing) the cost will impact the estimated schedule and constraining (perhaps compressing) the schedule will impact the estimated cost. This method provides these estimates of cost and schedule that are probabilistic (i.e., provide a range of possible outcomes with associated probabilities of attainment); a capability that is essential to analyzing the impact that affordability and budget constraints have on program cost and schedule and their associated risks... | 2012 | Risk |
| Is My Schedule Ready for Risk Analysis? | Mario Fountano | The ultimate goal of schedule risk analysis is the ability to provide project stakeholders with a risk mitigation plan that allows the team to make informed decisions concerning future cost and schedule performance. However, in order to perform schedule risk analysis, a project schedule must first be valid, and provide a clear path to project completion. This study presents the steps our team takes in order to evaluate the readiness of project schedules prior to integrating a risk analysis. Often, project schedules contain flaws in duration and logic, are left un-statused, or were never developed to capture the true nature of project work. The assessment process our team follows helps to build more confidence and transparency into project work. | 2012 | Risk |
| Utilizing Optimization Technique to Enhance Cost and Schedule Risk Analysis | Colin Smith, Brandon Herzog | Advancements in Monte Carlo simulations have enabled cost and schedule uncertainty analysis to be conducted on large, complex programs which could never be analyzed using conventional methods. Analysts are now able to predict with increasing confidence the budgetary requirements, schedule forecast, and risk mitigation measures on projects at all points during their lifecycle. These advancements in Monte Carlo technologies and methodologies have further lowered the barrier to entry for conducting uncertainty analysis; have reduced simulation times to mere fractions of a second; and have allowed for a comprehensive, integrated cost, schedule and risk analysis... | 2012 | Risk |
| The Unseen: Statistical Inference with Limited Data | Trevor VanAtta | Objective measurements of probability are often unavailable, and most significant choices under risk require an intuitive evaluation of probability.' -Nobel Laureates Daniel Kahneman and Amos Tversky. What are the odds of rolling a sum total of seven when tossing two dice? What is the probability of red turning up after a spin of a European roulette wheel? Most analysts, given a little time and a calculator, could answer these two questions with exact precision. For both of these questions, there is only one true correct answer. Such is the nature of probability analysis for questions that are decompositional (all possible outcomes can be determined), frequentistic (the experiment can be repeated an infinite number of times), and algorithmic (the results can be measured with numbers)... | 2012 | Risk |
| Applying the Pareto Principle to Distribution Assignment in Cost Risk and Uncertainty Analysis | James R. Glenn, Christian Smart, Hetal Patel, Lawrence Johnson | Significant effort and statistical knowledge is required in order to complete an accurate uncertainty analysis of a cost estimate. First, a representative sampling of data points must be collected in order to model appropriate distributions for each random variable in the estimate. Additionally, correlation between the random variables in the estimate must be properly assessed and applied. Finally, to generate the cumulative distribution of total costs (i.e., S-Curve) either the joint cumulative distribution function must be computed or an approximation technique such as Monte-Carlo simulation must be used... | 2012 | Risk |
| Inflation Cost Risk Analysis to Reduce Risks in Budgeting | Michael DeCarlo, Eric Druker | For any project there is a danger of unanticipated cost growth because inflation rates are extremely difficult to estimate. This presents a significant challenge to estimators. Predicting future inflation rates with some precision is possible, however, when the appropriate analysis is implemented. Even with previous recommendations from a major federal and commercial consulting firm, there is evidence indicating that the government has not been making adequate assessments of inflation rates for future budgeting. Without applying proper attention and techniques to the analysis and prediction of inflation rates, budgets run a higher than necessary risk for increased cost growth due to inflation prediction error... | 2012 | Risk |
| Diagnosing the Top Level Coefficient of Variation: An Alternative Approach | Daniel J. Andelin | The coefficient of variation (CV), defined as the standard deviation divided by the mean, is a useful metric to quantify the uncertainty inherent in a probability distribution, as it provides a relative (and thereby more intuitive) measure of spread than the variance or standard deviation. In the field of cost analysis, the CV can be used as a diagnostic to determine the level of risk and uncertainty built into an estimate and whether those levels fall within the expected range for a program of a given scope and complexity and at a given stage in its life-cycle... | 2012 | Risk |
| Real Data, Real Uncertainty | Alfred Smith, Jeff McDowell, Lew Fichter, Wilson Rosa | Central to any cost risk analysis and model are the uncertainty distributions assigned to point estimates. Ideally, the analyst will have a database of historical cost and technical information that can be used to objectively develop Cost Estimating Relationships (CERs) using approved statistical methods. From that analysis the analyst will be able to objectively define the shape and dispersion of the CER uncertainty distributions. Similarly, with a suitable database or expert opinions to draw upon, the analyst will be able to develop uncertainty distributions for the inputs to the CERs... | 2012 | Risk |
| Enhanced Scenario-Based Method for Cost Risk Analysis: Theory, Application, and Implementation | Bryan Flynn, Peter Braxton, Paul Garvey, Richard Lee | There is a growing realization within the cost-analysis community that estimates of cumulative probability distributions of cost, or S-curves, too often understate true, underlying risk and uncertainty. Several organizations cite cases where return program acquisition costs, or actuals, fall at the 99th+ percentile on S-curves estimated years previously. This degree of deviation from the mean is a legitimate possibility for any one acquisition program. After all, there's no such thing as an "average" program. Variation is expected... | 2012 | Risk |
| SAR Data Analysis, CV Benchmarks, and the Updated NCCA S-Curve Tool | Richard Lee, Peter Braxton, Kevin Cincotta, Brian Flynn, Ben Breaux | To support the development of better probabilistic cost estimates, the Naval Center for Cost Analysis (NCCA) has championed the development of the S-Curve Tool, which was well received at both the 44th Annual Department of Defense Cost Analysis Symposium (ADoDCAS) in February, 2011 (1), and the joint Society of Cost Estimating and Analysis (SCEA) / International Society of Parametric Analysts (ISPA) conference in June, 2011. This paper presents ongoing research to support both continued improvement of the S-Curve Tool and greater understanding of the nature of cost growth for major acquisition programs; its mean value (risk) and variability (uncertainty); and the components thereof... | 2012 | Risk |
| A Systematic Approach for Empirical Sensitivity Analysis on Monte Carlo Models | Matt Pitlyk | A key component of risk analysis is the sensitivity analysis performed on the input variables for Monte Carlo models with the goal of determining those variables that cause the most variation in the final distribution and identify the best candidates for risk mitigation plans. While the standard technique of calculating the correlation coefficient between the final distribution and each input distribution is appropriate for linear models, it is not sufficient to accurately identify the largest uncertainty drivers. In the case of the non-linear models it is neither appropriate nor accurate... | 2012 | Risk |
| Fitting Absolute Distributions to Limited Data | Blake Boswell | The choice of probability distributions is a critical component for cost risk and uncertainty modeling. When data is available, distribution fitting techniques, such as Goodness of Fit (GoF) tests and Information Criteria (IC), can be applied to determine distributions that accurately describe potential cost realizations; however, with limited data GoF tests and IC based methods provide little or no insight into the best distribution choice. Therefore, when data is limited it is standard practice in cost risk and uncertainty modeling to solicit expert opinion in the construction of triangular distributions with vertices representing the best case, typical, and worst case scenarios... | 2012 | Risk |
| The Importance of Software Cost Estimating Standards among a Diverse Community | Kyle Thomas, Jenny Woolley, William Black | Many cost analysis organizations within elements of the United States Intelligence Community (IC) have developed cost estimation and data collection methods that are based on years of Agency-specific historical programmatic data. These methods have been established over time, effectively creating independent approaches across the Community. While these techniques are valuable, the cost estimation and reconciliation process highlights differences in estimation methods that make analysis across the IC difficult... | 2012 | Software/Hardware |
| Learning Curve Analysis with EZQuant - An Overview | Michael Mahoney | Sure you learned how to calculate learning curves by hand while studying for the CCEA exam, but what will you use to calculate them from day to day? This session will provide an overview of the freely available DCAA EZQuant regression analysis shareware. EZQuant takes minutes to learn and is easy to use. Unit and Cumulative Average learning curve calculations and projections are supported. Since DCAA recognizes the software you don't spend time validating the tool. Natural logarithms are fascinating but EZQuant is just fast. | 2012 | Software/Hardware |
| Estimating for Lifecycle and Product Line Affordability | JoAnn Lane, Barry Boehm, Ray Madachy, Supannika Koolmanojwong | A significant challenge in systems engineering and acquisition is to justify investments in new systems and the evolution of existing systems and product lines. In an era of shrinking budgets, especially in the U.S. Department of Defense, choices are driven by affordability, and not just the affordability of the initial development. Affordability includes development, deployment, usage, maintenance, and retirement/disposal costs. A way to assess or estimate system or product line affordability is through Total Ownership Cost (TOC) analysis... | 2012 | Software/Hardware |
| Software Cost Estimating for Iterative and Incremental Development Programs | Bob Hunt, Jon Kilgore, Jennifer Swartz | Iterative and Incremental Software Development processes are at the heart of a cyclic software development process. Iterative and Incremental Software Development processes evolved in response to the weaknesses of the traditional waterfall model. Iterative and Incremental Software Development start with an initial planning and ends with deployment with the cyclic interactions in between. The goal is to deliverer what the user needs at the end of the process, not only what was envisioned at the beginning of the process. Iterative and incremental development is an essential part of the Rational Unified Process, Extreme Programming, and generally the various agile software development frameworks... | 2012 | Software/Hardware |
| Cloud Computing - Changing the Way We 'Do' Software | Arlene Minkiewicz | In 1961 at the MIT Centennial, John McCarthy opined "if computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility&. The computer utility could become the basis of a new and important industry."[1] In 2006, Amazon Web Services was launched providing computing on a utility basis. Since that time the notion of cloud computing has been emerging and evolving... | 2012 | Software/Hardware |
| Utilization of Visual Basic in Cost Estimating Tools | Jeremy Eden | As collaborative computing environments become more prevalent in all industries, the cost estimating industry is no exception to this movement. Increasing amounts of pressure are put upon cost estimators to develop tools that are robust in design, have long standing methodologies, do not require proprietary software or licenses, and are and easy to use by both the advanced cost estimator looking for maximum control and the novice simply trying to diligently support the early stages of a development program... | 2012 | Software/Hardware |
| A Standard Process for Software Code Counting | Betsy Legg, Brian Enser | Software Lines of Code or SLOC has long been one of the critical inputs for measuring the effort and time involved with software development projects. SLOC is by no means the only way to measure the size of a project but it is the most mature measure and is well understood by decision makers. Software cost estimating that relies on SLOC as an input depends on accurate and consistent counts. The problem is that no standard has been universally accepted for defining SLOC. Numerous code counting tools in use throughout the industry rely on interpretations of the definition of a line of code... | 2012 | Software/Hardware |
| COTS Estimating Metrics for Increased Cost Accuracy | Joshua Patapow, Eric Timinski | Observed cost analysis issue was that multiple Commercial Off-The-Shelf (COTS) Hardware/Software cost estimates were significantly higher than recent contract award values. Using data from those contract awards, an analytical approach and metrics were developed to ensure increased accuracy on future estimates. For COTS Hardware (HW) and Software (SW), a metric of the actual contract price as a percentage of the mean online price/list price (Part I) was developed. Additionally, an estimating relationship was developed (Part II) to calculate the cost of annual maintenance support as a percentage of the actual HW/SW initial cost and the annual maintenance support actual contract price as a percentage of a vendor quote... | 2012 | Software/Hardware |
| Estimation of Expedited Systems Engineering Schedules | Barry Boehm, Dan Ingold, JoAnn Lane | A major objective of many organizations is to reduce the calendar time needed for a project to perform its systems engineering (SE) functions, without compromising the resulting product's needed functionality and quality attributes. This paper will present the derivation of a model to estimate the necessary calendar time to perform the project's SE functions as a function of its product drivers (size, domain familiarity), process drivers (maturity, streamlining), project drivers (team cohesion, collaboration technology support), people drivers (knowledge, skills, and agility), and risk drivers (vs. relative needs for speed, quality, and scalability)... | 2012 | Software/Hardware |
| A Closed-Form Solution for the Production-Break Retrograde Method | Darrell Hamilton, Brian Gillespie | This article explores and discusses concepts surrounding the multi-step retrograde analysis process for learning curve production breaks that was popularized by George Anderlohr, in his 1969 Industrial Engineering article "What Production Breaks Cost". Mr. Anderlohr based much of his analysis using the cumulative average curve method, but the basic principles have been widely accepted and used to calculate the equivalent calculation using the unit theory learning curves... | 2012 | Software/Hardware |
| Targeting Affordability and Controlling Cost Growth through Should-Cost Analysis | Anthony DeMarco | On September 14th, 2010, The Honorable Ashton B. Carter; Under Secretary of Defense for Acquisition, Technology and Logistics, released a memorandum addressed to the acquisition professionals of the Department of Defense. The primary thrust of the memorandum was the current need for greater efficiency and productivity in defense spending. Secretary Carter provided guidance organized into five initiatives... | 2011 | Applications |
| An Application of Data Mining Algorithms for Shipbuilding Cost Estimation | Bohdan L. Kaluzny | The North Atlantic Treaty Organization (NATO) Research and Technology Organization (RTO) Systems Analysis and Studies (SAS) 076 Panel (NATO Independent Cost Estimating and its Role in Capability Portfolio Analysis) is a working panel generating independent cost estimates for NATO systems with the aim of standardizing how NATO countries conduct cost estimation. One of the systems analyzed by the SAS-076 Panel in an ex post exercise was Her Netherlands Majesty's Ship (HNLMS) Rotterdam Landing Platform Dock (LPD), an amphibious transport dock ship that was launched in 1997... | 2011 | Applications |
| Selection of Data Source for Systems Contractor Labor Rates and Overheads and Their Application | Brian Wilkerson, Wallace Riggins | Selection of the best data source can reduce the uncertainty in system contractor labor rates and overheads, facilitate improved government program budgeting, and produce savings for the government in contract negotiations. This presentation compares and contrasts the use of historical contract data vs. Forward Pricing Rate Recommendations (FPRRs) in determining contractor labor rates and overheads in the preparation of Independent Government Cost Estimates (IGCEs). The advantages and disadvantages of each approach are discussed. The development of forward pricing rates beginning in the proposal stage, through DCMA recommendation and DCAA audit to a bilaterally signed agreement is covered... | 2011 | Applications |
| Rolling On The Affordability River (While Managing The Acquisition Program In The Rapids) | Christopher Svehlak | From inception through execution, every acquisition program in the Department of Defense is now expected, even mandated, to incorporate aspects of affordability. This paper explains the role and importance of affordability; then, taking an empirical approach, it presents scenarios depicting actual events based on the author's observations and experience within the context of two separate program offices. Affordability, cost estimating, and financial management processes are described and detailed within the scenarios. Problem root causes and areas of strength are briefly discussed, and some topical research is offered as supporting and explanatory material throughout. Finally, affordability guidelines, relevant public law, current Department of Defense Instructions, and senior leader perspectives are presented, reinforcing the importance of affordability in acquisition programs. | 2011 | Applications |
| Commercialization Activities at NASA and Resulting Cost Implications | James Roberts, Torrance Lambing | Due to mandates of the current Obama Administration, much of the space launch activities that have traditionally been led and contracted out by NASA are being turned over to the Commercial Sector. NASA's role is changing in many instances from being a program manager - overseeing development of space launch hardware and conducting space exploration missions - to that of a supplier and manager of facilities and infrastructure to support the space development and launch activities of commercial entities... | 2011 | Applications |
| Reducing Maintenance Costs Using Beyond Economic Repair Analysis | Jerry Le May | A Beyond Economic Repair (BER) analysis compares the cost of repairing a product with the cost of replacement giving a company information to help decide if repairing a product is more economical than replacement. Using information from a BER analysis, repair procedures can be written so that once a pre-determined amount of time has been spent on repair without success, a product can then be replaced, spending the additional time on replacement rather than further attempts at repair. A BER analysis starts out as a prediction using anticipated repair costs for a new product to establish the amount of hours spent to attempt repair before stopping repair work and replacing the product... | 2011 | Applications |
| Cost by Capability: Funding the Right Mission Capabilities in a Cost Constrained Environment | John Scardino | In an environment of increasing demands on constrained resources, it has become imperative that Department of Defense (DoD) organizations be able to correctly identify the cost and proper allocation of resources across units to provide a sustained level of mission readiness. As operational units train and deploy across the globe, it is important to provide decision makers with the processes that will enable, implement, measure, and manage cost-wise readiness objectively... | 2011 | Applications |
| Earned Readiness | John Williams, John Scardino | The Department of Defense (DoD) faces complex challenges in today's globally strategic and economic environment. During these times of increasing demands and constrained and limited resources, it is imperative that organizations be able to correctly identify the proper alignment and allocation of resources across operational units. The ability to sustain readiness and performance levels over time at lower cost is critical. At multiple levels across the enterprise, leadership must be able to determine solidified readiness and financial goals, as well as have the ability to quickly analyze the readiness and financial performance achieved in meeting those goals... | 2011 | Applications |
| INCOSE Affordability Work Group " Design for Affordability | Edwin B. Dean, Joseph Bobinis | The Affordability Working Group of The International Council on Systems Engineering (INCOSE) is striving to understand and promulgate affordability within the systems engineering community. The group began at the 2010 INCOSE International Conference last summer, meets monthly by telephone, and will meet for two days at the end of January 2011 at the annual INCOSE Workshop. In the interest of intersociety sharing, this paper will share the perspective and status of the Affordability Working Group meeting at the INCOSE Workshop. | 2011 | Applications |
| Use of JCL Data for Programmatic Success | Rey Carpio | With surging interest in Joint Confidence Level (JCL) and corresponding expectations levied on the program management communities, what is a PM to do with JCL data and information? This presentation puts emphasis on the value and use of data and information that JCL tools and processes bring to the table. Appreciating the pedigree of the inputs used in JCL development, and the rigor of the JCL process, the presentation will cover the multiple JCL products and the interpretation of the data and information. The presentation will also cover the effective way to communicate the JCL results to the program managers and the stakeholders... | 2011 | Applications |
| Budgeting to the Mean | Rick Garcia, Casey Wallace | Decision makers (and policy) often require cost estimators and analysts to move to a higher percentage on the S-Curve to ensure enough budget is requested so that a program does not overrun its budget target. Although the request to budget at a higher confidence level is a pragmatic attempt to avoid overruns (under budgeting), there are other factors to consider besides simply moving to the right on an S-Curve; such as ensuring a cost estimate captures all relevant uncertainty and acknowledging acquisition changes that will impact a program after a budget has been set... | 2011 | Applications |
| Evolved Expendable Launch Vehicle (EELV) Discrete Event Simulation: Ensuring the Buck Results in a Bang | Colleen Craig, Scott DeNegre | An accurate cost estimate is an essential prerequisite when considering expansion or modification of a major government system. To make an informed decision, cost must be considered in light of the expected benefit of the expenditure, to ensure the investment is justified. We describe one method of performing this analysis, using the EELV supply chain as a demonstrative example. We demonstrate the methodology of Discrete Event Simulation (DES), and its application in the cost community, as a tool to measure the current performance of and potential improvement in the EELV system... | 2011 | Applications |
| Multiply or Divide? A Best Practice for Factor Analysis | Shu-Ping Hu, Alfred Smith | Factors are commonly used in engineering build-up equations to derive cost estimates. For example, software development hours are often estimated based on an estimate of source lines of code (SLOC) divided by a productivity rate: SLOC per hour. To develop the productivity rate, the analyst collects software size (SLOC) and development time (hours) metrics from a variety of programs. Ideally, these programs have similar characteristics to the one being estimated, for instance, similar complexity, language, resources (people and tools) and environment... | 2011 | Applications |
| Interconnected Estimating Relationships: Their Derivation and Application | Stephen Book, Amanda Feather | By the term Interconnected Estimating Relationships, we mean estimating relationships for hardware and software costs, schedules, weights, and below-the-line programmatic costs that are jointly impacted by each other or by the same drivers. There are two common examples of this phenomenon: (1) cost-estimating relationships and schedule-estimating relationships, which are interconnected for many reasons, but primarily because a projects schedule is a significant driver of its cost; and (2) cost-estimating relationships for hardware and software and techniques for estimating below-the-line costs, the latter of which is typically done by applying a factor or percentage to the hardware and/or software cost estimate... | 2011 | Applications |
| Using the New 881 WBS/CES for ERP Acquisition: Lessons Learned | Virginia Stouffer, Gerry Belcher | MIL HDBK 881's work breakdown structure/cost estimating structure (WBS/CES) is under revision, and one of the key new WBSs is an automated information systems WBS for enterprise resource planning (ERP). We used the draft 881 to create the program office estimate for the US Coast Guard's CG-LIMS system. Even with help from key committee members in defining the categories, we encountered some difficulties in assigning ERP costs to their proper categories, in accounting for all the ERP costs, and interpreting some of the newer subelements... | 2011 | Applications |
| Determining Cost Estimating Relationships for Nine FAA Solution Development Elements | William Barfield | The Federal Aviation Administration is responsible for management of our National Airspace System, which requires massive amounts of software development and maintenance. The writing and testing of large-scale software is expensive and involves many substantial costs in addition to the development of the basic software itself. In an effort to improve financial management practices within the FAA, new Cost Estimating Relationships (CERs) were determined for nine FAA Work Breakdown Structure elements pertaining to software development and delivery life cycles activities. CERs are regression equations are typically based on normalized actual costs of prior analogous software development... | 2011 | Applications |
| Standardizing the Cost Technical Baseline - Template and Tutorial | Diane Butler, Jason Newman | Over the last four years, the Department of Defense (DoD) cost community has experienced changes in cost policies, regulations and implementation procedures. Decision makers want more insight into cost estimate development, improvements in estimate quality, and a better understanding of program changes that affect funding levels. As managers of cost estimators, it is our responsibility to provide guidance on how these changes impact our analysts in the program offices... | 2011 | Applications |
| A Methodology for Multivariate Regression on Large Datasets | Matthew Pitlyk | Large datasets can present a formidable challenge to the analyst. Proper analysis of a long list of potential cost drivers can require enough regression computations and scatter plots to make discovering good relationships difficult. A methodical approach helps to keep the analysis organized and ensures that nothing is overlooked. This presentation will put forth a standardized way to approach analyzing independent variables in regression and an application of automating this approach in VBA... | 2011 | Applications |
| Defense Acquisition University's (DAU) EVM Checklist Workshop | Kim Hunter | As the Department of Defense (DoD) increases emphasis on Earned Value Management (EVM) as a project management tool, more and more program offices are finding themselves having to implement and then subsequently execute EVM on their contract. Although EVM is a project management tool, EVM responsibilities are routinely assigned to the business or financial management section which is typically staffed with financial management analysts and cost estimators who may have little or no EVM experience or training. Even if an EVM analyst is on staff, his or her experience and training can vary greatly... | 2011 | e-Track |
| An Improved Method for Predicting Software Code Growth: Tecolote DSLOC Estimate Growth Model | Michael Ross | This paper describes the Tecolote DSLOC Estimate Growth Model, which provides probabilistic growth adjustment to Technical Baseline single-point Estimates (TBEs) of Delivered Source Lines of Code (DSLOC) for New software and for Pre-Existing software, these estimates being sensitive to the maturity of the estimate; i.e., when, in the Software Development Life Cycle (SDLC), the TBE DSLOC estimate is performed. The model is based on Software Resources Data Report (SRDR) data collected by Dr. Wilson Rosa of the U.S. Air Force Cost Analysis Agency (AFCAA). This model provides an alternative to other software code growth methodologies such as Mr. Barry Holchin's (2003) code growth matrix. | 2011 | e-Track |
| Parameters in Parametric Cost Estimating | Myung-Yul Lee | Parametric estimating methods are one of the most desirable and have high creditable due to the fact that this method use actual hours for the weapon system we are estimating. The purpose of this paper is to study how to create a parametric estimating model for weapon system in the aircraft. C-17 Engineering organization will be utilized for generating estimating model... | 2011 | e-Track |
| Integrated Cost-Schedule Risk Analysis using Risk Drivers and Prioritizing Risks | David Hulett | Many cost estimates assume that the project schedule is engraved in stone. A risk analysis of the cost estimate is conducted using only cost risk drivers. In this analysis the impact on cost risk of uncertainty in schedule is often ignored or incompletely considered. This presentation discusses how cost and schedule risk analysis is integrated so that cost risk is affected by both traditional cost risk elements (e.g., uncertain material costs or labor rates) and schedule risk elements (e.g., fabrication or testing that may take longer and thus cost more)... | 2011 | e-Track |
| Historical Trend Analysis Analysed | Dale Shermon | This article describes three alternative approaches to historical trend analysis. First, the study considers the trend over time of the complexities of past systems. This results from the application of a parametric cost model (PRICE H) to the normalisation of historical projects costs and to the plotting of the complexity over time. | 2011 | Journal of Cost Analysis and Parametrics |
| A Probabilistic Approach to Determining the Number of Units to Build in a Yield-Constrained Process | Timothy P. Anderson | Many cost estimating problems involve determining the number of units to build in a yield-constrained manufacturing process, when it takes, on average, n attempts to produce m successes (m - n). Examples include computer chips, focal plane arrays, circuit boards, field programmable gate arrays, etc. The simplistic approach to this problem is to multiply the number of units needed, m, by the expected number of attempts needed to produce a single success, n. | 2011 | Journal of Cost Analysis and Parametrics |
| Statistical Foundations of Adaptive Cost-Estimating Relationships | Dr. Stephen A. Book, Daniel I. Feldman | Traditional development of cost-estimating relationships (CERs) has been based on full data sets consisting of all available cost and technical data associated with a particular class of products of interest, e.g., components, subsystems or entire systems of satellites, ground systems, etc. In this article, we review an extension of the concept of analogy estimating to parametric estimating, namely the concept of adaptive CERs - CERs that are based on specific knowledge of individual data points that may be more relevant to a particular estimating problem than would the full data set. | 2011 | Journal of Cost Analysis and Parametrics |
| Use of Earned Value Management Trends to Forecast Cost Risks | Daniel I. Feldman | This article uses earned value management trend analysis to forecast trends in BAC and BCWP. The resulting equations are then used to solve for the expected month at completion. With the month at completion date in hand, the article uses trend analysis to and the EAC at that month along with the BAC at that month far in the future to solve for VAC. By using variance against a baseline, the article shows how much risk this program will incur by the date at completion. | 2011 | Journal of Cost Analysis and Parametrics |
| An Application of Data Mining Algorithms for Shipbuilding Cost Estimation | Bohdan L. Kaluzny, Sorin Barbici, Garan Berg, Renzo Chiomento, Dimitrios Derpanis, Ulf Jonsson, David Shaw, Marcel C. Smit, Franck Ramaroson | This article presents a novel application of known data mining algorithms to the problem of estimating the cost of ship development and construction. The work is a product of North Atlantic Treaty Organization Research and Technology Organization Systems Analysis and Studies 076 Task Group "NATO Independent Cost Estimating and its Role in Capability Portfolio Analysis. | 2011 | Journal of Cost Analysis and Parametrics |
| Estimating Cost and Schedule of Reliability Improvement | Dr. David A. Lee, E. Andrew Long | We extend a well-established model of reliability growth in a reliability improvement program, the Army Materiel Systems Analysis Activity Maturity Projection Model (AMPM), to include a model of the programs cost. | 2011 | Journal of Cost Analysis and Parametrics |
| A Closed-Form Solution for the Production-Break Retrograde Method | Dr. Brian Gillespie, Darrell Hamilton | This article explores and discusses concepts surrounding the multi-step retrograde analysis process for learning curve production breaks that was popularized by George Anderlohr, in his 1969 Industrial Engineering article "What Production Breaks Cost." Mr. Anderlohr based much of his analysis using the cumulative average curve method, but the basic principles have been widely accepted and used to calculate the equivalent calculation using the unit theory learning curves. | 2011 | Journal of Cost Analysis and Parametrics |
| Using Earned Value Data to Detect Potential Problems in Acquisition Contracts | C. Grant Keaton, Dr. Edward D. White, Eric J. Unger | Government contractors report earned value information to government agencies in monthly contract performance reports. Though major differences may exist in the data between subsequent contract performance reports, we know of no government effort to detect these occurrences. The identification of major changes may locate and isolate problems and, thus, prevent million and billion dollar cost and schedule overruns. | 2011 | Journal of Cost Analysis and Parametrics |
| A Probabilistic Method for Predicting Software Code Growth | Michael A. Ross | A significant challenge that many cost analysts and project managers face is predicting by how much their initial estimates of software development cost and schedule will change over the lifecycle of the project. Examination of currently-accepted software cost, schedule, and defect estimation algorithms reveals a common acknowledgment that estimated software size is the single most influential independent variable. | 2011 | Journal of Cost Analysis and Parametrics |
| Life Cycle Cost Growth Study - 20 Science Mission Directorate (SMD) Missions | Claude Freaner | Previous studies have concentrated on examining development cost growth, excluding launch vehicle, mission operations and data analysis. This study looks at cost growth from a Life Cycle Cost perspective, including launch vehicle and mission operations and data analysis. Costs are separated into major WBS elements to see if any elements are more likely to have cost growth than others. The different WBS elements are tracked over time with snapshots of costs at major milestones to see where in the project life cycle the cost growth occurs. | 2011 | Management |
| How Cost Arises - How We Can Reduce Cost | Edwin B. Dean | This presentation is based upon nine years of research for NASA on how cost arises and how we can reduce cost. A summary of this research was first published as the 500+ page NASA Design for Competitive Advantage web site (1994-1998). This paper will illustrate how cost arises, suggest various means of reducing cost, and provide resources for those who desire to further this research. | 2011 | Management |
| Constructing a Price-to-Win | Frank R. Flett | Winning a competitive federal procurement requires more than just writing a good technical proposal. A win strategy that has been formulated without consideration of the competition is bound to fail. A thorough competitive assessment includes the identification of the custome's Most Important Requirements (MIRs), the assessment of our own and our competitors' strengths and weaknesses versus those MIRs, the investigation of our competitors' current market position and behavior on recent competitive procurements, and the postulation of the competitors' strategy options, probable strategy, and price for the opportunity being bid. Only then can we determine our own win strategy and our Price-to-Win (PTW)... | 2011 | Management |
| EELV Should Cost Review Overview and Lessons Learned | James Smirnoff, Karen Schaben, Joe Kabeiseman | In his September 14, 2010 Memorandum for Acquisition Professionals, Ashton Carter, the Undersecretary of Defense for Acquisition Technology and Logistics, directed that the manager of each major program conduct a Should Cost analysis justifying each element of program cost... In early March of 2010, as a response to significant pending contract price increases, supplier readiness uncertainty, and unresolved Defense Contract Audit Agency (DCAA) audit issues, Michael Donley, the Department of Defense (DoD) Executive Agent for Space directed a Should Cost Review (SCR) of the Evolved Expendable Launch Vehicle (EELV) program... | 2011 | Management |
| What to Know When Estimating Virtualized Environment Costs | Jennifer Woolley, Ryan Boulais, Sandra Williams | Virtualization, which allows a single piece of hardware to function as multiple virtual pieces of hardware, is one of the principal trends for current IT systems. As the technology has matured over the last several years, various organizations in both the public and private sectors have made the transition to operating in a virtual environment in an effort to reduce costs and streamline data access. With server, storage, and network virtualization becoming more prevalent, cost analysts are frequently asked to estimate and assess the costs associated with virtualization... | 2011 | Management |
| Building a Cost Analysis Improvement Group - Best Practices and Lessons Learned | Keith Robertson, Linda Williams, Erik Burgess, Jay Jordan | The NRO Cost Analysis Improvement Group (CAIG) has been in existence in various forms for over 20 years. It has migrated from an organization that provides independent cost estimates (ICE) for specific product lines to one that provides end to end program acquisition support... | 2011 | Management |
| Lessons Learned: A Case Study in Labor Cost Estimating in a Data-Scarce Environment | Kevin Schutt | Observations and lessons learned from labor rate and utilization development in a data-poor environment are discussed. A case study of an independent cost estimate for a large federal services contract forms the basis of the examples, which include a method for labor rate build-up. The method combines forward pricing rates and GSA schedules, and a mapping of labor categories from multiple disparate sources to those prescribed in the Request for Proposal (RFP)... | 2011 | Management |
| Enhanced Cost Analysis in Support of Aerospace Corporation's Decision Support Framework (DSF) | Mel Broder, Lubo Jocic, Inki Min | Since its inception fifty years ago, the Aerospace Corporation has been supporting government decision making in many programs and at different levels. An objective technical analysis employing subject matter experts using many tools, models, and methodologies is the backbone of Aerospace's Decision Support Framework. Strong customer demand for timely analysis and advances in modeling capabilities have enabled introduction of fast, concurrent, engineering processes into decision support. The Concept Design Center (CDC) and Concurrent Program Definition Environment (CPDE) engines were built to enhance timeliness of design and program definition processes... | 2011 | Management |
| You Really Don't Have to Lose a Million Dollars a Year, a Cost/Price Analytical Journey through the World of Winemaking | Michael Thompson | It's been said that the best way to lose a million dollars a year is to open a winery, the question that as cost analysts is why? By conducting interviews with several small winery owners I found that their business model is more often than not reliant on the pricing structure of large wineries in their region, rather than determining how much they can afford in direct and indirect costs. | 2011 | Management |
| Cost Analysis Process in Manufacturing Industry | Mostain Dara Billah | In the common conflict of interest, there exists between daily operations/manufacturing, sales/marketing, and cost and finance. These conflicts can contend to be as healthy; however it cannot be avoided in a manufacturing environment. Even though these facts may be right, but for last 35 years I have experienced a different opinion that I would like to share with you all, today... | 2011 | Management |
| Best Practices in Aerospace Cost Estimation: Observations from US Air Force and NASA | Robert Georgi, Benjamin Watson | Both the United States Air Force and the National Aeronautics and Space Administration (NASA) develop aerospace projects with cutting edge technology. This paper will provide an overview of cost estimation for the respective organizations based on the authors experience with NASA and Air Force projects. When developing aerospace projects applying cost estimation in a formal manner in organizational structure and personnel training can improve program estimates and risk management... | 2011 | Management |
| Integrating Earned Value Analysis and Independent Cost Estimating for Large, Multiyear System Procurements | Greg Lochbaum, Cole Kupec, Eric Mosier, Chris Massey | The NRO Cost Analysis Improvement Group has integrated the Earned Value Analysis Center of Excellence (ECE) with the Independent Cost Estimating (ICE) group. The intent of this consolidation is to continue to improve the estimating and earned value products that are provided to Agency decision makers. Both the ECE and ICE groups are finding the integration mutually beneficial. The NRO CAIG often uses earned value (EV) information to help develop and validate its high level conclusions reached in independent cost estimates (ICEs). This approach has, at times, led analysts astray... | 2011 | Management |
| Mathematical Lessons Learned from a Year's Worth of ICEs | Ryan W. Boulais, Brett Dickey | Having conducted over 20 Independent Cost Estimates (ICEs) across the Intelligence Community (IC) over the past year, there were a few common mathematical concerns that needed addressing in multiple estimates. These concerns appeared time and again from a cost method and application point of view. Three such topics will be discussed in this paper: calculating and applying correlation, calculating a cost factor from an average rate, and properly implementing distributions in Monte Carlo simulations such that the Mean point estimate and the Mean output from the simulation are the same... | 2011 | Management |
| PARS II: Redefining Program Oversight & Assessment at the Department of Energy | Simon Dekker | Requiring a new method for capturing actionable project performance data from its contractors, the Department of Energy recently implemented its Project Assessment and Reporting System (PARS II), built on Dekker PMIS technology, and revamped its project reporting procedures to better align project performance with strategic Departmental goals. PARS II provides a web-based system that captures accurate, timely, complete, and verifiable project performance data from contractor sites around the country. The system fosters greater transparency and provides project oversight staff a single, online, and secure vantage point from which to monitor up-to-date performance data generated directly from the project sites Earned Value Management systems... | 2011 | Management |
| NASA Implementation of JCL Policy | James Johnson, Charles Hunt | For the past two years NASA has been implementing Joint Cost and Schedule Confidence Level (JCL) analysis within its projects. This analysis is sponsored by NASA Agency level policy, performed by individual NASA projects (and is some cases programs), and reviewed by a NASA non-advocate organization. This presentation will cover the Agency's rationale for implementing JCL, the evolution of the JCL policy, and the lessons learned with regards to project implementation. This presentation will also address the current measures that NASA is sponsoring to make the JCL methodologies, processes, products, tools, and communication better for the JCL community. | 2011 | Management |
| Obtaining Greater Efficiency and Productivity in Defense Spending " The Carter Memo | Tom DuPre, Kristen Schulte | The Nov 3rd, 2010 Ashton Carter memorandum directing initiatives within the Department of Defense to obtain greater efficiency and productivity in Defense spending presents an ongoing challenge to the services. How do PEOs and PMs reduce the costs of their programs without reducing capabilities? This presentation outlines processes to set affordability goals early in the acquisition life cycle and a strategy to help PMs focus on achieving should cost versus will cost estimates. There is an essential role for the professional cost estimator to help set affordability targets at Milestone A that will be treated as Key Performance Parameter (KPPs), and to participate in systems engineering tradeoff analysis in support of Milestone B decisions... | 2011 | Management |
| Comprehensive Assessment of Contract Performance using Earned Value Management Data | William Laing, Colleen Craig, Richard Lee, Scott DeNegre | Purpose: This presentation outlines a set of data views that provide a comprehensive assessment of contract performance using Earned Value Management (EVM) data. The proposed views provide alternatives to traditional EVM metrics and are intended to provide early-warning indicators of contract performance... | 2011 | Management |
| Overcoming Challenges in Estimating Advanced Technology Programs | Zachary Jasnoff, Dan Nussbaum | During the Source Selection process, many different types of cost estimating are required by both the Source Selection Board (Government) and the Contractor (Performer). This includes analysis such as Cost Realism, Analysis of Alternatives, Ghosting the Competition and Growth Estimating. Often times, both Government and Contractors have no consistent, repeatable and traceable way of performing these types of analysis. This leads to poor and/or inconsistent decision making during the Source Selection process. However, using parametric methods, these types of analysis are easily accomplished in a fraction of the time of traditional methods... | 2011 | Management |
| Improving Baseline Execution - A Parametric Approach | Steve Sultzer | Too often projects are undertaken without good visibility into the executability of the program baseline. Although baseline analysis and review (including IBR's) may include staffing and Integrated Master Schedule (IMS) review, the baseline is often not compared back to the underlying cost drivers upon which the program was based. This can lead to baselines which appear to be consistent from a staffing/hours/schedule perspective, but which may be unrealistic based on the size, scope, and risk profile of the program... | 2011 | Management |
| The Shortcut to Fully Burdened Cost of Fuel Analysis | Kristopher Atkins, Richard Cline | Frequent attacks against resupply convoys, along with their growth in both length and size, have caused the Department of Defense to investigate ways of shrinking the military's logistics tail. A large logistics tail impedes the mobility, flexibility, and concealment of tactical forces and grows with fuel demand. The DoD is aiming to reduce fuel demand by integrating energy-based analysis of alternatives into the early stages of systems acquisition. An analytic capability that measures fuel delivery costs and assesses the value of investments in efficiency is needed to evaluate competing alternatives... | 2011 | Management |
| Instrument Schedule Delays Potential Impact on Mission Development Cost for Recent NASA Projects (Follow-On Study) | Kristina Kipp, Stephen Ringler, Erin Chapman, Luke Rinard | This study explores instrument schedule delays and their potential impacts on mission development cost for recent NASA projects. Schedule data collected at key milestones for a number of NASA instruments is used to compare planned and actual instrument development times. The study shows average instrument development schedule growth is on the order of 30%. Comparing last instrument delivered development time and mission development cost growth shows a positive correlation, indicating that instrument schedule delays may increase total mission development cost. Instruments are binned by various categories such as instrument type, mass, power, etc. to explore specific trends. The results of this study can be used for planning purposes by project and program managers in charge of future NASA development efforts. | 2011 | Management |
| How to Estimate and Use Management Reserve in an EVM System | Mark Infanti | When using an Earned Value Management System (EVMS), a Management Reserve (MR) is both expected and a 'best practice'. But the misunderstanding of how to establish it, what it is for and how to use it, extends from government agencies to contractors. MR is defined as An amount of the total budget withheld for management control purposes, rather than being designated for the accomplishment of a specific task or set of tasks (ANSI/EIA 748-B) | 2011 | Management |
| United States Military Rank | Michael Brozyna | Many new cost analysts have no idea about the U.S. military rank structure. This presentation goes through the enlisted, warrant and officer ranks of the Services. Insignias and salutations will be offered. It also will give a frame of reference of pay. It also shows how the ranks are abbreviated differently across Services. No math involved. | 2011 | Management |
| Extending FEA and DODAF to Support Cost Modeling | Andreas Tolk, Resit Unal, Charles Keating, Johnny Garcia, Holly Handley | The Federal Enterprise Architecture (FEA) and the Department of Defense Architecture Framework (DoDAF) are widely recognized, applied, and often mandated methods to define future systems and their interplay with current solutions to close capability gaps with fit for purpose solutions. However, the focus is currently on operational capabilities and the cost aspect is easier done separately or as an afterthought once the 'optimal technical solution' is specified. Current research at Old Dominion University in two related domains have the potential to contribute to a paradigm shift in cost modeling based on architectural descriptions. | 2011 | Models and Methods |
| Operating and Support Cost Estimating Methods: An approach to estimate the US Navy's future cost of Ballistic Missile Defense | Brian A. Welsh, Elizabeth A. Koza, Paul L. Hardin III, Marc Greenberg | The Missile Defense Agency's (MDA) Aegis BMD program provides a defensive shield against short and mid-range ballistic missiles. Currently a portion of US Navy ships are dedicating some of their at-sea operations to BMD. Additionally, the Sea Based X-Band (SBX) radar system is supporting BMD. Over the next seven years, MDA will add Aegis BMD capabilities to Romania and Poland. After these land-based sites are constructed, the US Navy will be responsible for operating and maintaining the systems and supporting infrastructure at each site... | 2011 | Models and Methods |
| Utilizing The Capabilities Knowledge Base for Cost Benefit Analysis and Analysis of Alternatives | Chadd Sibert | Department of Defense (DoD) leadership is rigorously evaluating potential acquisition programs much earlier in the programs lifecycle than they have in the past due to the increased scrutiny of defense funding. Two key items are the December 2008 DoD Instruction 5000.02 addressing Milestone-A Analysis of Alternatives (AoA) and recent Department of the Army guidance requiring Cost Benefit Analysis (CBA) for all decision points. This in turn greatly increases the demand for cost information to support these early programmatic decisions. However, are there any tools available to support this effort? If so, how would you utilize them?... | 2011 | Models and Methods |
| Assessing Impact of Funding Constraints to Cost and Schedule | Darren Elliott, Antonio Rippe | Cost risk and uncertainty over the past few years have drawn high interest in the cost estimation industry. Significant effort has been invested by the cost community to develop standard practices and refining tools to standardize how analysts to conduct risk and uncertainty analysis and examine the results. The most common output has been the cumulative distribution function (S-Curve) from which a target confidence level can be identified and a resulting cost target can be generated. This result is normally defined as the risk-adjusted estimate and phased over time to generate a desired funding level for the program... | 2011 | Models and Methods |
| Applying Development Cycle Electronics Hardware Sub-Product Cost Models to Project Execution | David Bloom | Over the last decade, many advances have been accomplished by notable organizations in applying parametric model techniques to establish cost estimates for deliverable electronics systems and electronics products. At the Electronics Center of Raytheon Space and Airborne Systems, we utilize commercial software applications to perform these kinds of tasks... | 2011 | Models and Methods |
| Trade Space, Product Optimization and Parametric Analysis | Doug Howarth | This paper shows how to bound, build and assemble trade spaces for product optimization. The advent of computerized tools that describe available trade spaces has changed not only the nature of optimized product design, but that of parametric cost studies as well. Because these tools allow broader analysis, engineers produce more potential designs and parametricians must produce many more estimates in support of them. | 2011 | Models and Methods |
| Objective System Acquisition Decision Making Utilizing the Analytic Hierarchy Process (AHP) | Hisham Jalil | Today's increasingly complex government acquisition environment engenders increasingly critical decision points for acquisition leaders and decision makers. Perpetual advancements in technology coupled with increasing vendor capability often present one such key decision point for today's technologically complex, mission critical systems. Government decision makers must ask and answer the following question: Do we choose an organic or service acquisition approach for this system? That is, do we build, implement, operate, and maintain the system?... | 2011 | Models and Methods |
| A Comparison of Military and Commercial Submersible Systems, Cost Environments, and Methods for Estimating Submersible Development and Production Costs | Janet Vacca-LeBoeuf, Greg C. Bell, Brian Harris | The U.S. Navy and the U.S. Special Operations Forces (USSOCOM) have established requirements for wet and dry submersible vehicles capable of transporting combat teams and their equipments to and from the shoreline of enemy territory. The submersibles are carried to the vicinity of the enemy either by surface vessels or mounted upon the decks of fleet attack submarines. A recent high profile program, the Advanced Seal Delivery System (ASDS), was initiated to be the principal vehicle for performing this kind of mission. A fleet of as many as six or eight of the 55 ton units was planned... | 2011 | Models and Methods |
| QuickCost 5.0: A Space Project Cost Model | Joe Hamaker | This paper describes the latest release version of the QuickCost suite of cost estimation models for space projects. This version of QuickCost includes databases and models in spreadsheet format for five space project application areas which are 1) automated spacecraft, 2) orbital transfer stages (including a variable for human rated), 3) orbital modules (including a variable for human rated), 4) launch vehicle stages (including a variable for human rated) and 5) liquid rocket engines (including a variable for human rated)... | 2011 | Models and Methods |
| Cost Analysis using Random Forest Prediction | Karen Mourikas, Denise Nelson, James Schimert | What to you do when your cost data is not well suited for linear regression analysis? As an alternative, we experimented with Random Forests, an example of using a computer intensive learning algorithm instead of assuming a parametric model. Random Forests are a collection of Decision Trees. Trees can capture non-linear relationships and interactions among predictors. Individual decision trees are interpretable, however, not necessarily great predictors. By suitably fitting a collection of trees, and averaging the tree predictions, Random Forests results compete with the best machine learning predictors... | 2011 | Models and Methods |
| Multicollinearity in Zero Intercept Regression: They Are Not Who We Thought They Were | Kevin Cincotta | Asked about the identity of the members the opposing team after being defeated by them, a famous football coach once said, They arewho we thought they were. Unfortunately, this is not the case when it comes to multicollinearity in zero intercept linear regression. It is not what we think it is, and the consequences of ignoring the distinction between it and its nonzero intercept counterpart can be devastating... | 2011 | Models and Methods |
| The Infrastructure Service Provider (ISP) Cost Model | Kyle Thomas, Belinda Nethery, Carol Wilson | The current political and economic environment has created a focus on efficiency and productivity, particularly with Intelligence Community and Department of Defense spending. One area for improvement that has been highlighted is that of Information Technology (IT) services. A Government Agency identified IT as an area in which efficiencies could be achieved and has been taking steps to deliver IT as services across the enterprise. Beginning with the FY12-17 Program Build, there has been a desire to project the cost of IT services as an enterprise activity rather than by specific programs or contracts, as they had been in the past... | 2011 | Models and Methods |
| Analysis of Parametric and Database-Driven Cost Estimates in the Transit Industry | L. Brian Ehrler | Most major transit projects require some form of grant, either Federal or State, to be financially viable for a city. For those cities considering a transit project there are billions of dollars available each year from the US Department of Transportation (1). However, to qualify for these dollars the transit agency for the city, must demonstrate that the proposed transit project is cost effective... | 2011 | Models and Methods |
| An Elicitation Method to Generate Minimum-Bias Probability Distributions | Marc Greenberg | As the US Department of Defense develops less mature, more advanced and more complex systems, there is an ever-increasing burden on the cost analyst to employ methods of eliciting requirements, schedule and cost uncertainties from one of more subject matter experts (SMEs). Arguably, the most common technique a cost analyst uses today to elicit such data is to ask each SME for the lowest, most likely and highest value which, consequently, produces a triangular distribution... | 2011 | Models and Methods |
| Economic Elasticity of Tactical Missile Costs | Raymond P. Covert | Economic elasticity is defined as the ratio of the percent change in one variable to the percent change in another variable. In missile production, it is hypothesized that two forms of elasticity dominate: price elasticity of demand and production cost improvement. The former implies prices and demand are inversely proportional, meaning if like goods (e.g., missiles) are cheaper, we tend to buy more of them. Production cost improvement is defined simply as the reduction of costs of successive units. Cost improvement curve theories, such as unit learning, explain the converse effect: If we can buy a greater quantity of an item, its relative cost decreases between successive units.... | 2011 | Models and Methods |
| Modeling to Establish the Affordability KPP | Samuel Toas, Greg Hogan | Dr. Ashton Carter's November 2010 memo provided a number of initiatives aimed at achieving better buying power for Defense spending. The guidance presented several new requirements to be completed during Milestone-A and Milestone-B such as the establishment of Affordability Targets. One or the common threads across both acquisition phases is an analysis of performance vs. cost. Programs are required to generate a target cost KPP and provide the necessary cost tradeoff curves which justify the established target... | 2011 | Models and Methods |
| Dynamic Help Desk Resource Modeling | Sujoy K. Roy, Nolin Huddleston, Steve Sheamer | This paper discusses the concept of Dynamic Helpdesk Resource Modeling for a major Information Technology System. This methodology utilizes regression analysis based on the concept of learning curves to project the resource load for a three tiered help desk system based on a number of factors including system user population and maturity. The paper uses the implementation of an Enterprise Resource Planning System throughout the United States Navy as the user system with a three tiered help desk model as the basis for analysis... | 2011 | Models and Methods |
| A Probabilistic Approach to Determining the Number of Widgets to Build in a Yield-Constrained Process | Timothy P. Anderson | Many cost estimating problems involve determining the number of Widgets to build when it takes, on average, M attempts to produce N successes. Examples include computer chips, focal plane arrays, circuit boards, FPGAs, etc. The simplistic approach to this problem is to multiply the number of Widgets needed, N, by the average number of attempts needed to produce a success, M... | 2011 | Models and Methods |
| A Methodology to Improve the Predictability of the CER with Insufficient Data in Korean Weapon System R&D Environment | Dong Kyu Kim, Sung Jin Kang, Yong Bok Lee | Parametric cost estimating models have been used widely to obtain appropriate cost estimates in early weapon system acquisition process. Parametric cost estimating models are composed of some CER(Cost Estimating Relationship)s based on regression analysis with historical data. However, there are many restrictions in developing Korean version CER because of the insufficient number of projects and the abnormal data characteristic such as multicollinearity, outlier existing, heteroscedasticity, etc. So, the diverse regression methods have been studied in Korea to improve the predictability and stability of the each CER respectively. | 2011 | Models and Methods |
| RealTime Analytics | Eric Druker, Blake Boswell | As organizations increasingly rely on complex analytics using simulation software in their decision making, an inescapable trade-off has arisen: The more robust the analysis, the longer it takes to use to make a decision. One of the primary reasons for this is the time required to run excursion scenarios using new or modified assumptions. Running these excursion scenarios often takes hours, requiring the scheduling of additional meetings before decisions can be made. And, since these scenarios are necessarily run outside of the meeting room, leadership is limited in the number of scenarios they can request... | 2011 | Models and Methods |
| Faster & Better Ways to Determine an Engineering Labor Rate for Pre-Award Development Contract Cost Estimates | Michael Sweeney, Brian Fersch | The accuracy of cost estimates is an enduring issue in the DoD acquisition community. DoD programs too often overrun cost and are subsequently cancelled - a reality which puts the American warfighter at risk. One chief aspect of cost estimates with significant risk and uncertainty surrounding it is the labor rate used for contractor engineering in support of the specific project; if the wrong labor rate is used it can very detrimentally impact an entire estimate, thereby putting DoD programs at risk of cancellation. Cost estimators across the Government are continuously looking for ways to improve their methods of labor rate formulation... | 2011 | Models and Methods |
| Building CERs and SERs for Enterprise Resource Planning (ERP) | Dr. Wilson Rosa, Travis Packard, James Bilbro, Max Hodal, Brian Kolstad | AFCAA is pleased to present an abstract for an Enterprise Resource Planning (ERP) paper. We will present the results of several years of data collection and study of ERPs, with the intention of providing and improving ERP cost metrics | 2011 | Models and Methods |
| Introduction to the Operating and Support Cost Analysis Model (OSCAM) | Mary Mertz, Geoffrey Pawlowski | The Operating and Support Cost Analysis Model (OSCAM) is a family of tools developed and sponsored by the Naval Center for Cost Analysis to estimate operating and support (O&S) costs for current and future Naval weapon systems. The US OSCAM family of models includes: OSCAM Ship, OSCAM Shipboard Systems, OSCAM Air, OSCAM EFV, and OSCAM USAF. OSCAM is a system dynamics-based model, which provides a structured methodology for dealing with complex systems having many interacting components. This approach enables the user to capture the dynamic behavior of a system while allowing for a flexible design, which can easily be enhanced and expanded... | 2011 | Models and Methods |
| Joint Cost Schedule Model(JCSM) - Recent AFCAA Efforts to Assess Integrated Cost and Schedule Analysis | Antonio Rippe, Greg Hogan, Darren Elliott | The Space Division of Air Force Cost Analysis Agency (AFCAA) supports Air Force (AF) and Department of Defense (DoD) Major Space Acquisition programs by providing thorough, effective, independent cost estimates (ICEs) and conducting special studies for decision makers. Recently AFCAA has initiated a research task to assess the potential for developing a joint cost-schedule model and the usability of the model... | 2011 | Research |
| Fuel Cells - Turn Up the Heat! | Arlene Minkiewicz | The fuel cell concept was first introduced in 1893 by Sir William Grove. Fuel cells remained a conceptual curiosity until the 1960s when the US space program identified a requirement for extended life batteries for which fuel cells offered a promising solution. The current focus on green technologies has caused an increase interest in consumer uses of fuel cells for transportation, residential and commercial power supply, emergency power backup and portable power supply for consumer electronics... | 2011 | Research |
| Using Project Performance Data in Cost and Schedule Analysis | Fred Kuo, Mike Stelly, Darren Elliott | Numerous projects developed at NASA now use the Joint Confidence Level (JCL) approach to simultaneously estimate cost and schedule uncertainty associated with the project. One crucial step in JCL analysis is the development of proper uncertainty bounds. These bounds are often the opinion of subject matter experts, or are derived from cost and schedule estimating relationships. However, as a project progresses, over time the bounds around certain elements and tasks are likely to change... | 2011 | Research |
| NRO CAIG O&M WBS and Duration Guidance | Ryan Timm, Gary Kanady, Jenny Moose, Sara Wise, Lori Zondlo | The NRO CAIG has developed new guidance for O&M duration and an O&M WBS for cost estimation purposes. The goal of these efforts is to aid in standardizing the scope of O&M activities to be estimated for NRO program independent cost estimates. The new O&M WBS may be used in conjunction with the Space Acquisition WBS for total life-cycle cost estimation. The O&M WBS can be used for estimation of: -Site O&M Costs (Mission Ground Station, COOP site, Factory, etc.) - Space and Ground System O&M Tails - Individual Projects/Programs O&M Costs | 2011 | Research |
| Enhancing Cost Realism Through Risk-Driven Contracting: Designing Incentive Fees Based on Probabilistic Cost Estimates | Sean P. Dorey, Dr. Josef Oehmen, Dr. Ricardo Valerdi | Cost overruns in defense system development efforts do not seem to be caused by a lack of cost estimation tools and best practices. A number of sophisticated cost estimation guides have been published, for example by the Army, Air Force, NASA, GAO, RAND, and SSCAG. There also exist extensive recommendations by professional societies like ISPA, SCEA, SSCAG, SCAF, and EACE. In an unbiased world, applying these best practices would generate reliable and accurate cost estimates. However, the problem is that the cost estimation community tends to focus on the technical variables contributing to cost risk... | 2011 | Research |
| An Approach to Estimate the Life Cycle Cost and Effort of Project Management for Systems Centric Projects | Leone Z. Young, Dr. Jon Wade, Dr. Ricardo Valerdi, Dr. John V. Farr, Dr. Young Hoon Kwak | One of the key challenges in predicting life cycle costs (LCCs) is to develop an accurate top down method that can be used in the early phase of system s life cycle to estimate hardware, software, integration, and management costs. Models have been developed for systems engineering (SE) costing component of management, but the literature is void of project management (PM) costing methodologies. Thus, the research study proposes a framework for evaluating PM costs required for the conceptualization, design, test, and deployment of large-scale systems centric projects... | 2011 | Research |
| Cloud Nine, Are We There Yet? | Arlene Minkiewicz | In 1961 at the MIT Centennial, John McCarthy opined if computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility. the computer utility could become the basis of a new and important industry [1]. In 2006, Amazon Web Services was launched providing computing on a utility basis. Since that time the notion of cloud computing has been emerging and evolving... | 2011 | Research |
| Ideas from Measurement Theory for Quantifying Intangibles | Mitch Robinson | Measurement theory, the mathematical study of measurement scales, helps us quantify attributes consistently with: 1) the attributes' key properties"; and 2) how we intend to use the attributes. Psychologists, economists, and mathematicians originally developed measurement theory to provide a rigorous basis for quantifying intangible attributes, such as subjective probability and utility. The theory similarly has promise for quantifying intangibles in which the cost community has novel interests, such as complexity, difficulty, technology level and design heritage... | 2011 | Research |
| Joint Confidence Level Analysis and the Dynamic Integrated Cost Estimator (DICE) Model | Graham Gilmer, Colin Smith | Since 2006, NASA has been leading the way in the establishment of Joint Confidence Level (JCL) guidance and the integration of cost and schedule estimation. Recently, in response to the standardization of NASA JCL policy, Booz Allen Hamilton identified the need for and developed a comprehensive model to further enable this technique. The Dynamic Integrated Cost Estimator (DICE) model prototype is Booz Allen's initial response to NASA's emerging area of JCL analysis... | 2011 | Risk |
| Real-time Risk for the Operations Environment | John Teal | Traditional risk management and cost risk analysis is an important and evolving field, but many analysts admit that it is as much of an art as a science. Although quantitative in nature, assessing the likelihood and impact of risks is generally a series of predictive guesses made by subject matter experts. When successful, the recipients of risk reports embrace the process and make use of risk reports; in other programs the process of tracking risks is regarded as an academic exercise, which often relegates analysts to the back-office, out of sight and out of touch with real-world production and operations. Ideally, the risk management process should provide users quantitative, objective, timely and relevant risk information... | 2011 | Risk |
| Joint Confidence Level of a Parametric Software Cost and Schedule Estimate | Michael A. Ross | This paper summarizes a regression-based approach for developing, describing, and calibrating Cost and Duration Estimating Relationships (CDERs) (Ross, 2008) and then describes a simple method for determining joint and conditional probabilities of estimates based on these CDERs. The paper concludes with a practical example of developing a CDER from Software Resources Data Report (SRDR) data collected by Dr. Wilson Rosa of the U.S. Air Force Cost Analysis Agency (AFCAA) and then implementing the developed CDER with joint and conditional probability calculation in a Tecolote ACE model. | 2011 | Risk |
| The Perils of Portability: CGFs and CVs | Peter Braxton, Richard Coleman, Richard Lee, Brian Flynn, Megan Guild, Jack Smuck, Kevin Cincotta | With the provisions of the Weapon Systems Acquisition Reform Act of 2009, or WSARA, raising the importance and visibility of risk analysis, it is an opportune time to revisit how risk benchmarks based on historical data are both derived and applied. This paper will focus on two key summary risk metrics: (1) the cost growth factor, or CGF, which captures historical cost growth for individual programs and is used collectively to capture the pervasive tendency to underestimate cost; and (2) the coefficient of variation, or CV, as a measure of the uncertainty in both historical and new estimates... | 2011 | Risk |
| Testing S-Curves for Reasonableness: The NCCA S-Curve Tool | Richard Coleman, Peter Braxton, Richard Lee, Brian Flynn | The requirement of the Weapon Systems Acquisition Reform Act of 2009 to disclose in accordance with paragraph (2) the confidence level used in establishing a cost estimate for a major defense acquisition program or major automated information system program and the austerity of current Department of Defense (DoD) budgets has brought about an increased interest in risk analysis and widespread use of the S-curve (the cumulative distribution function) of cost. As this interest intensified, experience in the use of S-curves widened... | 2011 | Risk |
| Use of EVM Trends to Forecast Cost Risks | Roy Smoker | This paper develops a risk burn-down metric based on a forecasted variance at complete estimated from trend (time series) data. Using only the first 18 data points of a 43 point series, projections of the budgeted cost of work performed (BCWP) and the budget at complete (BAC) are solved simultaneously to identify the value of BCWP that equals the value of BAC at some future date in time... | 2011 | Risk |
| Understatement of Risk and Uncertainty by Subject Matter Experts | Peter Braxton, Richard Coleman | The authors' paper The Correct Use of Subject Matter Experts in Cost Risk Analysis, presented at the Naval Postgraduate School (NPS) Acquisition Research Symposium (ARS) and Department of Energy Cost Analysis Symposium (DoECAS), explored alternative methods for conflating the risk distributions of multiple subject matter experts (SMEs). This paper builds on the results therein by establishing a mathematical foundation for the conflation of triangular distributions and by presenting the results of an independent experiment on expert assessment of uncertain quantities... | 2011 | Risk |
| The Implementation of Risk Analysis in Proposal Evaluation and Cost Models | Marcus Oberholzer, Travis Winstead | Herren Associates has been performing cost analyses and proposal evaluations for many Navy clients over the past 20 years. After the receipt of a sole source production proposal, Herren Associates provides a cost and pricing evaluation to determine if the proposal is fair and reasonable. To enhance this service offering and respond to our clients' need to incorporate risk and uncertainty into its cost modeling process, we developed a solution using Monte Carlo simulation and innovative statistical methods to provide negotiation cost ranges rather than simply a point cost estimate... | 2011 | Risk |
| Cost Risk Allocation Theory and Practice | Christian Smart | Risk allocation is the assignment of risk reserves from a total project or portfolio level to individual constituent elements. For example, cost risk at the total project level is often allocated to individual work breakdown structure (WBS) elements. This is a non-trivial exercise in most instances because of issues related to the aggregation of risks, such as the fact that percentiles do not add. For example if a project is funded at a 70% confidence level then one cannot simply allocate that funding to WBS elements by assigning each its 70% confidence level estimate... | 2011 | Risk |
| Quasi-Monte Carlo Methods: Combating Complexity in Cost Risk Analysis | Blake Boswell | Cost analyses rely on probabilistic numerical methods to estimate the impact of risk and uncertainty associated with systems technical definitions and cost estimating methodologies. Such methods involve modeling risk and uncertainty as probabilistic distributions, applying iterative sampling techniques using Monte Carlo (MC) methods, and deriving risk and uncertainty adjusted statistical measures; however, the accuracy of statistical measures resulting from probabilistic analyses is directly dependent on the number of samples considered. Therefore, in probabilistic numerical methods, a trade-off exits between accuracy of results and computational complexity... | 2011 | Risk |
| Effective Use of Cost Risk Reports | Alfred Smith | Cost estimates are generated in Constant Year (CY), Then Year (TY) and something called cost risk-allocated TY dollars. Several papers have been published describing various ways to allocate risk dollars from a selected level in the Work Breakdown Structure (WBS) to ensure WBS elements sum at the percentile of interest. Cost risk-allocated results are typically used as the basis for estimating the projects budget, but, rarely used as the basis for the subsequent analysis to identify cost and uncertainty drivers in the model (cost risk reports)... | 2011 | Risk |
| Covered with Oil: Incorporating Realism in Cost Risk Analysis | Christian Smart | When Jimmy Buffett sang the words All of those tourists covered with oil in his song Margaritaville he probably never imagined that this phrase might apply to crude oil instead of suntan lotion. Both the cost and the environmental impact from the 2010 oil spill in the Gulf of Mexico were much worse than anyone had expected or could have predicted. It was, in the words of financial writer Nassim Taleb, a black swan an unexpected event with tremendous consequences. These types of events, like Hurricane Katrina in 2005, the giant tsunami in the Indian Ocean in 2004, and the financial crisis that began in 2007 are all examples of events with huge impacts that are hard to predict... | 2011 | Risk |
| Risk Based Estimating and Alternative Selection Using Value Analysis | Gregory Brink | The selection between project alternatives and corresponding plans of action is one that is fraught with many facets of uncertainty and risk. In order to enhance the reliability and quality of the information provided to decision makers, an approach of performing a Risk-Adjusted Value Analysis is suggested for project, process, and product alternative evaluation and selection. The use of this assessment technique based in Value Methodology allows for the multi-dimensional weighing of performance, cost, and duration attributes under the influences of risk and uncertainty... | 2011 | Risk |
| SEER-SEM to COCOMO II Factor Convertor | Anthony Peterson, Dr. Derrick Tate, Dr. David A. Wyrick, Dr. Ricardo Valerdi | Previously [Madachy, Boehm, 2008], Rosetta Stones have been developed for converting COCOMO II estimate inputs into corresponding SEER-SEM or True S inputs, or vice-versa. Most of those Rosetta Stone mappings between factors are one-to-one, but some are one-to-many. Often these factors use disproportionate definitions. Additionally, each model contains factors that unique to that model... | 2011 | Software |
| Estimation Challenges for 21st Century Software Systems | Barry Boehm, Bradford Clark, Ray Madachy, Wilson Rosa, Thomas Tan | Several future trends will present significant future challenges for the sizing and cost estimation of 21st century software systems. Prominent among these trends are: 1. Rapid change, emergent requirements, and evolutionary development. These trends will require more attention to cost uncertainty ranges, requirements volatility, and a new factor called the Incremental Development Productivity Decline (IDPD). ... | 2011 | Software |
| Function Point Analysis: One Size Fits All | Dan French | There are many challenges when faced with creating estimates for software development projects. However, without a doubt the single largest driver for cost, effort and duration is project size. The typical approach to sizing a software project, if it is tried at all, is to estimate software lines of code (SLOC). While this may be the most expeditious method, history has shown that this approach usually produces highly inaccurate estimates. IBM developed function points as an alternative method to bring about more consistent and accurate project sizing for use in software project estimation. This methodology, while not a perfect solution, has enjoyed great success in more than 30 years of use world-wide... | 2011 | Software |
| Is There Magic Associated with Software Benchmarks? | Donald Reifer | Benchmarks have proven to be a useful tool for determining whether or not software cost, productivity and quality estimates are realistic and make sense. When used properly, benchmarks can be a very powerful tool during negotiations because they set expectations. While many may question the numbers, benchmarks that are produced with rigor and care can pass as solid bases for comparison. However, when done haphazardly, they can be very dangerous. This talk will discuss how benchmarks are developed... | 2011 | Software |
| The Challenge of Agile Estimating | Heather Nayhouse, Christina Donadi | Agile development is one of the buzzwords of 2011. But what exactly is agile development and how is it impacting the software world? Agile development refers to a group of software development methodologies based on iterative development, where requirements and solutions evolve through collaboration between self-organizing cross-functional teams. Agile strives to set goals and plan the path of a project to a release while maintaining ability and openness to change based on feedback and new information. An added challenge to understanding and estimating is that, across the software development community, agile development may have different meanings and levels to which it is incorporated into a project... | 2011 | Software |
| Software Cost Estimation Using a Decision Graph Process: A Knowledge Engineering Approach | Sherry Stukes, Dr. John Spagnuolo | At The California Institute of Technology/Jet Propulsion Laboratory, methodologies for Flight Software (FSW) cost estimation and documentation are determined that allow for efficient concurrent and consistent analysis within a tight schedule constraint. This knowledge is structured or "engineered " to facilitate the implementation of FSW cost estimation by others who wish to serve as practitioners in the field... | 2011 | Software |
| Enterprise Resource Planning Systems: Sizing Metrics and CER Development | David Brown | Over the last 10 years, Enterprise Resource Planning (ERP) systems have seen increasing use within government agencies. One of the key components to any software cost estimate is a sizing metric. Unlike estimates for a traditional AIS, ERP estimates cannot use a traditional metric such as source lines of code (SLOC) because (1) ERPs are almost always based on commercial off the shelf (COTS) software; and (2) ERP implementation requires a significant amount of effort outside the categories of software development and software procurement. Reports, interfaces, conversions, and extensions (RICE) objects are often suggested as the sizing metric of choice for ERP... | 2011 | Software |
| Cost Factors Contributing to the Choice of Aerospace Advanced Materials | Paul Shields | The materials selection process for the most recently proposed military and civil aircraft has centred substantially around the use of advanced carbon composites versus conventional high strength aluminium alloys. The former material type has been selected for use in increasing proportion. For example, Airbus's A400M is 35% advanced composite by weight; the JSF is 40%; the Boeing 787 Dreamliner is circa 50%; the proposed Russian MS21is 45%; the A350 XWB is 52% and the Bombardier C-Series is more than 50%... | 2010 | Estimating |
| Improving ERP Estimating in the DoD | Wison Rosa, Charles Silvanic, Max Hodal, Brian Kolstad | AFCAA is pleased to present an abstract for an Enterprise Resource Planning (ERP) paper in two parts. The first part presents the results of several years of data collection and study of ERPs, with the intention of improving ERP estimating. The second part is a case study where AFCAA used this information to estimate a major ERP. AFCAA envisions this paper resulting in two presentations in the same track, one for each part... | 2010 | Estimating |
| Cost Benefit Analysis for Incorporation of Prognostics | Jerry Le May | Prognostics and Health Monitoring (PHM) technology report information about the operational condition of a system or product. This information can be used to notify the user of necessary service, warn of impending failure, order replacement parts and\or consumables, notify Police and Fire personnel of an emergency and in some cases save maintenance costs by reducing unnecessary maintenance or service. A Cost Benefit analysis provides the basic cost information for a customer to determine if the benefits of incorporating PHM into a system are worth the cost... | 2010 | Estimating |
| Estimating Issues Associated with Agile Development | Bob Hunt | New software development methodologies with combinations of old and new ideas are getting increasing public attention. These ideas all emphasized close collaboration between the programmer and business experts; face-to-face communication; frequent delivery of software units; and tight, self-organizing teams. Agile is a frequently used developmental process that follows this paradigm. As the use of Agile Development expands, the cost analyst is faced how this process affects the basic estimating fundamentals. This paper will discuss the best estimating practices that should be applied to agile programs and will recommend a set of techniques for agile estimation including... | 2010 | Estimating |
| Review of Three Small-Satellite Cost Models | Melvin Broder, Eric Mahr, Daniel Barkmeyer, Erik Burgess, Wilmer Alvarado, Samuel Toas, Gregory Hogan | The problem of estimating costs for small satellites is more vexing that it would seem. Small satellites, as described here, generally weigh less than 1,000 pounds, and are sometimes much smaller and significantly different from much of what exists now. Costing these kinds of satellites is not simply a matter of scaling down from larger systems. It requires incorporation of new data sets and consideration of new modeling methods. The challenge of addressing small?satellite development is important, considering the emerging priority for developing much smaller, modular or interactive space systems... | 2010 | Estimating |
| Applying the Army Fully Burdened Cost of Fuel Methodology to Analyses of Alternatives | David Hull | In the current conflicts in Iraq and Afghanistan, enemies are attacking United States (U.S.) military convoys that transport necessary supplies such as fuel, ammunition, food, and water in an effort to sever logistical resupply routes. Department of Defense (DoD) leadership is concerned about a having a large logistics footprint that supports U.S. forces because large convoys are attractive targets. Energy inefficiency is one of the main causes of the large logistical convoys. In an effort to eventually decrease the size of the U.S. logistics footprint, DoD leadership is putting into place procedures to better understand military fuel demands and become more fuel efficient... | 2010 | Estimating |
| Developing Far-Term Portfolio Cost Estimates for a System of Systems | Kevin Foley, Ian Cappitelli | Traditional cost estimation relies upon data such as requirements, schedules, and relevant historical data provided by a program office to develop an estimate. Significant problems arise for programs with far-term (>10 years) start dates because this data may be unavailable and program offices may not exist. Historically, many commercial and government agencies would either ignore these future costs or develop a simple Rough Order of Magnitude (ROM) estimates. Additional complexities arise when these far-term programs interact with existing near- and mid- term programs... | 2010 | Estimating |
| Lessons Learned from New Subsystem Development in Manned Space | Robert Georgi | The effort to develop new technology is complicated. We will examine the effect when a bottom up risk adjusted schedule is not used. Low technical readiness level (TRL) components carry particular risks which can play havoc with budgets and deadlines. Commercial parametric models such as Galorath's SEER are designed for estimating hardware at a TRL of at least 4, component and/or breadboard validation in laboratory environment. When engineers are challenged with realizing truly advanced technology concepts the cost risk (amount it can exceed the parametric estimate) is 110% to 400% based on limited historical evidence... | 2010 | Estimating |
| Determining the Cost of the Certification and Accreditation Process using Expert Opinion and Monte Carlo Simulation | Amanda Flynn, Belinda Nethery, Kyle Thomas, Amanda Gerstner, Brett Dickey, Christina Kanick, Peter Braxton | The Certification and Accreditation (C&A) paper will address an empirical approach to determining the total cost and schedule of completing the C&A process for a given network or system. The requirement for and policy guidance on how to conduct C&A is covered by Director of Central Intelligence Directive 6/3 (DCID 6/3) along with other DoD and Federal guidance. DCID 6/3 requires agencies to complete a rigorous process of reviews to ensure that the networks and systems are secure. The customer felt that C&A was consuming extraordinary amounts of time and effort but wanted to document and understand the real impact before pursuing changes to the process... | 2010 | Estimating |
| How to Manage a Program Effectively | Bruce Fad, Dale Shermon | This paper will describe the importance of cost estimating to Program Control. Specifically it will consider the policy for Earned Vale Management (EVM) and the need to consider the deviation from the baseline program in an EVM System (EVMS). Through the integration of a parametric model with an EVMS application, on based upon frequent re- evaluation of the baseline, it is possible to increase the confidence in the EVM metrics... | 2010 | EVM & Scheduling |
| Technical Content and Risk Based Integrated Baseline Review ? PMAG Approach | Nhung Tran, Mun Kwon | Integrated Baseline Review (IBR) is a technical review process integrated with schedule and cost baselines. Risk identification should be the focus of the IBR process which includes IBR preparation, Performance Measurement Baseline assessment, IBR execution, and management processes. Frequently, many IBRs have evolved into a box check process with an emphasis on in-plant CAM interviews and limited preparation by the government program office team. It is crucial to understand that an IBR is a process, not just an event. Properly planned and executed Integrated Baseline Reviews (IBRs) is a key element of a Program Manager's risk management approach... | 2010 | EVM & Scheduling |
| Ship Construction Estimates at Completion: A New Technique Using the Weibull Function | Shawn Rudolph | This presentation summarizes recent earned value management (EVM) research at Technomics, Inc. exploring new, improved methods for generating Estimates at Completion (EAC). Our approach is based on a significant amount of burn rate data resulting from serial ship procurement. We define burn rate as the incremental change in the cumulative value of Actual Cost of Work Performed (ACWPCum) divided by the time over which the change occurred. Though some research suggests that Rayleigh modeling of the burn rate is preferred, we have studied an alternative regression technique based on the Weibull probability density function, known as the Hybrid Weibull EAC Model... | 2010 | EVM & Scheduling |
| Integrated Critical Scheduling | Donna Pucciarello, Jared Mathey, Dustin Sexton | The Integrated Critical Scheduling (ICS) methodology is an in-depth guide to ensure client and stakeholder overall objectives are exceeded. Integrated scheduling is one of the most critical facets of project management within a developmental government program. An Integrated Master Schedule (IMS) is the foundation upon which a project's cost, schedule and risks are objectively managed. It also serves as the foundation for time-phasing of the project's resources. Thorough IMS data analysis yields accurate current and predictive indications of the health of the project as well as increases the value of complimentary products such as the Work Breakdown Structure (WBS), Integrated Master Plan (IMP) and Organizational Breakdown Structure (OBS)... | 2010 | EVM & Scheduling |
| SMC/PMAG - Control Account Manager (CAM) Notebook Evaluation | Eddie Hall, Nhung Tran, Mun Kwon | The Control Account Manager (CAM) notebook is the key document for assisting the CAM and government action officer in the integration and management of the control account. The CAM notebook enables single thread analysis of the technical scope, integrated master schedule (IMS), resource loading profile, earned value (EV) performance data, and subcontractor costs. Past and present practice tends to focus on the financial and EV performance data, schedule and program plan data. The technical scope of the control account has largely been ignored... | 2010 | EVM & Scheduling |
| Integrated Master Planning Formulation-PMAG Approach | Nancy Droz, Mun Kwon | The Integrated Master Plan (IMP) is an essential program tool that spans from the beginning of the life- cycle acquisition framework through source selection, through program execution and up to program sell- off. Though the IMP has only three or four levels, the determination of program elements, significant accomplishments(key events), accomplishment criteria and possible tasks its development provides crucial insight to successful execution of the remainder of the program's life-cycle. Furthermore, the conduct of risk assessments to include risk identification and mitigation, as well as accounting for unknowns-unknown is a key step in the development of an executable IMS... | 2010 | EVM & Scheduling |
| Programs and the Future of Project Management | Simon Dekker, Vance Kotrla | Every government agency is concerned about funding. Those able to make the wisest capital investments succeed while others fall behind in cost and schedule overruns. For contractors and suppliers, increasing globalization has resulted in widespread competition in what used to be domestic markets. Vendors and service providers now routinely bid against global companies, even for local business. In order to remain competitive, businesses are re-examining their management processes to ensure all operations demonstrably serve the larger goals of the organization... | 2010 | EVM & Scheduling |
| A Case Study in EAC Growth | Richard Coleman | The responsibility for the reporting of EACs is well established, and the methods for prediction, as well as the shortcomings in those methods are well known. This should result in good EACs and a good sense of the risk in those EACs. Occasionally, however, growth of apparently totally unexpected scope occurs. Why does this happen, with all of our experience and all of our processes? This paper will describe the growth of several EACs from the original prediction through several snapshots of what was known and what was reported... | 2010 | EVM & Scheduling |
| Developing a Milestone-A Analysis of Alternatives Cost Estimate | Chadd Sibert | Department of Defense (DoD) leadership is rigorously evaluating acquisition programs much earlier in the system's lifecycle than they have in the past due to the increased scrutiny of defense funding. This in turn greatly increases the demand for cost information to support these early programmatic decisions... | 2010 | Life Cycle Cost |
| Estimating Life-cycle Cost of West Virginia Fiber Reinforced Polymer (FRP) Bridge Decks | Sidharta Sahirman | The main objective of the research was to study the economic viability of West Virginia Fiber Reinforced Polymer (FRP) bridge decks. Life?cycle cost of those bridge decks were estimated for conducting such analysis. Three main differences that distinguish the way the life?cycle cost of FRP deck was estimated are: (1) the manufacturing cost of a FRP bridge deck was estimated using learning curve theory; (2) cost savings in support structures when FRP is chosen as opposed to the alternative bridge deck was modeled; and (3) the service life was estimated based on factor method to minimize the subjectivity of the estimates. The three case studies for West Virginia FRP deck projects show that based on the estimated life?cycle cost, FRP decks are financially viable under certain conditions. | 2010 | Life Cycle Cost |
| A Cost Estimating Framework for Materials Obsolescence in Product-Service Systems | Francisco Romero, Rajkumar Roy, Essam Shehab, Kalyan Cheruvu | Obsolescence happens when a component becomes no longer available from stock of own spares or being procurable or produced by its supplier or manufacturer. This is an issue that will definitely affect long-life support systems. The rapid change in technology during the last few decades has exacerbated this problem, especially for EEE (electronics, electromechanical and electrical) components, as their life-cycle is becoming increasingly shorter. This is the reason why most of the research on obsolescence carried out so far has been focused on dealing with EEE components obsolescence... | 2010 | Life Cycle Cost |
| Developing an Efficient and Repeatable Life Cycle Cost Estimation Process within an IPT Framework by Leveraging Parametric Tools and Constant Client Communication | Zachary Jasnoff, Kevin McKeel | One of the major challenges that federal IT program offices encounter is to develop accurate and defendable life cycle cost estimates, given shrinking budgets and limited availability of key resources. Additionally, the program offices frequently have neither embedded cost estimation resources nor tools required to develop Life Cycle Cost Estimates (LCCEs). Working in a collaborative environment, PRICE Systems, Alon Inc, and Logapps have developed a repeatable process to deliver high-quality LCCEs that can stand up to GAO and OMB scrutiny... | 2010 | Life Cycle Cost |
| Life Cycle Cost Estimation of System of Systems | Kishore Gagrani, Manmeet Grover, Yousuf Mohammad | Mark Maier's System of Systems definition [Maier 1998] dis-integrates systems from SoS. In that it calls each individual system as Constituent System (CS) which is independently developed, and managed. But new CS are often added into SoS in different time frame, with many COTS products and these CS can dynamically come and go... | 2010 | Life Cycle Cost |
| Use of Life Cycle Cost Estimates in OMB-300 Budget Reporting | David Brown, Kevin Cincotta | Major capital asset investments within the federal government are required to submit program budget estimates as part of the OMB-300 Business Case. In cases where the program has already completed an estimate of life cycle cost (LCC) within another document (such as an Analysis of Alternatives or Economic Analysis), it is often desired to use the LCC as the basis for the initial OMB-300 budget submission. However, the process of conversion from an LCC to an OMB-300 budget involves potentially complex adjustments for inflation, scope, and reporting format... | 2010 | Life Cycle Cost |
| Innovative Procurement Approach for Satellite Constellations on Institutional Market | Herve Joumier | Procurement of satellite constellations on Institutional market has been limited to GPS constellation for a long time. The 1990's saw the emergence of commercial constellation such as Iridium or Globalstar. Due to the magnitude of the investment, building up such kind of projects has never been easy. All these projects went through bumpy rides with more or less success in meeting their objectives when they actually survived... | 2010 | Management |
| Business Modeling: Cost Analysis In The Operations Environment | John Teal, Andy Beck | Traditionally, cost estimators find themselves estimating elements of major acquisitions, whereas operational environments rarely require the same level of cost support. However, operational programs are under the same pressures to reduce costs and manage limited resources efficiently. This paper explores the application of cost and quantitative analysis in the operations environment, and illustrates practical examples of cost-related analytic toolsets. Specifically, this paper will describe environments apt for business modeling, propose metrics related to cost analysis that usefully measure operational performance, and illustrate tools that allow the cost analyst to perform quantitative analysis and communicate with stakeholders... | 2010 | Management |
| Transforming Estimating and Pricing Organizations to Attract and Leverage the Strengths of the New Workforce | Stanton Smith, Ian Callanan, Eva Young | The incoming workforce to replace the Baby Boom/Reagan Cold War defense buildup professionals is going to require a new organizational model to attract the professional talent to maintain effective Estimating and Pricing organizations. Northrop Grumman Electronic Systems Estimating and Pricing Organization, located near Baltimore Maryland, has piloted and implemented a new organizational and management technique model and will be sharing the concept, implementation, and results in this paper... | 2010 | Management |
| Change Management Data Collection Tool: Findings and Lessons Learned | Laura Lucas | This presentation relates the benefits and pitfalls of the online survey that the author designed and implemented. Using example questions and responses from the actual data collection tool, the author relates findings and lessons learned. Topics include: how the data collection tool strengthens future cost estimates, where the data fail to inform estimate inputs, and why the author recommends certain design choices... | 2010 | Management |
| Who Moved My Milestone? Cost and Risk Implications of Selected Changes in the New DOD 5000 Defense Acquisition Framework | Peter Hantos, Nancy Kern | As a significant milestone in the DOD's continuous self-assessment process, an important document, the Defense Acquisition Performance Assessment (DAPA) report, was released in early 2006. The report " in its sweeping and integrated assessment " attempted to consider all critical aspects of defense acquisition, and made recommendations for each of the major elements of the defense acquisition system. On the basis of the panel's recommendations, in December 2008 three major changes were instituted in the Technology Development Phase of the DOD 5000.02 Defense Acquisition Framework... | 2010 | Management |
| "Dan Galorath on Estimating" | Dan Galorath | Dan Galorath for over 25 years has been a leader in the estimating industry, and finally has begun to blog about his experiences and what he's still learning. This incredibly wide?ranging presentation will bring to life some of the high points of his BLOG. Dan will discuss metrics, effectiveness measurement, the real costs of a variety of undertakings, what is ITIL versus ISO 9000, the risk of IT projects, software project costing challenges and solutions, and lots more. He weighs in on a wealth of topics both timeless and contemporary. | 2010 | Management |
| Service Cost Estimation with Uncertainty Using Agent Based Simulation | John Erkoyuncu, Rajkumar Roy, Essam Shehab, Kalyan Cheruvu | This paper aims to contribute to the decision making in integrating uncertainty to service cost estimation by providing a novel approach through agent based modelling (ABM) for the early stage of bidding for Contracting for Availability. The paper presents the ABM architecture describing the agents that represent the customer and industry, which captures responsibilities in areas including failure, repair, supply and availability management. Furthermore, internal and external rules are defined to capture the interaction between the cost drivers. The major outcome relates to the robust and accurate what-if scenarios of the dynamic interplay among the many cost uncertain elements... | 2010 | Management |
| The 1989 NASA Space Exploration Initiative Cost Risk Estimate and Related Issues | Edwin Dean | This paper examines cost related issues that arose during and after performing the 1989 cost risk estimate for returning to the Moon and going on to Mars (The Space Exploration Initiative (SEI)). It will describe how the cost risk estimate was performed and issues that arose from that process. It will also examine cost related issues that arose from follow on personal research driven by performing that estimate. The observations and perspective contained in this paper are personal and may differ from the perspective of the SEI project and NASA. | 2010 | Management |
| Federal Budget Process: A Quick Primer | Michael Brozyna | Most people have little understanding of the Federal appropriation process. Yet, it has an enormous impact on the cost estimating community. This briefing is a light-hearted romp through the process in an attempt to explain some of the relationships and terms used. It may also help explain why your children and grand children owe tens of thousands of dollars each to foreign creditors. It won't explain why you think you pay too much in taxes. There will be no equations used in this presentation. | 2010 | Management |
| Risk Based BOE Analysis " PMAG Approach | Imran Ahmed, Mun Kwon, David Wang | The Government Accountability Office (GAO) Cost Guide is a compilation of best practices from across the Federal Government that can serve as a benchmark for cost estimation. The Guide provides useful information to assist the acquisition community in the development of reliable, comprehensive, defensible, and well- documented cost estimates... | 2010 | Management |
| Contract Negotiation in a Multi?objective Scenario using the Concept of Dynamic Shared Negotiation? Spaces and Soft vs. Hard Affordability Limits | Anil Ray, Jorn Mehnen, Paul Shields | In today's business environment, a shift in the competitiveness paradigm Business that Manufacture is becoming essential. It covers the whole bandwidth from designing for performance to designing for affordability and finally delivering a cost effective product with added value that the Buyer is willing to pay for. Making the decision as to whether it is most cost effective to procure or to manufacture for reasons of affordability requires analysis of alternatives to achieve a convincing compromised solution that satisfies Seller and Buyer. Historical data verification or tacit experiences of Seller and Buyer tend to be the norm but often decision makers miss the choice of the best possible solution... | 2010 | Methods |
| Using Fourier Analysis in Evaluating Program Office Estimates | Angela Lemke, Matthew Latham, Stephen Cox | Fourier analysis encompasses both the decomposition of a function into simpler pieces, often called analysis, and the corresponding operation of rebuilding the function from those simpler pieces, referred to as synthesis. While often used as a tool in analyzing harmonic functions, Fourier analysis methods can also be used to investigate and determine contributions of various components to a more complex output function... | 2010 | Methods |
| Study on developing CERs with Insufficient Data in Korean R&D environment | Sung Jin Kang, Yong Bok Lee, Dong Kyu Kim | Parametric cost estimating models have been used widely to obtain appropriate cost estimates in the weapon acquisition process. A parametric cost estimating model is composed of some Cost Estimating Relationships(CERs). Generally, CERs are regression models which have been based on cost and technical data. In order to obtain a good cost estimate from CERs, it's important that gathering the sufficient historical data and selecting a proper method for statistical satisfaction under restricted condition is essential... | 2010 | Methods |
| Simple Mean, Weighted Mean, or Geometric Mean? | Shu-Ping Hu | There are three commonly used methods to calculate an average (i.e., mean): the simple average, weighted average, and geometric average. Analysts often use averages as estimating guidance when predicting nonrecurring (NR) costs using ratios of NR costs to first unit (T1) production costs. For example, when the average of the NR to T1 ratios is determined, the NR cost can be estimated as a factor of its T1 cost. Here, the simple average is simply the arithmetic mean of these ratios while the weighted average is derived by weighting each ratio by its T1 cost. Consequently, deciding which average (i.e., factor) is the most appropriate metric for estimating the nonrecurring cost is frequently debated... | 2010 | Methods |
| The Business Case for Bootstrapping: When You're Stuck with Incomplete Data, Here's How You Make it Work! | Brett Gelso, Glenn Grossman, Eric Druker | The purpose of inferential statistics is to reliably extend sample characteristics to a population. Mild assumptions such as the Central Limit Theorem and Identically and Independently Drawn (IID) estimators ensure that, based on a given level of precision, samples will be representative of the population from which it was drawn. However, when samples are small, most standard test statistics will be insignificant, because sample characteristics may fail to adequately capture variation in the population. Parametric Bootstrapping is an alternate approach which resamples the underlying distribution of the sample in order to estimate population characteristics. The objective of this presentation is to illustrate how to use Parametric Bootstrapping for Cost Estimating with small samples to enhance defensibility of cost estimates... | 2010 | Methods |
| Improving CER building: Getting rid of the R2 tyranny - Building a CER with the Median | Pierre Foussier | The purpose of the author is always to get the maximum " for prediction purposes " of the data which are available to the estimator (assuming their reliability has been already checked). This paper is a contribution to this effort. We will start by a classification of the cost estimating techniques, showing that parametric " of which a clear definition will be given " is just a technique among others (which should deserve more consideration). Nevertheless this paper deals with parametric studies, because it is still the most used technique... | 2010 | Methods |
| Testing for the Significance of Cost Drivers Using Bootstrap Sampling | Daniel Feldman | Most regression-based statistical modeling aims to derive, on the basis of historical data, an algebraic expression for estimating a quantity of interest, such as cost. Such an algebraic expression is characterized by one or more fit parameters, which are typically coefficients or exponents. Once these fit parameters are determined, it is a logical next step to test for their accuracy by a process called statistical inference. The fit parameters of a cost-estimating relationship (CER) are each directly linked to a candidate cost driver... | 2010 | Methods |
| Neural Network Cost Estimating Relationships | Edwin Dean | Regression, in various forms, is the tool of choice for parametric cost analysis (PCA). Neural networks are a largely unused analysis tool within the cost community. However, neural networks are an effective alternative to all forms of regression for PCA. This paper will illustrate the use of neural networks for PCA and will highlight similarities and differences with respect to the use of regression for PCA. For example, regression provides a linear or transformed linear approximation of parameters for a given set of data. Neural networks provide a purely nonlinear approximation of the parameters for the data... | 2010 | Methods |
| New Research in General Error Regression Model (GERM) Significance Testing | Kevin Cincotta, Andrew Busick | A limited but significant literature exists with respect to significance testing in General Error Regression Models (GERM). GERM refers to cost estimating relationship (CER) development in the absence of specific functional form assumptions and distributional assumptions about the error term. It is sometimes called a distribution-free method of CER development. Within the cost estimating community, GERM was first popularized by Book and Young (1997). More recently, Anderson (2008) proposed A Distribution-Free Measure of the Significance of CER Regression Fit Parameters Established Using General Error Regression Methods, which won best paper in its track at the 2008 SCEA Conference... | 2010 | Methods |
| Detecting Anomalous Cost Data in an Integrated Data Warehouse | Lawrence Brown | Data quality is always a key issue when using historical data to predict future costs. Often a small data error in the historical data can lead to much more significant errors in predicted costs. The problem becomes more acute if data is integrated from several sources, the sources systems have evolved, the source systems were not designed to support analysis, or extensive business rules are required to allocate or assign attributes to the data. This paper discusses systematic techniques, used by the Air Force Total Ownership Cost (AFTOC) program, to find anomalous data... | 2010 | Methods |
| Return on Investment Metrics: An Army Perspective | Noel Bishop | Cost analysts have several Return on Investment (ROI) metrics (e.g., Benefit to Investment Ratio (BIR), Net Present Value (NPV, and Break Even) in their tool box when conducting an economic or comparison of benefit analysis. All of these metrics have their utility, however one metric may be more appropriate than another depending on who the decision maker is and the type of project... | 2010 | Methods |
| Using the Joint Integrated Analysis Tool (JIAT) | Daniel Schwartz, Melissa Cyrulik | The Joint Integrated Analysis Tool (JIAT) is administered by the Office of the Deputy Assistant Secretary of the Army " Cost & Economics (ODASA-CE) and Tecolote Research, Inc. JIAT is a web-based application that provides seamless linkages between cost estimating tools, engineering design models, modeling and simulation tools, as well as capability, performance and operations and support databases. JIAT incorporates several cost estimating industry standard tools and databases which allow end-users to run and query these resources from their desktops... | 2010 | Models |
| Advancing the Art of Technology Cost Estimating- a Collaboration between NASA and Boeing | Mark Schankman, John Reynolds | Advancing the art of estimating the cost of developing technologies is a long term need for both government and industry. This paper describes a collaborative effort by Systems Engineers at Boeing and NASA resulting in cost estimating tools and analysis techniques that assist in evaluating the development cost of aerospace technologies... | 2010 | Models |
| Simulation-based Experimentation: How to Scope and Estimate Development Projects | Karen Mourikas, Denise Nelson | Simulation-based Experimentation consists of designing, developing, and executing an experiment using computer simulations representing real systems, with or without operators-in- the-loop. Estimating simulation-based experimentation differs from estimating an acquisition program which includes distinct development, production, and operations and support phases. The end result of simulation-based experimentation is an experimentation environment and results of analysis to address both internal and external customer concerns. To improve estimation of the cost of a simulation-based experimentation project, we developed an experimentation cost estimating methodology and toolkit... | 2010 | Models |
| Adapting Bottoms-up Cost Estimating Relationships to New Systems | Dale Masel, Robert Judd, John D. Dowler | For several years, Ohio University has been engaged in research to develop improved cost estimating procedures for jet engine components. The resulting methodology uses bottoms-up cost estimating relationships to estimate the cost for each of the manufacturing processes needed to create the features on the part, but these CERs are implemented in a parametric framework. This means that most of the user-entered attributes are high-level descriptions of part geometry and these attributes are used to map appropriate values to the attributes in the feature-level CERs... | 2010 | Models |
| Standardizing Space Systems O&M Cost Estimating | Robert Koury, John Long, Pete Stanley | This presentation addresses the importance of standardized Operations and Maintenance (O&M) cost estimation within the space cost community. Currently there are at least nine active O&M cost estimating models in use by just one branch of the United States Government. Although there is a relatively large number of models there do not appear to be a comprehensive view of O&M for space systems... | 2010 | Models |
| Cost Benefit Optimisation to Achieve Affordable Force Structures | Zachary Jasnoff, Dale Shermon | This paper will describe the application of parametric cost models to the problem of selecting the optimum Force Structure when limited defence budgets impose considerable restrictions on the shape and magnitude of the armed forces. The use of Cost Benefit analysis by means of a Combine Operational Effectiveness and Investment Appraisal (COEIA) has been a methodology used to select alternative Force Structures in many procurement agencies. It is now possible to optimise this process through the linkage of parametric models and optimisation tools such as Phoenix Integration's Model Center. | 2010 | Models |
| Logistics Requirements Funding Summary (LRFS) Cost Estimating Tool (CET): A Quick Cost Estimator for Logisticians | Charles Gu, John Ko, Jeremy Eden, Zach Pryor, Brandon Cain | Similar to Life Cycle Cost Estimate (LCCE), the Logistics Requirements Funding Summary (LRFS) captures Integrated Logistics Support (ILS) related costs for a program throughout its life cycle. The LRFS provides visibility into the logistics requirements of a program which is needed during Program Objectives Memorandum (POM) and budget submissions. Additionally, it helps to plan and quantify requirements, identify and defend funding, and serves as the ILS input to the LCCE... | 2010 | Models |
| GDOT Planning Level Cost Estimation Review Study | Jeff Carroll, Dave Cox | Cost escalation has become a major concern in virtually every field of capital project development. On top of regular inflation, the volatility of the global oil market is tied directly to most construction related items (primarily asphalt), thereby causing shifts in cost by the rising and falling oil prices. Within the transportation sector, cost escalation has attracted attention at the federal, state, regional, and local government levels for highways, transit, and other modes... | 2010 | Models |
| Time Phasing Cost Estimates in MS Excel Models | William Black, Matt Reiley, Jeffrey Jaekle | One of the most critical, and often difficult, elements of developing a cost model is the determination of how to phase costs across the life of an estimate. Agency Cost Positions (ACPs) and Independent Cost Estimates (ICEs) are often conducted early in a program's lifecycle, before detailed technical specifications, let alone schedules, have been developed. Even in those cases where a decent schedule does exist, issues still arise concerning the best way to translate technical and programmatic milestones into executable cost/budget profiles. This task proves even more daunting as external factors, such as the needs of other programs, are introduced into the estimating and budgeting process... | 2010 | Models |
| Here, There Be Dragons: Considering the Right Tail in Risk Management | Christian Smart | The portfolio effect is the reduction of risk achieved by funding multiple projects that are not perfectly correlated with one another. It is relied upon in setting confidence level policy for programs that consist of multiple projects. The idea of a portfolio effect has its roots in modern finance as pioneered by Nobel Memorial Prize winner Harry Markowitz. However, in three recent ISPA-SCEA conference presentations, The Portfolio Reconsidered in 2007, The Fractal Geometry of Cost Risk in 2008, and The Portfolio Effect and the Free Lunch in 2009, the author has demonstrated that the portfolio effect is more myth than fact... | 2010 | Risk |
| Constructing Bounds for S-Curves | Christopher Mehl, Michael Popp | After years of developing point estimates analysts knew were uncertain, the estimating and acquisition community has embraced the concept of viewing a cost estimate as a potential distribution of cost represented by a Cumulative Distribution Function (CDF), commonly called the S?curve. Unfortunately the very thing the S?curve was intended to counter "the implication of preciseness inherent in a point estimate"has become a preciseness about percentile values and levels of risk. Decision makers, for example, have come to view the 50th percentile as an absolute and wonder why the number budgeted at 50% is shown as 30% when the estimate is updated... | 2010 | Risk |
| PMAG Risk Based Integrated Program Management | Mun Kwon | Introduce the risk based integrated program management framework describing major tenets behind the framework to enhance the space acquisition programs development success. The major tenets of the framework will be discussed are 1) program risk management and the decision based program risk tracking, 2) managing program management assurance 3) life cycle based system integration, 4) application oriented training. This risk based integrated program management framework is designed to primarily focus to coach and mentor the program managers and project managers through the hands on OJT to produce risk based products and services to strengthen the organic integrated program management capabilities. Key findings and lessons learned of space acquisition programs will be addressed during the briefing... | 2010 | Risk |
| Build Your Own Distribution Finder | Alfred Smith | A basic step in a cost uncertainty analysis is to define the distribution of every uncertain element in the cost model. Identifying and then defending these distributions is a fundamental challenge. If data is available, the preference is always to perform a statistical analysis to arrive at an objective assessment. Several commercial tools are available for finding the distribution that best describes the shape and dispersion of the sample data set, but they are not in agreement.. Are they all correct? Have these different methods been subjected to an independent validation and verification (IV&V)? Is there a best method to derive the best fit?... | 2010 | Risk |
| Risk Identification and Visualization in a Concurrent Engineering Team Environment | Jairus Hihn, Debarati Chattopadhyay, Robert Shishko | JPL is NASA's prime center for deep space missions. In response to the need to reduce the cost and time to complete early concept studies and proposals JPL created the first concurrent engineering team in the aerospace industry: Team X. Started in 1995, Team X has carried out over 800 studies, dramatically reducing the time and cost involved, and has been the model for other concurrent engineering teams both within NASA and throughout the larger aerospace community. The success of Team X has also spawned new related teams such as Team I for instrument studies and in the past year the Rapid Mission Architecture Team for trade studies... | 2010 | Risk |
| Modeling of Inflation Uncertainty in Cost-Risk Analysis | James Glenn, Robert Meyer, Paul Hardin | Completion of a thorough cost-risk analysis is a vital part of a credible cost estimate. The GAO Cost Estimating and Assessment Guide identifies seven steps which should be completed in order to complete a valid cost-risk analysis. These steps include: placement of probability distributions around WBS elements or inputs to WBS elements to capture various types of uncertainty, accounting for correlation between inputs and interpreting results from a Monte Carlo simulation model... | 2010 | Risk |
| Budget-Constrained Joint Confidence Level Approach | Peter Frederic | As long as we have tried to understand and quantify the uncertainty associated with our cost estimates, we have understood that cost risk and schedule risk are highly interrelated. The current emphasis on Joint Confidence Level (JCL) analysis at NASA has focused a great deal of attention on this relationship... | 2010 | Risk |
| JCL in a Nutshell: Exploring the Math of Joint Cost & Schedule Risk Analysis Through Illustrative Examples | Eric Druker | With agencies such as NASA now requiring joint cost and schedule risk analysis, it is important to take time out to understand the mathematics underlying both and how it relates to their integration. Rather than focusing on difficult-to-penetrate equations, this presentation will examine the math behind Joint Cost & Schedule (JCL) risk analysis using illustrative examples. These examples will demonstrate that the combination of cost & schedule risk analysis can lead to counterintuitive results. Following these demonstrations, several recommendations will be made to the community to encourage discussion and feedback regarding this emergent methodology... | 2010 | Risk |
| Understanding Risk in the Budgeting Process from a Portfolio Point of View | Ryan Boulais, Brett Dickey, Matt Reiley | The Office of the Director of National Intelligence (ODNI) Cost Analysis Improvement Group (CAIG) is responsible for conducting independent cost estimates (ICE) for Intelligence Community (IC) acquisitions at or above the major system acquisition (MSA) threshold of $170M. IC agencies are required, by law, to budget to the ODNI CAIG ICE or endorsed Agency Cost Position (ACP) for all acquisitions above a $500M threshold... | 2010 | Risk |
| Capturing Risk Interdependencies: The CONVOI Method | Blake Boswell, Mike Manchisi | Traditionally, risks impacting cost and/or schedules are often modeled as independent. However, it is often the case that the occurrence of one risk influences the probability of another occurring. For this reason, identifying and accounting for relationships between risks is essential to producing accurate cost estimates. Accounting for risk interdependence is often neglected by conventional risk analysis programs because risks are usually generated independently by technical experts focused on a single part of the system making it difficult to formulate and quantify relationships among risks... | 2010 | Risk |
| AFCAA Cost Risk and Uncertainty Analysis Metrics Manual | Wilson Rosa, Alfred Smith, Jeff McDowell, Lew Fichter | The Cost Risk and Uncertainty Handbook (CRUH), published by the Air Force Cost Analysis Agency (AFCAA) in July 2007, is a reference for approved methods, practices, and reporting requirements needed to produce a realistic, defendable cost uncertainty analysis. CRUH addresses the question how to do it. CRUAMM provides guidance on what to base it on - that is, it provides data and data-driven uncertainty distributions for potential use in your cost model or for use as cross-checks. This paper will present a status on development of this supplement to the AFCAA CRUH... | 2010 | Risk |
| More Trouble with Estimating at the 80th Percentile | Tim Anderson | This presentation addresses the recommendation within the Weapon System Acquisition Reform Act (WSARA) of 2009 that defense acquisition program cost estimates be reported at the 80th percentile of their cost probability distributions. The goal of this recommendation is commendable. Defense acquisition program costs have typically been under-estimated. Thus, it is believed that planning for cost at the 80th percentile will reduce the number of defense programs being placed in the unenviable situation of discovering that cost growth has caused the program budget to be insufficient... | 2010 | Risk |
| Risk-Based Return On Sales (ROS) As a Tool For Complex Contract Negotiations | Peter Braxton, Richard Coleman, Michael Burton | With the requirement in the Weapons System Acquisition Reform Act of 2009 to estimate at the eightieth percentile, both government and contractor organizations need to understand the impact on the acquisition process, in particular on complex contract negotiations. This paper will show how a risk-based approach to negotiations, simultaneously considering the government's paramount concern of contract price and the contractor's paramount concern of return on sales (ROS, or margin), can lead to a more mutually beneficial contracting arrangement. This paper builds on the analytical foundations of Risk-Based Return On Sales (ROS) for Proposals with Mitigating Terms and Conditions (SCEA, 2009)... | 2010 | Risk |
| Server Virtualization and Government Organizations | Jennifer Woolley, Sandra Williams, Ryan Boulais | Server virtualization, which can allow a single server to replace multiple servers, is rapidly becoming commonplace. As the relatively new technology has matured, various government agencies have begun to make the transition to operating in a virtual environment in an effort to cut costs and streamline data access. With virtualization becoming more prevalent, new data is available to assess the cost impacts associated with this hardware baseline shift... | 2010 | Software & IT |
| SOFTWARE SCHEDULE ESTIMATING LINKED TO COST ESTIMATES | Sarah Springer, Daniel Feldman, Greg Hogan | Recently there has been growing emphasis within the cost community on schedule estimating, and exploring the link between cost and schedule for Department of Defense (DoD) programs. As software development has become an increasingly important part of acquisition for the DoD, the need for better software cost and schedule models has grown. The intrinsic link between cost and schedule allows the acquisition community to adequately time phase resources to ensure you meet predicted schedules. Although a number of commercial models exist that will perform these functions, visibility into the data behind Cost Estimating Relationships (CERs) and Schedule Estimating Relationships (SERs) is preferred in order to build a comprehensive program model... | 2010 | Software & IT |
| Moving Desktops to the Cloud - The costs and benefits of Virtual Desktop Infrastructure solutions | Ryan Timm, Leon Halstead, Michael Richards, Justin Smith | Cloud computing is revolutionizing IT server operations and organizations are beginning to embrace the cloud computing paradigm for desktop computing. Cloud computing refers to users receiving IT services over the LAN, WAN, or internet from a datacenter with dynamically scalable virtualized resources. Two approaches to serving desktops from the cloud are Virtual Desktop Infrastructure (VDI) and Terminal Services-based remote desktops. These desktop replacement solutions move desktop processing and storage to the datacenter and require a client at the user's desk which transmits keyboard and mouse inputs to the datacenter and receives display information... | 2010 | Software & IT |
| The Economics of Cloud Computing: Addressing the Benefits of Infrastructure in the Cloud | Ann Repczynski, Ted Alford, Gwen Morton | Cloud computing is a new approach to providing IT services at a greatly reduced cost relative to current distributed IT environments. The potential savings are expected to be significant... so much so that the Office of Management and Budget is embracing cloud computing as the preferred federal IT environment. Why is this necessary? First, the federal IT budget for 2010 has grown to nearly $76 billion, and is expected to reach $88 billion for 2011. The federal government cannot maintain this spending trajectory. Secondly, the proportion of spend required simply to maintain IT infrastructure"at least $20 billion, or more than 25 percent of the 2010 budget"crowds out opportunities to invest in new initiatives and technologies... | 2010 | Software & IT |
| Software Maintenance: Debunking the Myths | Donald Reifer, Jill Ann Allen, Brian Fersch, Barbara Hitchings, James Judy, Wilson Rosa | This paper provides a snapshot of the findings, conclusions and recommendations of a year-long Joint U.S. Army and Air Force study on the topic of software maintenance. Using facts gathered during fact-finding, the paper focuses attention on dispelling myths about maintenance. While focused primarily on weapons systems, the conclusions offered seem generally applicable to any system, either military or commercial. The ultimate goal of our recommendations is to improve practices used for software maintenance in the field. | 2010 | Software & IT |
| Comparative Analysis of Software Maintenance Modeling in the COCOMO II, SEER, SLIM and True S Cost Models | Donald Reifer, Jill Ann Allen, Brian Fersch, Barbara Hitchings, James Judy, Wilson Rosa | This paper provides a snapshot of the findings, conclusions and recommendations a year-long Joint U.S. Army and Air Force study on the topic of software life cycle maintenance and cost modeling. The paper starts by defining the work that government life cycle support centers typically perform as part of their normal jobs using a Work Breakdown Structure (WBS). Next, the paper rates how well the popular COCOMO II, SEER, SLIM and True S cost models estimate each of the work tasks in this WBS. Gaps uncovered are then highlighted and rules of thumb for filling them are offered as the paper concludes with recommendations for providing fuller coverage for these important operations, maintenance and support activities. | 2010 | Software & IT |
| Improving Software Cost Estimates Using the Univariate Model | Brian Opaska | Despite its drawbacks, the univariate (first-order) model for estimating software remains a widely used technique throughout the government and private sector. This presentation will provide three techniques that can be applied to improve the consistency and accuracy of this method. These techniques involve using a reliable code counting sizing metric, performing thorough analysis of the actual SLOC count, and normalizing the SLOC... | 2010 | Software & IT |
| Function Point Analysis: Introduction and Basic Overview as an Alternative to SLOC-based Estimation | Tucker Moore | When analyzing and estimating the costs for a software application, most cost estimators inherently rely on problematic, SLOC-centric estimation techniques. Cost analysts typically strive for concrete, tangible numbers to work with in order to produce relationships and parametric trends in historical and future cost data... | 2010 | Software & IT |
| Estimating S O A, As Easy as 1 2 3 | Arlene Minkiewicz | Metcalf's Law tells us that the value of a telecommunications network is proportional to the square of the number of users of the system [1]. Service-Oriented Architectures (SOAs) capitalize on this phenomenon. Through a set of standard interfaces, services (software based capabilities) are made available to any consumer willing to follow the rules for interface and consumption. The loose coupling provided by the standard interfaces enables this plug and play capability. Taking advantage of such a notion promises great gains in efficiency for anyone looking to create interoperable, scalable applications that share information across boundaries... | 2010 | Software & IT |
| Costs of Achieving Software Technology Readiness | Arlene Minkiewicz | Technology is advancing with lightning speed. Many of the complex systems required by the Department of Defense (DoD) and NASA require technological innovations to achieve sophisticated mission and state of the art accomplishments. Technology readiness assessments for complex systems and subsystems are important for both the government and its contractors to ensure proposed technologies meet program requirements. Without good technology assessments, programs may be funded that have little or no chance of success because success assumes use of technologies not yet confirmed to be realistic. The defense acquisition community recognizes this and has developed detailed guidelines for Technology Readiness Assessments (TRAs)... | 2010 | Software & IT |
| Software Cost Estimating Relationships | Jennifer Leotta | Software cost overruns are a common problem for the majority of software development projects. With the ever increasing amount of software present in current Department of Defense (DOD) programs, it is extremely important to generate an accurate software cost estimates. There are many complex models that estimate software development productivity and costs. This paper builds upon the principles of these models to look for a simple regression model that can be used to generate accurate and defendable cost estimates for software development programs. | 2010 | Software & IT |
| Parametrics as a Function of Project Management: A Tactical Example | Krista Stroh | Over the past few years as budgets have become tighter, government agencies have been trying to do more science and research with less resources. As a result, the National Aeronautics and Space Administration (NASA)'s International Space Station ( ISS) Program employs cost estimation as a function of project management in order to make better decisions. The cost estimators and project controls professionals, through the use of parametric tools, have been able to execute effective project controls for various customers, including the program manager, project engineers, and the procurement office... | 2009 | Applications/Tools |
| Integrating Homegrown CERs in a Single Commercial Estimating Toolset | Anthony A. DeMarco | Most organizations use a combination of commercial models, public models and proprietary CERS to develop confidence in their estimates. Homegrown CERs in the form of simple cost factors, equations, curves, and rules of thumb build confidence in estimates if they are derived from known, reliable data. However, most organizations have difficulty hosting, sharing and integrating their CERs with other estimating models. Homegrown CERs are commonly hosted in arcane Excel spreadsheets, and are not easily understood, audited or reused... | 2009 | Applications/Tools |
| True Cost Benefit Analysis Framework | Peter Pizzutillo | This paper introduces the innovations implemented in TruePlanning by PRICE Systems. Building on the success of the industry's first true System of Systems Estimating Framework, PRICE Systems has enhanced this framework to include cost benefits analysis capabilities. Previously, organizations needed a collection of tools to create cost/benefit models; estimate system cost and benefits, and evaluate or rate the assessment of these alternatives. The paper shows how, in a single application, heterogeneous systems can be integrated, costs can be derived parametrically or bottoms up, benefits are estimated and payback and assessment criteria are evaluated... | 2009 | Applications/Tools |
| Migrating Calibrated PRICE Hardware Project Files to TruePlanning for Hardware | James E. Otte | This paper is intended to provide enough detail for an experienced PRICE Hardware client to understand the primary features of the True Hardware model. This paper will include a discussion of why PRICE developed a new architecture and Hardware estimating model As well as a brief explanation of enhanced capabilities of TruePlanning for Hardware, A comparison of PRICE H and TruePlanning for Hardware will be closely examined., Explain the underlying modeling methods and assumptions of TruePlanning for Hardware | 2009 | Applications/Tools |
| Estimating the Cost of a Depot Level Mobile Maintenance Capability | Angela Lemke | Today, the US Army is exploring options for an aviation sustainment level maintenance capability that can effectively cover initial drop in readiness. The Mobile Maintenance Capability concept provides key capabilities to sustain expeditionary Army aviation forces during operations in austere or immature theaters. In order to support early program documentation and budgetary submission requirements, an initial life cycle cost estimate of a mobile maintenance capability was performed. Since, numerous alternatives have been proposed. This paper describes the assumptions, methodology, and results of the life cycle cost estimate for one of those options. The estimate was prepared using current Army guidance for an ACAT III program. | 2009 | Applications/Tools |
| Contemporary Alternatives to Classical Risk Analysis Challenges | Chris Hutchings | Risks intrude on every aspect of our daily routines, whether it be through investing, playing a sport, weather forecasting, or just simple activities such as eating or driving; we are constantly in harms way. In Project Management there are aspects of risk in several key areas, including the development of requirements, the rapid growth in technology and the instability of the operational environment, all of which ultimately adversely effect the three tangible measurements of effectiveness for any project, Performance, Schedule and Cost. | 2009 | Applications/Tools |
| SOURCING STRATEGY BEST PRACTICES - BRINGING SPEND ANALYSIS TO THE DOD | Christopher Milo, Patrick W. Baranowsky II | DoD reliance on commercial outsourcing expanded dramatically over the past several years, growing 78 percent in the decade ending FY 2006.1 As outsourcing has grown, so too has appreciation for the potential of cost growth due to lax oversight of acquisition activities. DoD has issued a new mandate through DoD Instruction 5000.2, directing designated managers throughout all components to implement effective strategic sourcing procedures. These procedures explicitly include execution of periodic spend analyses... | 2009 | Applications/Tools |
| Through-Life Estimating (TLE) " the need to estimate through program life | Arlene Minkiewicz, Dale Shermon | The objective of this paper is to introduce the concept of using one estimating tool to estimate hardware, software, services and IT programs consistently from cradle to grave using appropriate estimating methodologies. This paper will define the term Through-Life Estimating and describe the challenges of estimating an aerospace or defence program over the typically long gestation period of 20 to 30 years " estimating Through-Life. | 2009 | Applications/Tools |
| Introduction to the Joint Integrated Analysis Tool (JIAT) | Daniel L. Schwartz, Melissa Cyrulik | A major challenge for U.S. Army cost analysts is bring together current engineering design, capability, and performance data in their cost estimating processes and analyses. While individual models and tools address these areas, it is a tedious, manual process to identify and bring together the relevant results. The Joint Integrated Analysis Tool (JIAT) is being developed by the Office of the Deputy Assistant Secretary of the Army " Cost & Economics (ODASA-CE) to address the requirement for a multi-linked system of integrated cost estimating engineering design/capability and performance model tools... | 2009 | Applications/Tools |
| Comparative Analysis of Spacecraft Schedule by Classical and Quantum Monte Carlo Simulations | David R. Graham | A schedule risk management methodology currently accepted at NASA is based on the represen- tation of durations or costs of risky tasks by distribution functions taken from project history or suggested by experts and using Monte Carlo simulation of the project schedule or cost estimate. Though empirical, this expert opinion methodology has proven efficient in providing the basic uncertainty of project cost and duration for the current plan without further mitigation. A new method of schedule analysis calculates the correlation function of randomly delayed tasks and exhibits systematic milestone delay with fully symmetric (normal) individual task distributions... | 2009 | EVM/Schedules |
| Improving Cost Estimating during Program Execution through Integrated Program Management | Wendy Freeman | During program execution, Earned Value Management (EVM), the Integrated Master Schedule (IMS) and Technical Performance Measures are integrated with risk management in day-to-day program management. The program's cost estimate needs to take into account program performance, including contract performance, in order to estimate the cost and schedule resources necessary to complete the program and ensure sufficient budget is available. This presentation will explore an analysis tool which links the program management disciplines together (a best practice as described in the GAO Cost Assessment Guide) in order to produce a probability distribution around the cost estimate while providing actionable information to the Program Manager. The results will be demonstrated on a Major Defense Acquisition Program. | 2009 | EVM/Schedules |
| PERFORMING STATISTICAL ANALYSIS ON EARNED VALUE DATA | Eric Druker, Dan Demangos, Richard Coleman | Some Earned Value Methods, particularly those described in the equations on the DAU Gold Card, suffer from the shortcomings that they are backwards looking (they do not make a prediction of the final CPI) and do not allow for inferential or descriptive statistics. This leads to the propensity for these estimates to tail-chase as the CPI changes over time, meaning that the EAC for an over running program will systematically lag in predicting the overrun, and the EAC for an under running program will systematically lag in predicting the under run... | 2009 | EVM/Schedules |
| Is Earned Value Management (EVM) in the future of Performance-Based Services Acquisitions (PBSA)? | Anthony James Chandler | Three key objectives of the PBSA contracts are to measure, provide incentives and assist in the management of performance provided by service contractors. The dollar value the Federal Government spends on service contracts is estimated over $147B. Given the dollar value, period of performance, criticality and complexity of some service contracts, metrics are being called out to aid in contractor performance visibility... | 2009 | EVM/Schedules |
| Anchoring EVM Analysis Approaches in the GAO Guide | Patrick K. Barker, Mary Kate Lenz, Ammar Chaudry | The growing size and complexity of Defense programs and a higher level of awareness of uncertainty places an ever-increasing emphasis on the integration of risk, earned value, cost, technical performance and schedule in the Acquisition community and creates the need for new communication tools for program managers. The US Department of Defense and related Federal organizations have updated several guidance documents and policies to encourage this integration, the latest evidence being the release of the GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs This landmark guide places a significant amount of space dedicated to the role played by Earned Value Management (EVM), as well as other key program management disciplines... | 2009 | EVM/Schedules |
| Integrating Systems Engineering with Earned Value Management | Patrick K. Barker | This seminar explores the inter-relationship of system engineering (SE) and earned value management (EVM). Often, project control specialists see the planning of the cost and schedule baselines as an isolated activity disassociated from the technical or engineering aspects. This type of thinking can significantly hamstring the development of the earned value baseline and likewise jeopardize the ability to control project scope. This lesson explains basic SE terms and concepts and shows similarities between SE and EVM. Hands-on requirements generation and WBS building exercises in this seminar help students recognize the value of linking the two disciplines together... | 2009 | EVM/Schedules |
| Incorporating Financial Sustainability Considerations into the Planning and Design of New Facilities in the National Parks | Fred Richardson, Matthew Tetreault | The backlog of maintenance in the National Park System is reaching crisis proportions. The total deferred maintenance (DM) backlog was estimated at $8 to $10 billion in 2008. This problem has worsened in recent years despite a concerted effort to combat it. The National Park Service (NPS) cannot hope to limit and reduce deferred maintenance without significant increases to its budgets for the Repair & Rehabilitation and Line Item Construction programs, which combined provide less than $100 million annually to address DM. Faced with this seemingly intractable problem, park planners should heed the maxim, First, do no harm. ... | 2009 | Management |
| Leadership AND Management: Two Necessary Talents | David Peeler | Recent scholarship and over a decade of management down-sizing have emphasized the value of leadership over management. This prioritization has produced profound detrimental effects on many organizations. Most of the consequences are not fully realized by senior leadership, but are felt by so many flattened organizations' employees as chaos, disorder, confusion and/or turmoil within the workplace... | 2009 | Management |
| The Lockheed Martin Aeronautics Transformed Estimating System | Cleo Liles | Taking advantage of company initiatives in the areas of technical planning and earned value management, Lockheed Martin Aeronautics has instituted an initiative to re- engineer the company's estimating processes and systems. This presentation will describe the needs that drive this action, the plan for this transformation, and the benefits that will result from its implementation... | 2009 | Management |
| PMAG's Program Startup Assistance | Colonel Mun Kwon | The Program Management Assistance Group (PMAG), Space and Missile Systems Center (SMC), Los Angeles AFB, is enhancing program baseline executability. PMAG is establishing robust integrated baselines for our space acquisition programs by formulating integrated program risk analyses. As a result of PMAG's accomplishments with the programs it supports, it is the desire of Senior Air Force leadership that the PMAG methodology be spread throughout the USAF Acquisition community... | 2009 | Management |
| The Challenge of Estimating Space System Costs...Modeling the Relationship with Acquisition Strategy | Paul S. Killingsworth | In recent years, a considerable number studies and blue-ribbon reports have highlighted the problems associated with the acquisition of military space systems. These problems are manifested by the fact that almost all major space programs under development by both the DoD and Intelligence Community have encountered (or are currently encountering) breaches of their cost, schedule, or technical baselines. The author hypothesizes that understanding the impact of a program cost estimate and its associated risk within the context of the complex program acquisition process may help us better understand the sources of cost growth... | 2009 | Management |
| Market Mapping for Production Optimization | Douglas K. Howarth | This paper describes methods for product optimization using market mapping. It is possible to optimize products by examining market conditions as over time and comparing them with evolving cost structures. Market mapping facilitates this analysis. Products have important attributes associated with them, which includes their demand curves, prices and costs. Market demand maps reveal market boundaries and underserved price and quantity regions that producers can seek in order to improve product viability through reduced competition. | 2009 | Management |
| NASA Productivity Study | Joe Hamaker, Tom Coonce, Bob Bitten, Henry Hertzfeld | This study was performed at the request of the NASA Administrator and examined NASA productivity in terms of economic efficiency. The analysis examined historical trends of NASA bang for the buck metrics over time, and compared NASA productivity to other organizations including the Air Force, the NRO, ESA and commercial space taking into the account the relative complexity of the missions. Past studies with a similar focus were reviewed and their results summarized. Finally, recommendations for improving NASA productivity were solicited from industry, government and academia and are documented in the study. The results of the study were briefed to the Administrator in late December 2008 and a journal quality paper is being prepared for submission to the Journal of Cost Analysis and Parametrics. | 2009 | Management |
| Dan Galorath on Estimating | Dan Galorath | Dan Galorath for over 25 years has been a leader in the estimating industry, and finally has begun to blog about his experiences and what he's still learning. This incredibly wide?ranging presentation will bring to life some of the high points of his BLOG. Dan will discuss metrics, effectiveness measurement, the real costs of a variety of undertakings, what is ITIL versus ISO 9000, the risk of IT projects, software project costing challenges and solutions, and lots more. He weighs in on a wealth of topics both timeless and contemporary. | 2009 | Management |
| An Approach to Corroborating the Impact of Recruit Quality and Recruiting Mission on Resource Requirements | Dr. Robert Clemence, Jeremy Heusner, Robert Love, Raissa Nourieva, Meredith Sachs | The Army Deputy Chief of Staff for Personnel, G-1 manages people and money to meet the Army's needs for soldiers and civilian professionals. As the sponsor for all Army manpower programs and appropriations, the G-1 reviews and validates programs and funding requests, distributes people and dollars among competing requirements according to Army policy and priorities, and oversees the execution of approved programs. Manpower costs are the Army's largest expense. As the Army increases its authorized strength, the rising costs of its recruiting and retention programs are under scrutiny by the Army leadership, the Office of the Secretary of Defense, and by the Congress... | 2009 | Management |
| Federal Appropriation Process? A Quick Primer | Michael Brozyna | After numerous conversations with private industry cost estimators, I realized that most people have no idea how the Federal Govt appropriation process works. You have to understand how your biggest customer works. This briefing explains some of the basic concepts and shows how they relate to our business environment. The budget cycle, appropriation law, important terms and earmarks will all be covered. Go ahead; study the equation adjustments that may change the estimate outcome by a few million dollars at an Earned Value Management or Learning Curve Theory lecture. Then come see how that estimate fits into the big picture. This session will be a light hearted effort to understand the mind? numbing process that now works in the trillions of dollars. Bring your sense of humor. | 2009 | Management |
| Cost Analysts/Estimators...the Necessary Professionals | Dr. John C. Bredfeldt | Most cost analysts and estimators came to the profession through contact with the Department of Defense either as a contractor, or as a government employee. Yet, cost analysts/estimators have unique skills that could greatly help the decision-makers in a great number of private and public arenas. This briefing will identify a number of skill sets that cost analysts should have and then identify how those skill sets could be applied for decision purposes in the both the public and private worlds. Some micro-economic concepts will be used to show how the cost analyst/estimator should consult in the private world... | 2009 | Management |
| Capabilities-Based Cost Estimating: Capabilities Knowledge Base (CKB) Analysis Tools and Applications | Chadd Sibert | Cost analysts in today's environment often find themselves being asked to analyze solutions identified to fill capability gaps that are poorly defined. Often, only a requirement or desired capability set is known. Is this sufficient information to produce a cost estimate? Are there any tools available to support this effort? If so, how would you utilize them?... | 2009 | Methods |
| Systems Cost Engineering - Program Affordability Management and Cost Control | Dale Shermon | Cost estimating for major engineering hardware, software, service and IT projects is critical for ensuring a realistic evaluation of the project or program, to assess its affordability, and for managing the ongoing costs realistically. Traditional cost engineering methods are time consuming, expensive and occasionally irrational. Organizations are increasingly turning to parametrics as a means of managing cost and program affordability... | 2009 | Methods |
| Methods and Challenges in Early Cost Estimating | David Hull | Department of Defense (DoD) leadership is currently making decisions on acquisition programs much earlier in the system's lifecycle than they have in the past, and leadership demand for cost information to support these decisions is growing rapidly. Since its inception two years ago, the ODASA-CE Early Cost Team's primary mission has been to find ways to ensure that the cost information to support these early decisions is available and reliable... | 2009 | Methods |
| A Refresher On Engineering Economics | Joe Hamaker | Abstract: This PowerPoint Briefing is intended as refresher training or introductory training to the discipline of engineering economics as practiced by cost analysts. Basic concepts such as the purpose and intent of engineering economics, the time value of money, the effects of inflation, the selection of a discount rate, net present value, future value, annual equivalents amounts and internal rate of return will be clearly explained in a jargon free way. The basic set up of engineering economic problems will be covered as well as tricks and tips, common errors and how to avoid them. Realistic examples will be used that cost estimators might normally encounter. | 2009 | Methods |
| Abstract: Cost Estimation as a Linear Programming Problem | Kevin Cincotta, Andrew Busick | We tend to think of cost estimating as a non-linear, non-integer problem in which the quantities of items we will buy are known, but the costs of each item are unknown. But what happens when the reverse is true? In other words, what happens when every item procured is commercial off the shelf (COTS), with a known unit price... | 2009 | Methods |
| Cost Estimating:Back to Basics | Dr. John C. Bredfeldt | We, as cost estimators, are often criticized for inaccurate estimates of new weapon systems, construction projects, or program concepts. We frequently comment that a cost estimate is only as good as the definition of that being estimated and the risks inherent in the effort being estimated. However, how often does the estimator request the program content information in such a manner or in sufficiency to develop the most credible estimate possible. I propose to remind estimators of several micro-economic principles involved in estimating, which oftentimes are inviolate due to the nature of their being... | 2009 | Methods |
| Weapon System Design Trade Offs: Using Life Cycle Cost (LCC) | F. Quentin Redman, George L. Stratton, Andrew T. Crepea | Cost Performance Trade-Off analysis is fundamental to the Systems Engineering process. A cost performance trade study, is a procedural search for a design solution that balances achieved systems performance, effectiveness and cost against the desired or required values for these features and the system. Life cycle cost (LCC) has become increasingly important as systems are designed with longer useful technological lives. Within this paper we will discuss the methodology and rational behind the methodology for using life cycle cost estimates within these weapon system design trade-offs... | 2009 | Methods |
| ENTERPRISE WIDE COST MODELING: A SYSTEMS ENGINEERING PERSPECTIVE | Yousuf S. Mohammed, Dr. Han P. Bao | In the present research the problem of Enterprise wide cost modeling is approached from a systems engineering standpoint. What this does is to use each stage of product life cycle to obtain useful information that helps in estimating the cost of the system. Once a generic framework is developed for estimating the core cost, layers of other factors that affect the cost are applied to the core cost such as risk and uncertainty, maintainability, supply-chain and socio-economic conditions... | 2009 | Methods |
| The System Capability Architecture: Enabling Capability-Based Cost Analysis | Katherine E. McCormack, Martha A. Roper | The current Department of Defense (DoD) cost environment requires early investment decisions. In order to improve acquisition outcomes, these early decisions require cost analysis at a time when limited program information is available. The limited program information can sometimes be just a list of desired capabilities... | 2009 | Methods |
| Simple Parametric Model for Estimating Development (RDT&E) Cost for Large- Scale Systems | Paul Hardin, Robert Jones, Anna Irvine | This paper summarizes OSD sponsored research regarding the development and implementation of a simple approach for estimating RDT&E cost. The approach is based on the systems engineering analysis data that is resident in system architecture products. The approach uses authoritative data that is normally available for ACAT I, II, and III programs, i.e., Information Support Plans (ISP), Department of Defense Architecture Framework (DoDAF) views, Selected Acquisition Reports (SARs), and Budget Exhibits... | 2009 | Models |
| A Method for Estimating the Development and Production Cost of Advanced Radar Systems at the Subsystem and Component Level | David Stamm, Greg Bell, Paul Grim | Budget constraints and high development and production costs have combined to limit the deployment of sensor systems needed for ballistic missile defense and other important military applications. To reduce the cost of new radar systems, designs must be based on reuse of components and subsystems. Similarly, in-service radars are being modified and upgraded to extend system performance and enhance supportability and sustainability. This shift in emphasis has created a need for a procedure to estimate radar modification costs and radar development and production programs where components and subsystems are reused in the early stages of the planning and budgeting cycle... | 2009 | Models |
| An Analytical Framework for Cost Estimating Relationship Development | Christian Smart, George Culver | Cost estimating relationship (CER) development, particularly for spacecraft hardware, has traditionally been method-centric. Much attention has been paid to parameter calculation and goodness-of-fit statistics for particular methods. However, parametric modeling involves other critical factors that are often overlooked, including model validation. These factors are discussed, and a complete analytical framework for developing CERs is presented. Three popular methods for spacecraft hardware CER development are compared in the context of this framework, namely Minimum Unbiased Percent Error (MUPE), Minimum Percent Error with Zero Percent Bias Constraint (MPE- ZPB), and log-transformed Ordinary Least Squares (LOLS)... | 2009 | Models |
| Experimentation Estimating Toolkit | Karen Mourikas, Denise Nelson | Experimentation is a discovery process that evaluates benefits of new technologies or concepts by simulating operations and assessing the outcomes. This process provides a deeper understanding of issues surrounding the problem area, as well as more insight into cause and effect. Experimentation efforts can range from relatively small-scale efforts to highly complex events involving large numbers of participants. In order to estimate the cost of an experimentation effort, the scope of the experiment must be well understood... | 2009 | Models |
| Contractor Costs for Life Cycle Cost Estimating | Lew Messing, Samuel Hendrix | The purpose of this study is to analyze contractor costs and develop cost estimating relationships (CERs) in order to increase the fidelity and defendability of costs associated with contractors as part of life-cycle cost and program office cost estimates. At this time we are not aware of any developed methodology to estimate contractor costs for various position types or seniority levels and believe that this type of information could significantly impact the development of cost estimates throughout the DoD and IC... | 2009 | Models |
| The Lifecycle Integration Framework -- Extending Affordability Simulation through Cost and Engineering Model Interoperability | Zachary Jasnoff | For affordability targets to remain effective across all program phases, cost estimating models must be interoperable with engineering tools to avoid a "disconnect" between each phase of the lifecycle and affordability objectives. Initiatives such as Cost as An Independent Variable (CAIV) are enhanced through interoperability with engineering design tools giving engineers, project and business managers insight into achieving affordability goals... | 2009 | Models |
| Comparing Different Methods for Deriving Cost-Dependent CERs | Dr. Shu-Ping Hu | Parametric cost estimating relationships (CER) are developed using historical data. Regression analysis is used to determine whether the independent variables can help explain the variation in the dependent variable. There are two different types of independent variables in CERs: technical parameters (e.g., weight, power, etc.) and cost-dependent parameters (e.g., first unit production cost, Prime Mission Product [PMP] cost, etc.). When CERs are generated, they are based upon actual data from completed projects whether or not the independent variables are cost- dependent or hardware design-driven... | 2009 | Models |
| Excessive Project Cost " Where It Comes From: A Quantum Perspective | Ilya M. Fishman, David Graham | It is well known that project costs and durations often exceed their planned values. In the accepted practice, this phenomenon is described by asymmetric distribution functions for project tasks or WBS elements. To make predictions reasonable, Low, Most Likely and High values of these distributions are skewed to higher cost and longer durations, with distribution Means exceeding the Most Likely values. Currently, this empirical expert opinion methodology satisfies mounting needs for project cost and duration guidelines... | 2009 | Models |
| Manpower cost Estimating Tool for Enhanced Online Reporting (METEOR) | Thomas Demas, Anupam Kumar | Acquisition of major weapons systems requires an assessment of the program operating and support (O&S) costs. The largest O&S cost contributor is in the area of Manpower, Personnel and Training. To improve the quality of Total Ownership Cost estimates, the Navy recently released a new manpower estimating database/model to help cost analyst capture both the direct and indirect costs associated with Manpower, Personnel and Training... | 2009 | Models |
| Parametric Cost Analysis Using Neural Networks | Edwin B. Dean | Regression, in various forms, is the tool of choice for parametric cost analysis (PCA). Neural networks are a largely unused analysis tool within the cost community. However, neural networks are an effective alternative to all forms of regression for PCA. This paper will illustrate the use of neural networks for PCA and will highlight similarities and differences with respect to the use of regression for PCA. For example, regression provides a linear or transformed linear approximation of parameters for a given set of data... | 2009 | Models |
| Statistical Foundations of Adaptive Cost-Estimating Relationships | Stephen A. Book, Melvin Broder, Daniel Feldman | Traditional development of cost-estimating relationships (CERs) has been based on full data sets consisting of all available cost and technical data associated with a particular class of products of interest, e.g., components, subsystems or entire systems of satellites, ground systems, etc. In this paper, we review an extension of the concept of analogy estimating to parametric estimating, namely the concept of adaptive CERs " CERs that are based on specific knowledge of individual data points that may be more relevant to a particular estimating problem than would the full data set... | 2009 | Models |
| A Distribution?Free Measure of the Significance of CER Regression Fit Parameters Established Using General Error Regression Methods | Timothy P. Anderson | General Error Regression Methods (GERM) have earned a strong following in the cost estimating community as a means of establishing cost estimating relationships (CERs) using non?linear functional forms. GERM has given rise to a wide variety of functional forms for CERs, but has so far lacked a means for evaluating the significance of the individual regression fit parameters in a way that is analogous to the roles played by the t?statistic in ordinary least squares (OLS) regression... | 2009 | Models |
| QUANTIFYING THE IMPACT OF PROPOSED RISK MITIGATION STRATEGIES: DOES THE COST TO MITIGATE EXCEED THE COST OF THE IMPACT? | Philip E. Beenhouwer | The risk management process involves a concise, well defined framework for identification, analysis, and mitigation of identified program risks. In order to effectively manage risks, handling options must be established and mitigation actions defined. Typically, mitigation actions are identified and acted upon without consideration of the cost to implement these actions. Decisions of whether to assume, control, transfer, avoid, and watch the risk are likely taken with little regard to their potential effect on the risk. It is not uncommon for the cost to mitigate the risk to exceed the cost impact of the risk itself... | 2009 | Risk |
| Risk-Based Return On Sales (ROS) for Proposals with Mitigating Terms and Conditions | Peter Braxton, Richard Coleman, Eric Druker, Bethia Cullis, Christina Kanick, Ameya Bapat, Jeanne Callahan, Bryan Caccavale | This paper addresses the unique challenges faced by both cost and risk analysts and decision makers when attempting to compute and convey the percentiles of cost and return on sales (ROS) for proposals with terms and conditions providing for equitable adjustments for certain components of cost growth deemed to be outside the contractor's control, such as inflation... | 2009 | Risk |
| DON'T LET THE FINANCIAL CRISIS HAPPEN TO YOU: WHY ESTIMATES USING POWER CERS ARE LIKELY TO EXPERIENCE COST GROWTH | Eric Druker, Richard Coleman, Peter Braxton | Kurtosis risk is the risk embodied in wider (fat-tailed) distributions. This is well known in the finance industry since the failure to account for it led to the collapse of the Long-Term Capital Management hedge fund in the late 1990s. Kurtosis risk occurs when the probability of extreme values is underestimated in a risk model. The cost analysis community has recently begun to acknowledge this risk; fat-tailed distributions for cost risk analysis have been proposed in various papers presented over the last few years... | 2009 | Risk |
| Use of Weibull Failure Rates | Virginia Stouffer | Often one of the largest costs of an operational phase, estimating reparable costs remains more an art than science. A number of different probability distributions are commonly assumed to underlie failure rates for predicting future reparable requirements. In the case where a limited amount of data is available on demonstrated failure rates, the Weibull can be a versatile and useful estimating tool. This paper demonstrates how observed failure rate data from a time-limited test can be adapted to a Weibull distribution for predicting future failure rates. ... | 2009 | Risk |
| Predicting Reliability | Virginia Stouffer | One of the biggest causes of operational cost overruns is the failure of delivered systems to meet required reliability targets. Often early fielding to achieve a desired defense function precedes government testing on reliability; in those cases manufacturer estimates of reliability are accepted as truth. This paper describes how manufacturer reliability estimates were evaluated, using early fielding data. The manufacturer used a software failure rate estimating package and adjusted for field derating, thermal conditioning, and operating environments. This paper describes how government evaluators used failure mode analysis and Chi-Square testing on available data to evaluate the contractor methods and in some cases, determine an alternate reliability estimate. | 2009 | Risk |
| The Portfolio Effect And The Free Lunch | Christian Smart | The portfolio effect is the reduction of risk achieved by funding multiple projects that are not perfectly correlated with one another. It is relied upon in setting confidence level policy for programs that consist of multiple projects. The idea of a portfolio effect has its roots in modern finance as pioneered by Nobel Memorial Prize winner Harry Markowitz. However, in two prior ISPA-SCEA conference presentations, The Portfolio Reconsidered in 2007 and The Fractal Geometry of Cost Risk in 2008, the author has demonstrated that the portfolio effect is more myth than fact. Additional cost growth data have been collected for an updated study... | 2009 | Risk |
| Risk, Uncertainty, and Trouble: Escaping the RUT of Program Instability | Colonel Brian Shimel | When we say we are stuck in a rut, we imply our often repeated actions are forced on us by things outside our control. As if to say, we are not really as crazy as we look. But what if we build our own rut and then act as if we have no choice? Risk and uncertainty are not what get us into trouble. We get into trouble when we ignore, or unwisely discount, risk and uncertainty. I call that Risk, Uncertainty, and Trouble...and it is a RUT of our own making... | 2009 | Risk |
| Techniques for Assessing Cost/Risk in Source Selections | Thomas Sanders, Ken Kennedy, Kristina Golden | Arguably the most important decision ever made in an acquisition weapon system program's life cycle occurs prior to any contract award. That decision recognizes a potential contractor's understanding of the cost, schedule, and technical risk inherent in his approach to satisfying government requirements as stated in a Request for Proposal (RFP). Although it cannot be validated, many acquisition professionals agree that the selection of the best contractor has a great deal to do with the success of a program "all the money in the world may not necessarily yield a successful program... | 2009 | Risk |
| A Simple, Universal Approach to Cost Risk Analysis: Integrating Simulations and Sensitivities | Eric Hawkes, Eric Druker | Within the last decade, probabilistic cost estimating techniques have increased in popularity as a quantitative method to bound the estimate and as a way to guard against optimism in the planning process. This paper begins with an objective look at the current techniques available to cost analysts. It then presents a new approach titled, Enhanced Sensitivity Analysis (ESA). ESA applies Response Surface Methodology concepts to the cost estimating environment. After walking through a case study using ESA, the paper then compares it with the input-based simulation approach... | 2009 | Risk |
| Why Not Do Project Risk Analysis Parametrically? | Daniel Galorath, Evin Stump | The dominant approach to quantitative project risk analysis is troublesome for several reasons: Subjective probability distributions assigned to WBS elements, Subjective correlations assigned between WBS elements, Limitation of risks to those associated with direct charge work, Ease of overlooking important risks, No feedback from project experience or lessons learned... | 2009 | Risk |
| MAKING STATISTICAL ANALYSIS ACCESSIBLE: THE RAMS TOOL FOR PERFORMING REGRESSION AND RISK ANALYSIS | Matt Lytton, Eric Druker, Gregory Hogan | The tools for evaluating the uncertainty around cost and schedule estimating relationships (CERs) using statistical techniques remain under utilized by the cost estimating profession today. Three primary estimating techniques, the standard error of the estimate (SEE), SME judgment, and various "industry guidelines" are frequently used to assess uncertainty. These techniques can introduce bias into the estimate and lead to uncertainty being misestimated by, in some cases, orders of magnitude. The false security of high-confidence level budgets developed using these techniques can have a dramatic effect on a program's chance of success... | 2009 | Risk |
| What Percentile Are We At Now (And Where Are We Going?) | Richard Coleman, Eric Druker, Peter Braxton, Bethia Cullis, Christina Kanick | For decades nearly all cost estimators have held to the premise that their estimates were at the 50th percentile, and most risk analysts have built their work on the presumption that this is so. This paper will show that it is unlikely that this key premise was ever true, and will explore the ramifications of the premise being false. Among other implications, this issue bears very strongly on current initiatives by many contractors (and agencies) to allow for cost growth in their proposals (and budgets.)... | 2009 | Risk |
| Compensating for "Lack of Tails" on Triangular Distributions | Melvin Broder | The use of triangular distributions for capturing cost risk has been popular because they have exceptional attributes for the collection of expert information on the uncertainties of cost. However triangles have one major drawback, the absolute limitation of risk at the extremes, i.e. the Low and High values have zero percent probability of exceedence. Conversely the normal or Gaussian distribution has infinite tails but information collection, although requiring only two parameters (mean and sigma), is often difficult to elicit from experts... | 2009 | Risk |
| Initial Results Building a Normalized Software Database using SRDRS | Michael Gallo, Paul Hardin, Elizabeth Koza, Robert Bailey | The Software Resource Data Report provides DoD cost analysts with a primary source of comprehensive, objective, and model-independent sizing, effort and schedule data on large DoD software programs. The SRDR does not rely on rigid, standardized data fields for collecting software effort and sizing information (a feature advocated by Government, industry, and academia during the design of the SRDR). This flexibility lessens the reporting burden placed on industry and reduces "data synthesis" errors... | 2009 | Software & IT |
| Organizing IT Project Information | Alan Williams | This paper describes a methodology for estimating IT projects. The span of knowledge required of an IT project estimator is much greater than just being able to characterize the software development. Having an understanding of the elements that comprise the organization infrastructure, deployment concepts, and security issues as well as the complex interactions within the environment are essential to preparing the estimate. The estimation methodology described here is composed of two parts: a formalized estimation structure, and a series of catalogues or mini-estimates that provide defaults to use as starting points for all critical estimation elements | 2009 | Software & IT |
| Optimized Project Management of Systems and Software Projects | Denton Tarbet | Successful software projects demonstrate good project management methods incorporating modern processes and effective utilization of metrics. This paper demonstrates the application of control theory methods to project management decisions and incorporates optimization of systems trade-offs to maximize project performance. High fidelity parametric estimation models used as the Basis of Estimate for the project also provide the decision tools for a Fully Managed Project... | 2009 | Software & IT |
| Cost Impacts of Server Virtualization | Leah Roberts, Jennifer Woolley, Alisha Soles, Will Black | In today's often budget strained economy, corporations and government agencies are constantly looking for ways to cut costs, streamline data access, and improve security. As a result, newer and more efficient ways of operating, such as the use of virtualized environments, are becoming more prevalent across the IT community. Server virtualization in particular, which can allow a single server to replace multiple servers, is rapidly becoming commonplace... | 2009 | Software & IT |
| Productivity Decline in Directed System of Systems Software Development | Ramin Moazeni, A. Winsor Brown, Barry Boehm | The emerging concept of Directed System of Systems (DSOS) describes the large-scale integration of many independent, self-contained systems in order to deliver unique capabilities. From Future Combat Systems to Fractionated Space Systems, these complex multi-systems are very interdependent to achieve DSOS's objectives. The synthesis of these very large systems often results in different sets of problems than those presented by the design of a single, but complex, system usually addressed by engineers... | 2009 | Software & IT |
| A Holistic Approach to Understanding IT Costs | Arlene Minkiewicz | When we talk to Information Technology (IT) leaders about better defining, selecting and managing their IT projects, they are quick to inform us that the projects represent less than a third of their budgets. Most of the IT budget goes toward paying existing people and support existing datacenters. What they really need is the means to perform an activity based analysis of the Total Cost of Ownership (TCO) for their IT operations. With this, organizations will have the right information to determine cost savings and additional costs associated with new projects and emerging technologies... | 2009 | Software & IT |
| Making the Case for SOA | Arlene Minkiewicz | A Service Oriented Architecture (SOA) is a computing environment in which applications are composed, rather than developed, through a set of standard interfaces. A SOA takes advantage of networking capabilities to compose applications through the coupling of services in a way that is independent of architecture, programming language, development platform and vendor... | 2009 | Software & IT |
| Use Case Estimating 2009 | Bob Hunt | Software projects are notorious for being completed behind schedule and over budget. One primary cause is the inability to accurately define project size, which ultimately leads to unreasonable effort and schedule estimates. Many project managers use software modeling tools, such as SEER-SEM, COCOMO II, and custom in-house tools for developing estimates; however, the accuracy of project effort and schedule estimates, from any modeling tool, is directly proportional to the accuracy of estimated project size. This presentation discusses the advantages and disadvantages of the Use Case concept as a requirements development tool. And, then presents a process for translating Use Case Points into size metrics, effort months, cost and schedule. | 2009 | Software & IT |
| Illustrative Example of Flight Software Estimation | Robert Georgi | The effort to effectively model the development of flight software for manned space is complicated because of unique approaches used by NASA. Data from previous efforts are frequently poorly documented. Also the infrequency of flight software development activities for new manned vehicles means that techniques from previous projects are not applicable to the present day due to the maturation of software languages. This presentation will explain the challenges faced in developing a parametric model-based software estimate and how these areas were addressed... | 2009 | Software & IT |
| IMPROVING SOFTWARE COST AND SCHEDULE ESTIMATING WITHIN THE FAA | William S. Barfield | The Federal Aviation Administration (FAA) initiated a data call and analysis to improve the estimation of software cost and schedule life cycles for new programs. The Air Traffic Operations (ATO) Investment Planning and Analysis (IP&A) group has taken the lead in this important effort. Reverse modeling is a novel approach to calibrate and validate program-specific software cost estimates by using cost estimating best practices and the SEER-SEM software cost estimating tool. This technique produces a data base of sufficient size to support the development of Cost Estimating Relationships (CERs)... | 2009 | Software & IT |
| CUMAV or Unit? Is Cum Average vs. Unit Theory a Fair Fight? | Bethia Cullis, Richard Coleman, Peter Braxton, Jacquelyn McQueston | The statistics of the regressions used to judge significance and best fit are inherently biased towards Cum Average because the Cum Average metric is smoothed by the very fact that it is cumulative. This paper will use generated data with and without error terms in an attempt to show the degree of this bias and will seek to generate some thumb rules or some insight to help the analyst decide between the two models... | 2008 | Analysis |
| NASA New Start Inflation Index | Brian Welsh, Kenneth Odom | Inflation is an important factor for cost estimating and budgeting. The Office of Management and Budget (OMB) publishes annual inflation guidance for all government funded programs. In certain government programs actual inflation is higher than OMB guidance. This is especially true for New Start National Aeronautics and Space Administration (NASA) programs. These programs involve complex new design work requiring a specialized work force, limited production quantities (typically one), and limited opportunities for commercial applications... | 2008 | Analysis |
| Parametric Cost-Estimation of an Assembly using Component-Level Cost Estimates | Dale Masel, Robert Judd | There are two approaches that can be taken for estimating the manufacturing cost of a product assembly. One approach is to separately estimate the cost of each of the individual components from the work breakdown structure (WBS) and summing these component costs to determine the total cost. Another approach is to derive a parametric model that uses information about the overall assembly to produce a cost estimate... | 2008 | Analysis |
| Taking the Next Step: Turning OLS CER-Based Estimates into Risk Distributions | Christina Kanick, Richard Coleman, Eric Druker, Peter Braxton, Matthew Cain | There has been an increased focused on uncertainty and risk in cost estimates as customers now routinely desire a range of potential costs, rather than a point estimate,. Most analysts know that when performing regression analysis the uncertainty of the estimate can be derived from the analysis of variance (ANOVA) statistics; however, only rarely is this uncertainty modeled in the risk analysis. This paper will touch briefly on the concepts behind the prediction interval, and then will delve into a deeper discussion of how prediction intervals can be converted into uncertainty distributions for estimates generated from regression-based cost estimating relationships (CERs)... | 2008 | Analysis |
| Multiplicative-Form CERs: A Progress Review | Mitch Robinson, Eric Mosier | Exponential and power functions, y = a?bc?x and y = a?x , have significant currency as CER functional forms; and the ready accessibility of the Excel Solver has put a tool for fitting them to data in everyone's hands. However, compared to their linear cousins, the mathematics of these multiplicative CERs is less easy and knowledge of their fundamentals is less widespread. It is thus a good time to improve our understanding of them, to ground future development and application firmly. We speak here to our initial efforts... | 2008 | Analysis |
| New Challenges for Cost Analysts: Creating Technical Baselines for Point Estimates | Samuel Toas, Joseph Frisbie, Duncan Thomas | As cost analysts are integrated into product teams earlier in program lifecycles, we are required to complete estimates and provide budget inputs well before the typical parameters of a program technical baseline are understood. Many times, these conceptual designs (either contractor designs or reference architectures) are based on immature technology rather than historically proven design solutions. This paper details how a satellite working technical baseline can be constructed using a parametric model that utilizes customer requirements, target performance, and cost objectives... | 2008 | Analysis |
| Estimating Support Labor for a Production Program | Jeffrey Platten | Common methods for estimating support labor are; Percent to Touch Labor, Fixed / Variable, Semi-Variable, and Improvement Curves. The problem with improvement curves is that rate variation may require an adjustment. The problem with the other methods is that they do not significantly address variation due to program maturity. Variation in support labor due to program maturity is usually different than the variation in touch labor due to program maturity. In other words, support labor does not follow the same improvement slope as touch labor... | 2008 | Analysis |
| Systems Engineering Affordability Tracking System | Karen Mourikas, Denise Nelson | Is your program on target? How do you define on target? How do you measure a program's progress? Systems Engineering Affordability Tracking (SEAT) can help you answer these questions. The SEAT System is a process and a tool to help produce a more affordable system, project, or program. SEAT goes beyond the current contractual obligations that traditional Earned Value Management (EVM) processes perform, by expanding the scope of the process and the targets tracked. All program phases from Early Development through the end of program are incorporated within the system... | 2008 | Analysis |
| Help or Hype? Estimating the Impact of Reverse Auction on Materials Procurement Cost | Sandra Enser | The first Navy purchase by reverse auction occurred in May, 2000. It was touted as a means of achieving significant cost savings, and initial results looked very promising. Since then, many Government agencies have adopted e-commerce as standard operating procedure, and companies have incorporated reverse auction and web-based procurement methods at varying levels of complexity. The purpose of this paper is to define the e- commerce processes, to identify which materials are most suitable for reverse auction, to assess the life cycle of any savings (one time vs. ongoing reductions), and to describe some of the benefits and pitfalls of these techniques... | 2008 | Analysis |
| Missile Defense Agency (MDA) Booster Cost Analysis | David Stamm, Greg Bell | The purpose of the presentation is to discuss the current fleet of MDA booster systems (employed as both interceptors and as target systems), common booster elements and components, basing, support systems, launch and range operations and cost estimating and analysis methods used to forecast, plan and budget their development, production, and operational costs. In addition to boosters, MDA payloads and launch elements will also be addressed. | 2008 | Analysis |
| The Percentile Problem: How Much Is Enough? | Kevin Cincotta | It is common practice for federal government agencies to require cost estimates at the 80th percentile. For example, the Air Force requires 80th percentile cost estimates as a matter of routine. Recently completed work for the Coast Guard also involved the mandated use of 80th percentile estimates to support budgeting. It is widely thought that this practice reduces risk, by making cost overruns less likely. The difference between the 80th percentile estimate and the point estimate (which may be a very large number, because the point estimate likely resides at a very low percentile of the true cost distribution) is often as defined as risk dollars.... | 2008 | Analysis |
| Pre-Milestone-A Cost Analysis: Progress, Challenges, and Change | Martha Roper | Pre-Milestone-A Cost Analysis: It's a relatively new concept in the defense community, but one very familiar to the early cost analysis research team at the Office of the Deputy Assistant Secretary of the Army for Cost and Economics (ODASA-CE). With over eighteen months logged in parallel research and application efforts, the time investment continues to produce dividends of both progress and salient lessons learned. More than ever, it is clear that early acquisition investment decisions must be cost-informed, and the demand for this early cost information is growing... | 2008 | Analysis |
| R2 vs. r2 | Shu-Ping Hu | Cost estimating relationships (CER) with multiplicative error assumptions are commonly used in cost analysis. Consequently, we need to apply appropriate statistical measures to evaluate a CER's quality when developing multiplicative error CERs such as the Minimum-Unbiased-Percentage Error (MUPE) and Minimum-Percentage Error under the Zero-Percentage Bias (ZMPE) CERs... | 2008 | Analysis |
| EVMS for Program Management: Where Does It Fit? | Simon Dekker | This workshop provides insight on Earned Value Management and its function within the broader scope of program management. Specific contributions of EVM to be discussed include the integrated management and analysis of scope, cost, schedule, and resources. Mr. Dekker also demystifies Earned Value Management concepts, and leads students to discover how EVM can be a natural byproduct of sound program management disciplines. | 2008 | Earned Value Management/Schedule |
| Utilization of EVM metrics and data in the development of Life Cycle Cost Estimates and Cost Benefit Analyses. | Anthony J. Chandler, Kenneth Thomson | This paper will help define and examine the challenges and opportunities of using EVM in the development of Life Cycle Cost Estimates (LCCE) and Cost Benefit Analyses (CBA). The purpose of the LCCE and the CBA is to ensure that programs and projects are acquired on the basis of LCC Return on Investment (ROI) or most benefit. The advent of ANSI/EIA-748 and the increased leveling of 0MB 300 requirements on programs have led EVM to the intersection ofLCCE and CBA... | 2008 | Earned Value Management/Schedule |
| Recognizing Earned Value Management (EVM) Gaming | Walt Majerowicz, Dorothy Tiffany | Why would a contractor or project manager intentionally reduce an Estimate-To-Complete or activity duration? What are the implications of your contractor keeping two sets of Earned Value Management "books," and how would you even know about it? Under what circumstances would a contractor manipulate schedule logic? What is a "rubber baseline?" These questions and others like it raise serious concerns that all project teams using Earned Value Management should recognize. This presentation examines some common and not-so-common gaming, abuse, and data manipulation techniques that some projects or contracts may employ. At best, these techniques are errors or misunderstandings on the part of the contractor, at worst they are outright fraud and could place the entire project outcome in jeopardy. | 2008 | Earned Value Management/Schedule |
| Early Warning Model for Acquisition Program Cost and Schedule Growth | Dr. Dan Davis | This paper uses the non-linear Rayleigh function to model the behavior underlying the problem-solving and engineering activities contained in research and development contracts and reflected in the cumulative cost accrual during the execution of those contracts. This paper examines 115 contract datasets. Using non-linear least squares with monotonically transformed and restricted parameters and the techniques of numerical computation, we examine the ability of the Rayleigh function to explain the variation of the actual cost data over time in each of these contracts... | 2008 | Earned Value Management/Schedule |
| Managing the People Side of Change | Phil Eastman | Over the last decades, many projects met technical requirements but failed to deliver their intended value to the organization. Employee resistance, declining productivity, slower than expected adoption of the solution, miscommunication and poor executive sponsorship presented barriers that were difficult for project managers to overcome. Change management provides the missing link " it delivers a structured approach to managing the "people side" of change introduced by your projects. Additionally, recent research has shown that the likelihood that a project meets its objectives is directly tied to the change management effectiveness of the project... | 2008 | Lessons Learned |
| Lessons Learned on Estimating Costs for Unmanned Aircraft Systems (UAS) | Charlie Kapaku, Chris Tuttle | This paper/presentation is aimed at provided lessons learned on performing cost estimates for the Predator, Reaper, and Global Hawk UAS. Paper/presentation will compare recent Mil Std 881 WBS guidance with actual estimates for the above programs and discuss unique items such as ground control stations and multiple comm links... | 2008 | Lessons Learned |
| Multicollinearity " Don't Throw The Baby (Independent Variable) Out With The Bath Water | Jim Byrd | One of the cardinal assumptions with respect to Ordinary Least Squares Regression is the assumption that the Independent Variables are independent of each other. That is to say they are not correlated with each other. Collinear independent variables do not lead to biased estimates, however, they do produce estimates with relatively large variances. This is another way of saying that the estimate may be very far from its true value... | 2008 | Lessons Learned |
| Five Questions a Project Manager Should Ask About Every Estimate | Anthony A. DeMarco | Every day, project managers make decisions on estimates. Since each decision can determine whether a project succeeds or fails, credible estimates are critical. Projects launched without a rigorous initial estimate have five times more probability of experiencing delays and cancellations. Even projects with sound initial estimates are doomed to overrun if they are not guided by management rules of thumb and formal estimates-to-complete. Therefore, project managers must be armed with simple metrics along with rigorous estimating models to be successful... | 2008 | Lessons Learned |
| Cost Overruns and Defense Contracting | Jennifer Leotta | This paper will show that incentive fee contracts in the Department of Defense (DOD) for the development phase are not effective in eliminating the cost overrun problem faced by the DOD. This will be examined first by analyzing the optimal share ratios with the Weitzman model on two specific DOD development programs, then through a comparison of various programs with extreme cost overruns and varying contract types and fee structures. Finally, reasons explaining the cost overruns will be explored. | 2008 | Lessons Learned |
| Acquisition Intelligence Cost Estimating | Tim Edem | Air Force weapon systems have become increasingly dependent on integrated intelligence information/data. Without it, some systems cannot execute their mission. Intelligence cost estimating has been a long identified need within Air Force acquisition organizations. Under the direction of HAF/A2, and HQ AFMC A2/5, the Intelligence Costing Working Group (ICWG) was established to develop an Air Force/Defense framework for identifying and estimating the cost of intelligence requirements. The ICWG is chaired by Air Force Materiel Command's Intelligence Squadron, Plans & Resources Flight (AFMC IS/A2X). | 2008 | Lessons Learned |
| Analyzing Cost Growth in Government Acquisitions | Richard G. Cline, Toan Nguyen, David Ross | This paper studies a sampling of historical programs that have experienced varying levels of cost growth. The chief purpose is to identify (common) causes; thereby offering an opportunity to make recommendations that can mitigate those causes | 2008 | Lessons Learned |
| CUSTOMIZATION OF SUCCESS ESTIMATOR TO CREATE FAR-COMPLIANT COST PROPOSALS | Charlie Griffin | Two years ago, the federal government-based business unit of Parsons Corporation decided on US COST's Success Estimator as its standard estimating software. Although government agencies have been a cornerstone of the Success user base for many years, the application did not provide out of the box cost reimbursable proposals from the contractor's perspective that were compliant with the Federal Acquisition Regulations (FAR)... | 2008 | Lessons Learned |
| CAD Model and Parametric Cost Model Integration " A Case Study | Christopher Rush | This paper explores the integration of CAD (Computer Aided Design) models and Parametric Cost Models. The author demonstrates, through means of a case study, an approach to CAD and Parametric Cost Model integration. CAD models are widespread in industry; they are an integral element of the design process in most companies. Since as much as 80% of a product cost is committed during the early stages of design, the integration of CAD and Cost models is intriguing. Such integration would seemingly provide enormous benefit to the product development process... | 2008 | Methods & Models |
| Combining Deterministic and Stochastic Processes in FAA Cost-Benefit Estimating | Katrina Hall, Mark Kipperman | For many years the FAA has struggled to provide quality products and services to the aviation community while controlling schedule and cost overruns. Agencies such as OMB, GAO, and the DOT OIG have stated that the FAA must become more fiscally responsible in providing these products. The agency has responded by developing in- house processes and procedures that will provide the framework to achieve this directive... | 2008 | Methods & Models |
| The Statistical Approach for USCM9 CER Development | Shu-Ping Hu, Michael Pfeifer, Nick Lozzi | The Minimum-Unbiased-Percentage Error (MUPE) regression technique was used to develop the Unmanned Space Vehicle Cost Model, Seventh Edition (USCM7) and Eighth Edition (USCM8) Cost Estimating Relationships (CERs). This is because MUPE is a well-established technique to model multiplicative error CERs. To prepare for USCM9 CER development, the Air Force Space and Missile Systems Center (SMC) and Tecolote Research have been evaluating various statistical approaches. This effort includes selecting a regression technique, defining CER selection criteria, and evaluating cost improvement modeling alternatives... | 2008 | Methods & Models |
| The Traveling NET Trainer Problem | Angela Lemke | As part of providing a life cycle cost estimate for an Army program, the costs of providing New Equipment Training (NET) to a number of continental U.S. and overseas locations were examined. The proposed solution uses a traveling salesman approach to obtain a close to optimal cost solution for sending a team of NET trainers to a number of CONUS and OCONUS US Army Depot and Intermediate Maintenance locations. Stochastic optimization methods to include more traditional Monte Carlo Simulation and novel approaches such as Ant Colony Optimization can be used to optimize NET costs using common software tools such as Microsoft Excel. | 2008 | Methods & Models |
| Using your Cost Estimate to Answer Budget Questions | Melissa Cyrulik | We can use the budget and estimate together to answer three key questions. First, is the budget sufficient for my program and are there years with budget shortfalls? Second, if the budget changes how will it affect my program? Third, which estimate elements should be watched carefully to remain on budget? The focus of this presentation is to show modeling techniques used to answer these questions and to propose how to display the results to management... | 2008 | Methods & Models |
| Using C-17 Program's Data to Build a Parametric Model for the Engineering Organization | Myung-Yul Lee | The Boeing Company, C-17 program, located in Long Beach, California has produced more than 171 cargo aircraft to the U.S. Air Force after its first flight on September 15, 1991. The Boeing C-17 Program Office has responsible for managing the Producibility Enhancement/Performance Improvement (PE/PI) contracts. The major efforts under the PE/PI contract are: Y Perform studies and analyses, Y Design, develop, test and prototype C-17 Weapon System improvements and enhancement, Y Develop change proposal for Production and retrofit incorporation into aircraft | 2008 | Methods & Models |
| Providing Visibility of the True Cost of Marine Corps Maintenance | Ted Kuusisto | Marine Corps Logistics Command has developed a suite of decision support capabilities to assist in the analysis of materiel life cycle management activities. The Marine Corps Equipment Readiness Information Tool (MERIT) with its Total Support Cost (TSC) module plays a key role in helping the Marine Corps Logistics Command determine affordable readiness. In its graphical and tabular presentation forms, TSC displays cost and cost metrics of Marine Corps weapon and logistics systems to provide reliable investment advice to senior Marine Corps leadership... | 2008 | Methods & Models |
| Adaptive Cost-Estimating Relationships | Stephen Book, Melvin Broder, Daniel Feldman | Traditional development of cost estimating relationships (CERs) has been based on full data sets consisting of all available cost and technical data associated with a particular class of Products of Interest (PoIs), e.g., components, subsystems or entire systems of satellites, ground systems, etc. In this paper, we extend the concept of analogy estimating to parametric estimating by deriving adaptive CERs, namely CERs that are based on specific knowledge of individual data points that may not be reflected in the full data set at the time that the original CER was derived... | 2008 | Methods & Models |
| Translating Cost Models from Excel into ACE: Best Practices and Lessons Learned | Sam Bresnahan | Translating life cycle cost models from Excel into ACE often proves to be a daunting task, especially for analysts who are novice or intermediate-level ACE users. This session will recommend a general approach to the translation process and identify several ACE best practices: combining Dynamic Equation Columns (DECs) and the SumIf( ) function to manage hardware configurations; accessing lookup tables with relative row addressing; and using the OpFieldedUnits( ) function to phase Operations & Maintenance costs... | 2008 | Methods & Models |
| Identifying Best-Value Technologies Using Analogy-Based Cost Estimating Methods and Tools | Mark Schankman, John Reynolds | Defense and Aerospace technology planners and engineers are faced with a challenging problem: how to select from many alternative technology research & development projects to provide the best value (meet needs at lowest cost). The problem is complex because: Defense and Aerospace needs depend on many performance requirements where each requirement requires the combination of several system attributes (eg, lethality, range, speed) each with a resulting cost impact., New technologies must be identified and developed to provide the required performance., The cost to develop these technologies is difficult to estimate due to inadequacies of existing cost estimating tools and methods. | 2008 | Methods & Models |
| Estimation of Non Recurring Costs in Manufacturing | Ronal Bradbery, Linda Newnes, Antony Mileham | This research aims to improve estimation of the non recurring costs (NRCs) of manufacture. Investment required prior to the first unit of production varies depending on the industry. However shorter production runs mean that manufacturing investment levels are increasingly relevant. The larger body of cost estimation research is focused on modelling product unit cost. Much cost estimation practice is informed by experience and understanding of costing relationships. The less frequent development of new manufacturing capacity limits options for research and the development of bodies of expert knowledge in the estimation of NRCs... | 2008 | Methods & Models |
| Quantum Modeling of Project Cost and Schedule | Ilya Fishman, David Graham | In conventional cost estimates, project cost is a stochastic sum of individual tasks costs. For the purpose of cost estimate, tasks could be performed in arbitrary sequence thus making the sched- ule links irrelevant. This disregard of task coordination and communication may be responsible for up to 40-50% of the total project cost... | 2008 | Methods & Models |
| Comprehensive Costing Model for WATer and Environmental Research Systems Network (WATERS Network) | Akshay Raut, Jaineel Aga, Thariq Kara, Jeffrey Glass | This presentation will review the development of a preliminary costing approach and model for a large scale, multidisciplinary scientific project being developed by numerous principal investigators across the country: the WATERS Network. This project is a joint initiative between the hydrologic science and environmental engineering research communities and funded by the National Science Foundation (NSF) to provide opportunities to further the scientific understanding of water as an integrated resource and the effects of anthropological activities. It is a bold environmental observatory initiative to transform research on the water environment by investing in new infrastructure and facilitate investigations (WATERS Network, 2008)... | 2008 | Methods & Models |
| A Non-Simulation Based Method for Inducing Pearson's Correlation Between Input Random Variables | Eric Druker, Richard Coleman, Peter Braxton, Joel Hughes | Several previously published papers have cited the need to include correlation in risk analysis models. In particular, a landmark paper published by Philip Lurie and Matthew Goldberg presented a methodology for inducing Pearson's correlation between input/independent random variables. The one subject absent from the paper was a methodology for finding the optimal applied correlation matrix given a desired outcome correlation. Since the publishing of the Lurie-Goldberg paper there has been continuing discussion on its implementation, however there has not been any presentation of an optimization algorithm that does not involve the use of computing-heavy simulations. ... | 2008 | Methods & Models |
| A Process for Translating Project Risks into Cost Risk | Robert Kellogg, Robert Bitten, Debra Emmons, Randy Persinger | Space system development projects typically have processes defined to manage technical and programmatic risks. These processes usually include some combination of risk identification, quantification, and mitigation. The standard 5x5 risk matrix can be used to display the current risks and the changes over time. The 5x5 risk matrix requires the quantification of the likelihood of a risk occurring and the consequence if the risk does occur. This information may be incorporated into the project budget as liens or it may be tracked separately from the budget process... | 2008 | Risk |
| Inflation Risk: Assessing Inflation Risk in Multi-Year Proposals | Chris Leonetti, Brian Welsh, Bethia Cullis, Richard Coleman, Peter Braxton, Eric Druker | Government contracts are often let for long periods of time " up to ten years or more. When contractors compete for these long-term contracts, the inflation built into the bids (or omitted!) introduces a potentially serious risk area. If the inflation numbers included in the bid are off, the contract will perform at a significantly different cost, and thus return on sales than projected, and, given the size of many government contracts, this risk may be very large in dollars as well, adversely affecting either the contractor or the customer, or both, depending upon contractual arrangements... | 2008 | Risk |
| What is Risk? | Peter Frederic | Or more importantly, what is Uncertainty? Risk is defined as the possibility of suffering harm or loss. Uncertainty is generally defined as being unsettled or in doubt but has the more specific statistical definition of the estimated amount or percentage by which an observed or calculated value may differ from the true value. Cost estimators tend to use the words risk and uncertainty interchangeably, which causes confusion, especially when communicating with people outside the cost community. To avoid this confusion, it is the statistical definition of uncertainty that should be the cost estimator's yardstick when we talk about risk or uncertainty... | 2008 | Risk |
| How Many Iterations Are Enough? | Alfred Smith | A common question regarding uncertainty analysis is: How many iterations are needed to obtain stable and reliable simulation results? The answer is always: It depends. Many factors influence the stability of the uncertainty simulation: how many uncertain elements are modeled; how many layers deep is uncertainty assigned; to what degree are uncertainty distributions skewed; what proportion of the distributions are left, center or right skewed; what is the proportion of simple vs. complex distribution shapes; how many cyclic relationships in the WBS; is correlation applied; and to what degree is functional correlation present... | 2008 | Risk |
| Knowledge-based, Risk-based, and Web-based Cost Estimation | Brian Lewis | We will introduce the Vanguard SystemTM, a new cost estimating software solution that focuses on knowledge-based generative costing models. As technology evolves and diversifies, costing by analysis (generative costing) rather than costing by analogy (parametric costing) is emerging as a critically necessary planning capability. Generative costing methods calculate the aggregate cost of an object by decomposing it into estimates for individual processes, activities, resources, or whatever fundamental units are required... | 2008 | Risk |
| Theory of Project Risk and Schedule Probability (Quantum Modeling) | Ilya Fishman, David Graham | Reliable calculation of schedule probability as a function of risk remains an extremely important and unresolved problem of project management. It is well known that real project durations might differ from originally planned by a substantial factor of 30%-100%. Existing methodology addresses the problem by establishing task distribution functions and con- ducting Monte Carlo simulations of milestone probability. These results fully depend on the ac- curacy of task distribution functions... | 2008 | Risk |
| U.S. Aerospace Industry Cost Risk Analysis Survey | Hollis M. Black III | The purpose of this research paper is to summarize how the U.S. Aerospace Industry (Government and contractor) develops and applies cost risk analysis to aid business decisions. This paper is based on a survey provided to 2400 SCEA and SSCAG members in early 2008. It summarizes current risk assessment methods and tools, and the benefits to business decisions. Survey responses (105) were received from 32 parent organizations, many with multiple sites. Organizations included 5 U.S. Government agencies, 12 major corporations, 13 support contractors, and 2 European agencies (Ministry of Defense and European Space Agency)... | 2008 | Risk |
| Software Risk Simulation | Jeffrey Jaekle, Eric Druker, Emily Jessen, Jacquelyn McQueston, Brian Brophy | Software cost estimation should always address a certain level of risk within the predicted cost. Major risks within any software development effort include code growth, schedule slips, and productivity rates. Historical data can be gathered for a program that can provide insight into the average factors to use for each one of these risks. By using a Monte Carlo simulation with the statistical distributions around each of these data sets a range of possible cost outcomes can be obtained... | 2008 | Risk |
| Monte Carlo Simulation Practicum | David Alley | Monte Carlo analysis is commonly used to predict the cost of future activity by acknowledging the uncertainty of the assumptions that contribute to the cost estimate and enabling the expression of those uncertainties in clever ways that bring forth and capture the subtle information that does exist... | 2008 | Risk |
| How to Capture Discrete Cost Risks in Your Project Cost Model | Dave Graham, Alfred Smith, Melissa Cyrulik, Robert Kellogg, Robert Bitten, Debra Emmons | Discrete cost risk analysis is defined in different ways by different cost analysts. It can mean the identification of specific risk under a specific scenario, the estimate of the cost impact due to that specific risk and/or the addition of the cost estimate for the risk into the cost estimate of the element, subsystem or system. Almost always, it can involve more than one risk under a given scenario. However, each scenario requires a risk adjusted cost estimate and there is generally only enough time to develop a few specific scenarios. NASA PA&E/CAD cost analysts are exploring simulation methods to perform discrete cost risk analysis... | 2008 | Risk |
| A Systems Thinking Model for Cost Growth Mitigation: Finding the Leverage Points | Michael Seibel | The incompleteness and inconsistencies of our ideas become clear only during implementation. This quote by Dorothy Sayers says much as to how cost growth happens. It's no surprise to the casual observer that our world is getting more and more complex. Unfortunately, we are negotiating the 21st century with a cave man mind that does not deal well with complexity. As a result, project teams continually under scope the content and difficulty of their programs... | 2008 | Risk |
| DEVELOPMENT AND PRODUCTION COST EQUATIONS DERIVED FROM PRICE-H " TO ENABLE RAPID AIRCRAFT (MDO) TRADE STUDIES | Tom Harwick | One of the challenges in performing engineering design trade studies is to meet the required cycle time or trade study tempo and to have sufficient data to model the trade study alternatives for each architecture concept. Reduced-form cost equations, derived from industry cost models, provide a convenient way to allow for rapid single as well as multiple architecture aircraft studies. This capability is particularly important in a Multi-discipline Design Optimization (MDO) environment... | 2008 | Software/Hardware/Tools |
| Revolutionizing Cost Estimating with TruePlanning 2008 | Arlene Minkiewicz | Successful project planning requires good cost estimating practices. Good cost estimating practices require that we think beyond the prediction of the cost, effort and schedule for hardware, software or systems.. The cost estimator must have a toolset and a mindset that facilitates unimpassioned trade-offs between scope, schedule, and budget in order to provide decision makers with all the information necessary to plan and manage the development of complex systems... | 2008 | Software/Hardware/Tools |
| Object-Oriented Estimation Techniques | Leah Upshaw | Most commonly available software estimation methodologies use either source lines of code (SLOC) or function point (FP) as the basis for the measurement of their size and scope. While these methodologies and tools may suffice for software projects that use structured software development techniques, it is more effective to use a methodology that takes advantage of the information provided by the use cases, scenarios, and more in depth object-oriented (OO) metrics for projects that employ such techniques... | 2008 | Software/Hardware/Tools |
| Improving Development Scheduling and Cost Flow in SEER-H | Ainsley Chong | SEER for Hardware is a leading hardware cost estimating and planning system. Until recently, SEER HW had basic visibility into the scheduling of estimates, limited to annual expenditure. An upcoming version of SEER HW will include a set of new features to help program managers better schedule tasks and manage their expenditures. This effort, sponsored in part by NASA IPAO, will be part of the 7.1 release of the SEER HW product... | 2008 | Software/Hardware/Tools |
| A Basis of Estimate (BOE) Tool for Project Estimates | Bob Fairbairn, James Miller, Rosemary Baize | This Basis of Estimate tool is under development at NASA Langley Research Center, for use in formulating and documenting project build-up estimates, as part of an effort to improve the quality and documentation of proposals | 2008 | Software/Hardware/Tools |
| Software Total Ownership Cost: Development is Only Part of the Equation | Daniel Galorath | Software development is a costly, often schedule driven activity, prone to compromises to meet schedule. Many of these compromises have far reaching impacts on the cost of software maintenance, total ownership costs and software sustainability | 2008 | Software/Hardware/Tools |
| Automating SEER for Software: A Quick Cost Estimation Process | John Teal, Gary Hellenga | Software estimates are useful tools for examining project scopes ranging from an entire system- of-systems program, down to a specific component of a small subsystem. In general, however, producing an estimate is neither quick nor easy. Even worse, maintaining the estimates as projects mature and undergo change is laborious " and consequently, tracking the progress of a project's estimates over the project lifetime is rarely effectively accomplished. What is desired is an estimation process that can respond with agility to changes in project direction and to the assumptions on which the estimate is based, without undue cost and effort in updating the estimates themselves... | 2008 | Software/Hardware/Tools |
| A Structured Framework for Estimating IT Projects and IT Support | Karen McRitchie, Ross Hunt | In today's global economy, Information Technology is a critical component of many companies' core business operations, impacting operational efficiency, customer relationship and the bottom line. These organizations need an agile and responsive IT organization with capability to support dynamic business goals. That means IT teams today must take on a much more diversified class of projects and initiatives and to be part of the business case evaluation team to determine feasibility and advisability of not only IT project development but the companies capability for deployment, implementation and support of these initiatives through the complete project life cycle... | 2008 | Software/Hardware/Tools |
| Hard Disk Storage Deflation: Is There a Floor? | Laura M. Friese, Allison Converse, David Wiley, Alisha Ssoles, Christopher Thomas | Storage system costs are a major component of many automated information systems (AIS) cost estimates. Vendor quotes or specific storage specifications are sometimes available to estimate the cost of these storage solutions, but often, especially for significantly future storage cost projections, a $/volume of storage is often applied to projected storage volumes. Storage costs historically deflate considerably over time; Cost Deflation vs. Technology Inflation of RAID (Redundant Array of Independent Disks) Storage Systems (Converse, Watkins, SCEA 2006) supports the claim that RAID storage costs deflate 30-40% per year... | 2008 | Software/Hardware/Tools |
| An Approach for Building a Normalized Software Database using SRDRs | Michael Gallo, Megan Holzman, Paul Hardin, Elizabeth Koza | During the last 2 1/2 years Technomics has collected software data for the Army. The Army recognized it will be many years before current ACAT I programs complete their software developments and hence receive the associated SRDR data. Because of this 'software data gap', the Army is funding research to collect software data from Army contractors for recently completed software developments. Technomics' data collection approach is similar to that of the SRDRs: collect data in its raw unadulterated form and collect meaningful definitions of the data to permit correct interpretation of the data. | 2008 | Software/Hardware/Tools |
| SE/IT/PM Estimating Approaches for Service-Oriented Architecture Environments | Hilary Eckberg, Kevin J. Snyder | Within the Intelligence Community, there is a movement towards breaking down stovepiped architectures and implementing service-oriented architectures (SOAs). The idea is that new programs will be integrated into an existing SOA framework, thus allowing data sharing between systems. Estimating costs, particularly Systems Engineering/Integration and Test/Program Management (SE/IT/PM) costs, for these new programs that will be sharing the system poses several problems. The questions that arise from this are as follows:... | 2008 | Software/Hardware/Tools |
| SEER for Hardware's Cost Model for Future Orbital Concepts (Far Out) | Lee Fischman | The Advanced Spacecraft Cost Model effort (Far Out) has sought to produce a model that can, with reasonable certainty, estimate orbital concepts lying 15 years into the future. In the process it has assembled a spacecraft database of over 200 missions, 150 of them associated with cost. Far Out has sought to predict the future largely by trending forward estimating methods calibrated in progressive chronologic windows against past missions. A number of high level (from the perspective of information provided) estimating methods have been attempted... | 2008 | Software/Hardware/Tools |
| Big Iron EVM: Implementation at AEDC " Initial Success Met Failure | Dave Peeler | rnold Engineering Development Center (AEDC) had experimented with Earned Value Management (EVM) for several years, under its support contract. With the source selection for FY04 consolidating the test and support contracts, several personnel recognized the opportunity to bring the entire center under the rubric of EVM. Understanding costs and particularly schedules has long been a supreme challenge to managers at AEDC. A focus on EVM would produce greatly enhanced insight into the test and maintenance aspects of Center operation. EVM had certainly helped transform the mission support aspect of things at AEDC... | 2007 | Earned Value Management |
| New Heavy Launchers, a Cost Estimating Challenge | Michael van Pelt | Estimating development and recurring cost for a new heavy launcher pose some challenges. It has been a long time since Saturn V was last flown, and cost estimate models based its development and launch cost may not accurately predict the effects of the current state of technology or development and production methods... | 2007 | Earned Value Management |
| Ending the EAC Tail Chase - An Unbiased EAC Predictor using Progress Metrics | Eric Druker, Richard Coleman, Jeff Jaekle, Elisabeth Boyadjis | The world of EVM has traditionally produced results that are unreliable - specifically, traditional EAC predictions are biased low; they are "in a tail chase." Many papers (by Christensen and others) show that all standard CPI and SPI-based EAC predictions are in essence lower bounds. While producing estimates for a customer whose EACs have traditionally tail-chased from start to finish, the authors stumbled upon a method of that takes into account the performance of completed programs of a similar nature... | 2007 | Earned Value Management |
| A Simplified Earned Value Management Approach | Dorothy Tiffany | Many organizations, especially agencies and departments within the federal government, are struggling with requirements to implement Earned Value Management Systems quickly. At the same time, issues such as culture, training and system architecture, have stymied those trying to respond to these new requirements in a timely manner. This presentation identifies many of the roadblocks to implementing EVM, and provides some ideas to bridge the gap until fully compliant EVM systems are attainable. Some very basic EVM principles and analysis will be discussed to show that simplified EVM may not be that hard to implement, after all. | 2007 | Earned Value Management |
| DoD Military Construction Inflation Index vs. Construction Industry Inflation Indices | Brian Welsh. Vince Russo | Designing and constructing a building is a very involved, detailed, and long term project. Buildings are made of many different commodities that fluctuate due to external conditions such as supply and demand forces, political events, and trade agreements. To help plan and estimate the cost and duration of a building, indices are used to determine the future price of construction materials based on previous prices. Department of Defense construction projects are recommended to use the Total Obligation Authority Military Construction Inflation Indices published by the Office of the Undersecretary of Defense... | 2007 | Earned Value Management |
| Accurately Estimating Class C Spacecraft Bus | Duncan Thomas, Meghan Connelly, Greg Hogan | This paper quantifies the cost and explanatory technical differences between Class A and Class C space vehicles, both of which are often included in space cost databases. This effort was completed by the Air Force Cost Analysis Agency (AFCAA) | 2007 | Earned Value Management |
| The Evolution of Hardware Estimating | Arlene Minkiewicz | Just as Nintendo evolves gaming and Apple innovates telecommunications, PRICE Systems examines how hardware estimation evolves as technology improves at breakneck speeds. This paper presents the findings of a study on the impacts of improving technology on the development and delivery of hardware and hardware systems... | 2007 | Earned Value Management |
| Automatic Layout of Hardware Effort/Duration Estimates Into Microsoft Project | Lee Fischman | With the introduction of the SEER-H Client for Microsoft Project, Galorath has mated parametric hardware cost estimation with a leading project planning solution. The "H Client" permits SEER-H projects of any size to be exported into Microsoft Project using either default or customized lifecycle templates, permitting estimators to evaluate and modify detailed schedules. This presentation demonstrates this capability and provides insight into how it was achieved. | 2007 | Earned Value Management |
| Challenges to Integrated Cost and Performance/Effectiveness Analyses | Kurt Willstatter, Andy Campbell | The defense acquisition community has made significant investments into enabling simulation of potential military systems early in the concept development phase. Furthermore, additional challenges in acquisition modeling have been imposed due to military requirements migrating from specifying systems to specifying capabilities. As such, analogous pressures have caused perturbations within the cost analysis community, especially to facilitate decision- making on future investments in a budget-constrained environment. Estimating the cost of concepts and/or capabilities requires new ideas and approaches to address accounting for sunk costs, sharing costs across multi- capability systems and multi-theater systems, and time- phasing of costs across limited numbers of assets. This presentation provides practical examples to address these issue(s). | 2007 | Management & Lessons Learned |
| COLLABORATIVE ESTIMATING, RESPONSIBLE, ACCOUNTABLE, COMMUNICATED AND INFORMED | Shamraz Razzaq | From their inception, cost estimating models have been operated by individual users interpreting the technical and programmatic information that they have gathered into the cost models for which they have been trained. However, there is now an opportunity for a change in this culture and the way in which Parametricians interact with each other and their customers... | 2007 | Management & Lessons Learned |
| The Evolving Launch Vehicle Market Supply and the Effect on Future NASA Missions | Robert Bitten, Debra Emmons, Claude Freaner | The upcoming retirement of the Delta II family of launch vehicles will leave a performance gap between small expendable launch vehicles, such as the Pegasus and Taurus class, and larger vehicles, such as the Delta IV and Atlas V families. This performance gap may lead to a variety of progressions including large satellites that utilize the full capability of the larger launch vehicles, medium size satellites that would require dual manifesting on the larger vehicles or smaller satellites missions that would require a large number of smaller launch vehicles... | 2007 | Management & Lessons Learned |
| Improving the Cost Estimating Capability at the FAA | Katrina Hall | For many years the FAA has struggled to provided quality products and services to the aviation community while controlling schedule and cost overruns. Agencies such as OMB, GAO, and the DOT OIG have stated that the FAA must become more fiscally responsible in providing these products. The agency has responded by developing in- house processes and procedures that will provide the framework to achieve this directive... | 2007 | Management & Lessons Learned |
| Myth Buster: Do Engineers Trust Parametric Models over Their Own Intuition? | Ricardo Valerdi | This paper explores the question of whether engineers trust parametric models over their own intuition when performing cost estimates. The approach to answering the question is similar to that of the popular television show MythBusters which aims to separate truth from urban legend using controlled experiments. We performed a controlled experiment with forty systems engineers and discovered that most of them trust their intuition more than a parametric model. This challenges a central assumption in the field of parametrics: that decision makers mainly base their cost estimation decisions on parametric models. We explain how this phenomenon can be scientifically possible based on theories of cognitive decision making. | 2007 | Management & Lessons Learned |
| Cost Estimating and Management in Virtual Manufacturing Networks | Charlie Stirk, Dennis Thompson | A virtual manufacturing network is a loosely coupled enterprise, which is formed by many partners (whole or parts of real companies) enabling a group of individual real enterprises to operate more efficiently and effectively, as if it is a single global enterprise to fulfill a specific mission. We focus on the costing capabilities needed to enable two particular types of network centric solutions: rapid response to manufacturing legacy parts, and manufacturing readiness and launch of new products/processes. For legacy parts where the original supplier may not be available, a new manufacturing network must be rapidly assembled to reverse engineer, plan the manufacturing processes and supply chain, and estimate lead-time and cost... | 2007 | Management & Lessons Learned |
| GAO's Cost Assessment Guide | Karen Richey, Jennifer Echard, Carol Cha | The U.S. Government Accountability Office (GAO) is responsible for, among other things, assisting Congress in its oversight of the federal government, including agencies' stewardship of public funds. Legislators, government officials, and the public want to know whether government programs are achieving their goals and what their costs are. The capability to generate reliable program cost estimates is a critical function necessary to effectively use public funds and to support the Office of Management and Budget (OMB) capital programming process. Without it, agencies are at risk of experiencing cost overruns, missed deadlines, and performance shortfalls, all of which are recurring problems that our program assessments too often reveal... | 2007 | Management & Lessons Learned |
| Turning Avoidable Guidelines Into Sensible Requirements" Credible Space Cost Estimating Policy | Jill Hardash | In the last five years NASA has experienced a renaissance in cost estimating. The small but remarkable crew of remaining estimators at the Agency started an effort to rebuild the cost estimating capability at all of the Centers, including NASA Headquarters. This involved growing the number of trained cost estimators, educating cost estimating customers and providing guidance on the discipline of cost estimating at NASA. Without these three components credible cost estimates within NASA would not be possible. Five years later, with a robust cadre of trained estimators, how has the cost estimating message been delivered to customers?... | 2007 | Management & Lessons Learned |
| Implementing a robust cost estimating process | Herve Joumier | How do cost estimating entities bring value in an organisation and how can survive such groups through time? These questions may not have easy answers and many counter-examples of short existence are observed both in industrial companies and procurement agencies. They tend to resurface later on in different forms but in most of the cases this reshuffling is accompanied with significant loss of experience, qualified personnel or even loss of reference data and cost models... | 2007 | Management & Lessons Learned |
| Realities of Cost As an Independent Variable (CAIV) - Stakeholder Perceptions | Greg Kiviat, Stuart Swalgen, Sebastian Botta | Cost As an Independent Variable (CAIV) has been shown to be an effective tool to identify and mitigate program risk by balancing effectiveness and affordability. CAIV establishes goals and provides cost feedback to development teams as requirements change and design details emerge. However, depending on the project phase (Concept, Technology Development, SD&D, Production and Operating & Support) and the level of stakeholder support (both customer and contractor), there is great variability in the level and scope of CAIV application and investment in cost analysis capability... | 2007 | Management & Lessons Learned |
| Cost, Technical, and Acquisition Differences Between Military and Commercial Communication Satellites | Duncan Thomas, Meghan Connelly, Linda Snow | The purpose of this briefing is to explain differences between commercial and military communication satellite acquisition and how these differences impact development and production costs. The goal of Air Force satellite acquisitions is to maximize operational capabilities while minimizing development and procurement costs. While sound in concept, these goals often end up being contradictory. One overarching contribution to this is the common misconception within government satellite acquisition community concerning the applicability of commercial practices to meet operational requirements... | 2007 | Management & Lessons Learned |
| Counterinsurgency Aircraft: An Analysis of Two Alternatives | Dave Peeler | Within military academic circles and the special operations community, a call is being made for an aircraft dedicated to the Counterinsurgency (COIN) mission. Given the Air Force's budget constraints brought on buy the dedication to the F-22A, advocacy for a COIN aircraft needs the concreteness of hard numbers. Building on Arthur Davis' COIN aircraft advocacy paper, this research focuses not on further advocacy, but on a process and method to actually procure a COIN aircraft. The acquisition focus is on US Special Operations Command's acquisition authority coupled with its GWOT mission responsibility, using commercial-off-the-shelf aircraft procurement to specifically address the need for an airborne COIN capability... | 2007 | Management & Lessons Learned |
| The Portfolio Effect Reconsidered | Christian Smart | It has recently been argued that funding for multiple missions benefits from a portfolio effect " that is, that the various risks of cost overruns for individual missions is less when considered as a group. This is based on modern financial theory that was originally developed by Nobel laureate Harry Markowitz. The result of this diversification, in theory, should allow NASA the ability to fund individual missions at, for example, the 61st percentile, while achieving an overall confidence that is much higher, such as the 80th percentile... | 2007 | Management & Lessons Learned |
| Cost Estimating Challenges for the Constellation Program | Christian Smart | NASA's Constellation program has introduced numerous cost estimating challenges. This new program is NASA's first large-scale development program of a new launch vehicle since the Space Shuttle. The Constellation program is similar to the Apollo program, especially with regard to hardware. However, the Apollo program began development in the 1960s. Thus application of the analogy-based cost methods that estimators often prefer poses significant issues for Constellation. NASA has met this challenge by developing and implementing multivariate cost- estimating relationships (CERs)... | 2007 | Management & Lessons Learned |
| A Cost Control methodology designed to apply for defense acquisition of South Korea. | Kyu Meyung Choi, Jongyou On | Defense system as a long lead program has the high probability of cost growth due to the uncertainty and risk. Therefore, meeting schedule, technical performance and preventing excessive cost overrun is the main concerning point to success defense program. CAIV, Cost as An Independent Variable, is taking shape in Department of Defense Acquisition of USA. The policy to adapt CAIV is to place more emphasis on cost control. The goal of CAIV is to shift the paradigm from more performance is better to an affordable balance between cost and performance is better.... | 2007 | Management & Lessons Learned |
| Pioneers of Parametrics | Ricardo Valerdi | This paper provides a historical account of the development of the field of Parametrics through information obtained during interviews with twelve pioneers of the field. Cost model developers, users, and practitioners were interviewed with the intent to capture their views on the impact between cost estimation research and practice. The individuals interviewed represent a diverse range of perspectives including academia, government, and industry. Each perspective sheds light on the areas in which the field of Parametrics has had an impact and which synergies have been influential in the development of the field. The implications of the findings are discussed in light of the future challenges for the field of Parametrics. | 2007 | Management & Lessons Learned |
| SE/IT/PM Trend Analysis | Alex Wekluk, Nathan Menton | Systems Engineering (SE), Integration and Test (I&T), and Program Management (PM) costs have traditionally been difficult items to estimate at any given point in a program. Cost Estimating Relationships (CERs) are often used in practice to predict the ultimate SE/IT/PM cost. The utility of a CER naturally lessens as a program matures since CERs do not typically take into account known performance to date or specific details of the program that are not captured in the CER variables... | 2007 | Methods |
| An Analysis of the Economic Impact for Airlines at LGA from a Tower Modernization Alternative that Closes Gates | Marc Rose | This paper examines the economic impact on airlines due to delay should the airport lose gates. This paper will describe the techniques and data used to develop the inputs for a queuing model of LaGuardia airport. The results of the analysis allowed us to estimate the economic impacts to the airlines. This, combined with the estimated costs, helped the FAA in making a decision on a new tower with no loss of gates versus upgrading the existing tower. | 2007 | Methods |
| Learning Curves Redux: A New Use for a Familiar Tool | Evin Stump, Alexandra Minevich | This paper proposes another use for learning curves, namely the scheduling of production operations. This is not entirely a new idea, but it is usually not formally implemented in the planning of production operations. Learning curves alone can't do this planning job effectively, but when combined with other appropriate, relatively simple logic, the result can be an automated scheduling process that predicts not only the cost of each produced item, but also the dates when production of each item will start and finish, plus a spread of labor hours and material costs by month, or even by week, if desired... | 2007 | Methods |
| CER Prediction Uncertainty | Matthew Goldberg, Richard Sperling | In this paper we compare the parametric bootstrap to the Delta method as tools for quantifying the uncertainty of predictions from a nonlinear cost-estimating relationship (CER) with a multiplicative error term. The bootstrap is a computationally-intensive resampling technique used to estimate bias and standard errors of statistical estimators. The validity of the bootstrap depends on asymptotic theory and is exact only in very large samples, whereas most CERs are estimated from small samples. The Delta method provides a linear approximation of the mean and variance for a nonlinear estimator... | 2007 | Methods |
| Work Scope Database for Cost Proposals | Robert Webster | In most situations the resources necessary to support the creation of proposals to win business opportunities are scarce and sometimes inadequate for the number of opportunities available. Waste for any reason in the proposal process reduces the number of opportunities a business can pursue. On large proposals the customer requires starting from a position of fact whenever possible. That requirement demands a searchable Labor Actuals and Labor Work Scope Database. This discussion focuses on the Labor Work Scope Data of past programs... | 2007 | Methods |
| Product, Sub-Market and Market Demand | Doug Howarth | Markets demonstrate aggregate demand curves in which the number of units purchased varies inversely relative to price. These curves mathematically link to the values that buyers place upon products offered for sale. Over time, multiple sub-markets form within larger marketplaces. Sub-markets have their own demand curves, which sum to the market aggregate demand curve. The demand curve for a hypothetical new product is a measure of buyers' demonstrated responsiveness to its specific valued attributes and product saturation in general... | 2007 | Methods |
| Economic Analysis Process Applied To the Worlds' Largest Wind Tunnel - RC1 Cooler | Dan Wyman, Robert Nelson | Economic Analysis Process Applied To the World's Largest Wind Tunnel - RC1 Cooler RC-1 Concerns o A single point of failure o Contains environmentally hazardous/toxic fluids o System wear out " aged technology | 2007 | Methods |
| Desired Characteristics of a Senior Cost Estimator | Hollis Black, Harold Bodin, Michael Herrington | Introduction. This paper provides a comprehensive overview of the skills, strategy, and work style of a Senior Cost Estimator. Our purpose is to encourage younger Estimators to proactively improve their knowledge, methods, and inter-personal skills, thereby increasing their value to the organization. Management relies on the contributions of Estimators in making sound business decisions. | 2007 | Methods |
| Hilbert's Problems for Cost Estimating | Peter Braxton, Richard Coleman | At the International Congress of Mathematicians at the Sorbonne in Paris in 1900, German mathematician David Hilbert boldly put forth a list of 23 theretofore unsolved problems in mathematics, which subsequently became quite influential in 20th-century research efforts. Though seven years late, the authors propose a similar list for 21st- century cost estimating and risk analysis. Because cost is an inherently but not purely analytical field, some of the problems take the form of broader issues to be addressed... | 2007 | Methods |
| Life Cycle Cost Analyses of C4ISR Systems | David Stamm, Greg Bell | Command, Control, Communications, Computing, Intelligence, Surveillance, and Reconnaissance (C4ISR) systems have emerged as a prominent area of defense investment, modernization, and transformation. In the future, C4ISR will continue to command an important share of defense spending due to the following attributes:... | 2007 | Methods |
| To b or Not to b - The y-intercept in cost estimation | Richard Coleman, Jessica Summerville, Peter Braxton, Eric Druker, Peter Braxton | There is considerable, if low-key controversy over the y-intercept in cost estimation. The controversy, although more common than one would suppose, has received surprisingly little attention in societies and publications, but the authors have found that it is a controversy nonetheless, and one with sometimes significant implications. There are proponents that argue that all or most cost estimating relationships (CERs) should necessarily pass through the origin, and those that argue that they need not do so... | 2007 | Methods |
| An Enterprise Model of Rising Ship Costs: Loss of Learning Due to Time between Ships and Labor Force Instability | Richard Coleman, Jessica Summerville, Peter Braxton, Eric Druker, Peter Braxton | Since the end of the Cold War, the perceived need for Navy ships has dropped, and so the shipbuilding budget has dropped. Seemingly coincidental with this budgetary pressure, and perversely aggravating the problem, ship costs began to rise steeply. We will set aside that ships have grown in weight by about three percent per year since World War II, and that ever-more weapon systems are being put into them, and confine ourselves to discussions of costs rising for ships beyond the increase in content. ... | 2007 | Methods |
| Kinetic Energy Interceptor (KEI) Affordability Process | Tim Christopherson | KEI is a development program in the emerging National Missile Defense system. An Affordability process was developed which leverages Design For Six Sigma (DFSS), Design for Manufacture and Assembly (DFMA), and Cost As an Independent Variable (CAIV). The emerging process was applied to a recent trade study, and a knee in the curve chart was developed using Launch Area Denied (LAD) as the measure of performance and Development and Test (D&T) cost as the independent variable. A step by step methodology is presented with unique, real-world results. | 2007 | Methods |
| Why ZMPE When You Can MUPE? | Shu-Ping Hu, Alf Smith | Multiplicative error terms are commonly used in the cost analysis field because experience tells us that the error of an individual observation (e.g., cost) is generally proportional to the magnitude of the observation rather than some fixed amount. In such cases, it is appropriate to hypothesize a multiplicative error term for a cost estimating relationship (CER). The Unmanned Space Vehicle Cost Model, Eighth Edition (USCM8) CERs were developed using the Minimum-Unbiased-Percentage Error (MUPE) method to model multiplicative errors. The MUPE method is also known as an iterative, weighted least squares regression... | 2007 | Methods |
| Forecasting Operation & Support Costs For Missiles and Munitions | Jerry Le May | Operation and Support (O&S) costs are an important component of Life Cycle Cost for military weapon systems. Accurate forecasting of O&S costs early in the program allows the customer and logistics services provider to construct the most cost effective support program, providing high levels of weapon readiness while saving the customer money. Missiles and munitions have a deployment and operations profile that is different from other weapons systems in that 97% or more of their available service life is spent in protected storage in a dormant state... | 2007 | Models |
| Integration of Cost History with Project Management Data for Improved Cost Estimates | David Graham, Dan Walkovitz | NASA is working to improve estimates-to-complete (ETC's) for projects as they evolve by integrating regularly produced EVMS data from these programs with NASA's CADRe system which captures historical cost, programmatic and technical data. This integrated body of data will be used during and at each transition between NASA Phases A through E: Concept Development " SDR; Preliminary Design - PDR; Detailed Design" CDR; Fabrication, Assembly, and Test " Launch; Operations and Sustainment - Disposal to generate an enhanced ETC... | 2007 | Models |
| COINCOMO The COCOTS Integration Challenges | A Winsor Brown, Vu Nguyen | The continued vision for the COINCOMO Tool has been to integrate all of the elements of the COCOMO "suite" of software development estimation tools, including the COCOMO II Charts [extension], COPSEMO, COSECMO, CORADMO and/or COAglMO, and COCOTS; and the systems engineering extensions COSYSMO and COSOSIMO. Note: COAglMO is a proposed new extension to COCOMO II similar to CORADMO, but utilizing Agile methods during the construction phase (which assumes that the products overall capabilities and performance requirements have been documented and an adequately documents architecture developed to address those requirements... | 2007 | Models |
| Defense Acquisition Performance Assessment " The Recommendation for Time Certain Development: Pipe-dream or Reality? | Peter Hantos | As a significant milestone in the DOD's continuous self-assessment process, the Defense Acquisition Performance Assessment report was released in early 2006. A far-reaching recommendation of the report is the introduction of Time Certain Development as a preferred acquisition strategy, making schedule a Key Performance Parameter. By mandating high confidence cost and schedule estimates, two arms of the well-known Iron Triangle (Cost, Schedule, and Performance) become fixed, leaving only requirements in the project management trade-space. The paper's objective is to offer engineering perspectives and a discussion of the underlying, parametric estimation issues that impact the successful implementation of the report's recommendation. | 2007 | Models |
| Cost Modeling Data Standards for Model Integration | Jeff McDowell | The objective of the work described in this paper is to develop data exchange mechanisms necessary to revolutionize the efficiency and accuracy of cost estimating through automated data exchanges with design and performance models and simulations. This development is intended as an early step towards fully integrating cost estimating into the modeling and simulation (M&S) domain, enhancing the prediction of the affordability of future systems, and the understanding of the costs of alternatives courses of action and operational scenarios... | 2007 | Models |
| Capabilities-Based Costing: Approaches to Pre-Milestone-A Cost Estimating | Martha Roper | The issue of early, rigorous evaluation of program costs is becoming more important as defense funding come under greater scrutiny. Often at this point in the life cycle, a requirement or desired capability is known, but the manifestation of the solution is unknown or described only at a high level. Can capabilities alone be used to produce a cost estimate? If so, how can we link the proposed solution to existing systems if only a particular solution's general capability set is known?... | 2007 | Models |
| Using Bottoms-Up Cost Estimating Relationships in a Parametric Cost Estimation System | Dale Masel, Robert Judd | The parametric approach to manufacturing cost estimation is preferable because of the limited amount of part data required to generate an estimate. However, the limited data means that the estimate cannot achieve the accuracy of a bottoms-up estimating system. A hybrid cost-estimating system has been developed and implemented to estimate the cost of jet engine components. The system utilizes a bottoms-up estimating approach for the actual cost estimates, but appears as a parametric system to the user. The inputs needed for bottoms-up cost- estimating relationships (CERs) are calculated from user-specified parameters based on relationships derived between part parameters and feature attributes. | 2007 | Models |
| Parametric O&S Cost Estimation Using Markov Chains and the Influence Function Method | Evin Stump, Alexandra Minevich | This paper reports on an effort at Galorath Incorporated to apply parametric methods to the system O&S estimation problem. As a goal, we wanted to reduce the number of user inputs as much as possible in keeping with maintaining decent accuracy. We also wanted a high degree of model flexibility so that the same general architecture would suffice for most kinds of vehicles and infrastructure. After some preliminary study, we decided that:... | 2007 | Models |
| Modeling the Life Cycle Cost of Protecting US Commercial Aircraft | Kirk Hoy, Kurt Willstatter | In support of the Department of Homeland Security (DHS) Counter-MANPADS program, Summit Engineering Group designed and constructed a detailed Life Cycle Cost (LCC) model to address various potential deployment alternatives. Summit Engineering Group leveraged the detailed cost groundrules and assumptions document it created, data from various DHS contractors, and other related data in constructing the comprehensive, releasable LCC model. This presentation provides the structure of the LCC model, rationale for it, and lessons learned in addressing US commercial aircraft costs. | 2007 | Models |
| Propulsion System Advanced Cost Model | Roy Smoker, Dan Feldman | MCR has developed an Advanced Cost Model for The Air Force Research Laboratory Space Vehicles Directorate (AFRL/VS) at Kirtland AFB, NM, to operate in concert with the Integrated Propulsion Analysis Tool (IPAT) developed by Advatech Pacific, Inc. IPAT is a conceptual design tool that can provide the physical mass properties of any new launch system that could come off the drawing boards during the next 20 years... | 2007 | Models |
| Update on the Jet Propulsion Laboratory Parametric Mission Cost Model (PMCM) | Erik Monson, Leigh Rosenberg | In today's dynamic, fast-paced world of unmanned space exploration, priorities can, and often do, change rather quickly. A single breakthrough discovery can quickly shift community focus to an entirely different direction. Scientists and engineers involved in the early conceptualization of unmanned missions must have the tools and procedures at their disposal to keep up with such changing priorities, while conforming to strict cost and schedule limitations. Parametric cost estimation is an increasingly important tool for use in the fast-turnaround world of mission studies... | 2007 | Models |
| Life Cycle Cost Normalization for Comparability | Kirk Hoy, Andy Campbell | System concept contractors are increasingly being tasked to formulate life cycle cost (LCC) estimates on their respective offerings. These estimates are often started during the concept development stage of the system acquisition life cycle. From the US Government standpoint, comparability of LCC data among competing vendors and its own internal LCC modeling is a fundamental goal. One way to accomplish this is through early creation and publication of detailed cost groundrules and assumptions that structure the LCC data/estimates being provided to simplify the comparisons and reduce much of the guess work... | 2007 | Models |
| COTS Risk Models: Basic Advice for Avoiding Pitfalls | John Teal | This paper will discuss a few of the common pitfalls to cost risk estimating, especially those caused by junior estimators lack of familiarity with popular risk tools. None of these tips break new ground, apply fresh theories, or even offer advice that is outside of the help section of each tool's operating manual. But nonetheless, these tips will help a new estimator achieve a new level of understanding and insight which will ultimately lead to higher quality and more reliable estimates... | 2007 | Risk |
| Developing Risk Adjusted Results with Limited Data | Gabriela Rohlk | This paper expands on work by Dr. Stephen Book, MCR by addressing additional distributions in performing quick-risk analyses. We provide an overview of the purpose and methodology and then develop the equations for implementation. Given a simple point estimate, this methodology, when combined with some assumptions, allows the decision maker to estimate the high confidence cost/benefits. The methodology is developed for three different assumed statistical distributions (Triangular, Normal & Log-Normal). Additionally a high-level discussion of cost/benefits risk analysis will be covered. | 2007 | Risk |
| Cost Risk as a Discriminator in Trade Studies | Steve Book | Prior to formal program initiation, analysts typically undertake trade studies to investigate which of several candidate architectures or designs can best provide a desired capability at minimum cost. However, the various candidates typically differ significantly in risk as well as in cost, but members of the Government or industry trade- study team do not have the time, and the candidate solutions usually aren't sufficiently detailed at this stage, to conduct a thorough risk analyses. Yet, those differences in risk, as well as in cost, should be taken into account to the extent possible during the trade- study decision process... | 2007 | Risk |
| Cost-Risk Analysis of Satellite Bandwidth Services | Sam Bresnahan | The purpose of this paper is to demonstrate application of basic risk- analysis techniques to a real-world cost estimating problem. Each year the U.S. Marine Corps must budget millions of dollars for satellite bandwidth services. Communication via satellite is frequently required during operations in theater, disaster relief, and any situation in which a secure communication infrastructure is not already present. The USMC is developing an Expeditionary Command and Control Suite (ECCS) that allows small teams of soldiers to establish secure satellite voice and data links with headquarters... | 2007 | Risk |
| A Cost Estimation Risk Assessment Process Implementing Lurie-Goldberg's Algorithm for Generating Correlated Vectors of Random Numbers | Paul Wetzel | A true cost estimate is not a single value. A true cost estimate is a probability distribution of anticipated cost. There are many high-quality cost estimating tools in the marketplace that provide "point solution" estimates, but there are few tools that perform a rigorous risk assessment sufficient to develop an accurate probability distribution of estimated cost... | 2007 | Risk |
| Cost Risk Allocation Objectives, Tendencies, and Limitations | John Sandberg | Allocating cost risk for building a budget is sweeping through the cost estimating community faster than cost estimators can keep up. Few understand the goals, behavior, implementation, or limitations of results generated by popular risk allocation methods. This paper provides perspective on the issue by going beyond the basic mechanics to examine general tendencies, trade-offs, and objectives of cost risk allocation heuristics with focus on the quantification and verification of the allocated results. Some important limitations of existing heuristics are discussed, including the inability to capture program priorities, schedule impacts, and contract vehicles, followed by recommendations on how to capture these in future allocation methods... | 2007 | Risk |
| Examination of Functional Correlation And Its Impact On Risk Analysis | Alfred Smith | The term functional correlation is often mentioned in papers and presentations, but the definition of the term remains nebulous enough to confound even senior analysts. Yet, functional correlation often appears in even the simplest parametric cost models. Cost estimate methods that share the same uncertain inputs or are simply a factor of another cost method are examples of functional correlation. For example, using the same labor rate (with uncertainty assigned) across many different work breakdown structure (WBS) elements causes these WBS elements to be correlated, intended or not... | 2007 | Risk |
| Taking a Second Look: The Potential Pitfalls of Popular Risk Methodologies | Eric Druker, Richard Coleman, Christopher Leonetti, Peter Braxton | As risk analysis has come into its own as a specialized field under the larger umbrella of cost estimating, certain practices and methodologies have been developed and generally accepted by the community at large. The creation of more standardized methodologies has opened risk analysis to a much larger population and skill set than previously allowed. Unfortunately, without a certain level of scrutiny, these accepted methods can lead to results that are not in accordance with the spirit of the methodology. In some cases, the combination of methodologies can even lead to results that are mathematically unsound... | 2007 | Risk |
| Global Economy " Global Risk | Bethia Cullis, Christopher Leonetti, Eric Druker, Crystal Hauser | As we progress towards an increasingly global economy, proposals are more and more frequently priced in multiple currencies. Exchange rates are not only difficult to predict, the risk associated with these predictions compounds the further into the future the predictions are made. In a recent proposal assessed by the authors, ~1/3 of the total proposed price was initially estimated in pounds (GBP.) The proposal team then used projected foreign exchange rates (a single point estimate per year of the period of performance) to determine the price in dollars... | 2007 | Risk |
| To support a Make/Buy Business Case to Successfully Respond to an Inspector General (IG) Audit | Savita Choudhry | This document provides an overview of the service PRICE Systems has provided to a Federal Program Agency to support a major IT modernization Project, which helped the Federal Agency pass an Inspector General (IG) audit. The selected case study was a Make/Buy analysis. Because the case study involves an analysis conducted to respond to an IG audit, the paper will review the methodology and steps undertaken by both the government and contractor from completing and delivering the estimate, analyzing the benefits of the COTS implementation scenario, answering specific detailed questions to the audit, to receiving Program Office acknowledgement of the tremendous value of the Cost Estimation (in accordance with GAO guidelines)... | 2007 | Software |
| Simple is Not Necessarily Better: Why Software Productivity Factors Can Lead to Bad Estimates | Mike Gallo, Paul Hardin | The use of simple linear productivity factors to estimate software development costs is rampant in DoD. Although these factors are easy to explain and use and are (usually) grounded on historical data, they nevertheless contain several cost estimating risks including:... | 2007 | Software |
| Software Code Growth: A New Approach Based on Analysis of Historical Actuals | Robert Jones, Paul Hardin | There are numerous methods available for estimating the growth in software source lines of code (SLOC). Most simply increase New SLOC, Modified SLOC, and Reuse SLOC by growth factors regardless of the characteristics of the SLOC or the distribution of New, Modified, and Reuse... | 2007 | Software |
| Software Development Schedule Realism: Prediction Band Tool | Alison Converse, Eric Druker, Jeff Jaekle | Knowing the probability of achieving an originally proposed schedule, as well as updated schedule proposals throughout a contract, provides decision makers insight when considering schedule changes, risk mitigation plans, funding and other decisions. The purpose of a schedule realism tool is to provide this insight by determining the probability that the software development schedule proposed will be met, as well as information about the distribution of the proposed schedule. ... | 2007 | Software |
| System of Systems Cost Estimation Solutions | Josh Wilson | An area in which several advocates are attempting to improve industry understanding of system of systems relates to estimating the cost and effort required to deliver capabilities rather than platforms. Currently, the significant offerings in this and closely related realms include COSYSMO, COSOSIMO, SECOST, SEER-H with TSV, PRICE PES, and PRICE TruePlanning Systems. This paper explores the history of each of these models, the origin of their methodologies and cost estimating relationships, the major size and cost drivers, and the extent to which each solution addresses the need for a system of systems estimating capability... | 2007 | Software |
| Software Estimation Through The Use of EV Data | Jeff Jaekle, Justin Greene, Eugene Cullen, Eric Druker | Service-Oriented Architecture (SOA) appears to be the next generation of IT architecture. The technology is growing rapidly in both commercial and government markets, with new products being delivered almost monthly. The Director of National Intelligence (DNI) is pushing for SOA implementations of enterprise architectures within the Intelligence Community. Recently, the DNI's Chief Information Officer released strategy and security documentation, as well as a reference model, for SOAs. Although the community is aware that SOAs are the way of the future, little is known about the costs associated with this implementation and subsequent maintenance... | 2007 | Software |
| Understanding Service Oriented Architectures and Their Costs | Kevin Snyder, Janelle McDonald | Service-Oriented Architecture (SOA) appears to be the next generation of IT architecture. The technology is growing rapidly in both commercial and government markets, with new products being delivered almost monthly. The Director of National Intelligence (DNI) is pushing for SOA implementations of enterprise architectures within the Intelligence Community. Recently, the DNI's Chief Information Officer released strategy and security documentation, as well as a reference model, for SOAs. Although the community is aware that SOAs are the way of the future, little is known about the costs associated with this implementation and subsequent maintenance... | 2007 | Software |
| Estimates and Measurement for a Software Product Line | David Seaver | In today's environment where technology rapidly changes, an increasing number of companies, agencies and organizations are realizing that they can no longer afford to develop multiple software products one product at a time. They are being pressured to deploy new products or add functionality to existing ones at a rapid pace. Simultaneously, companies must develop high quality products that minimize the cost to maintain, make effective use of very limited resources, compensate for an inability to hire, and achieve mass customization... | 2007 | Software |
| A Process for Mapping COCOMO Input Parameters to True S Input Parameters | James Otte | Oversight review panels for major programs require software cost analysts to complete software estimates using several software different cost models. This requires the software cost analysts to collect data for various cost models. This paper will provide a comparative survey and analysis using NASA flight projects. The paper is an extension of the Ray Madachy, Barry Boehm and Danni Wu paper completed by USC on the same subject. This paper provides a road-map for mapping COCOMO input parameters to True S input parameters. This will enable a software cost analyst to complete two or more software model estimates using one set of input parameters. | 2007 | Software |
| Software Total Ownership Costs Development is Only Part of the Equation | Daniel Galorath | Software development is a costly, often schedule driven activity, prone to compromises to meet schedule. Many of these compromises have far reaching impacts on the cost of software maintenance, total ownership costs and software sustainability. Maintenance itself is often treated as a level of effort activity where insufficient staff are asked to do too much and where a desired level or maintenance rigor is impossible to preserve. Unlike hardware, software doesn't wear out but with inadequate maintenance it just becomes functionally less useful and less reliable over time... | 2007 | Software |
| Applying parametric cost modelling to Technology Insertion | Arthur Griffiths, Dale Shermon | The paper will describe the UK MOD need to develop a Whole Life Cost modelling tool to enable the exploration and justification of up-front investment in different acquisition scenarios for characteristics that facilitate subsequent Technology Insertion. This paper will describe a case study regarding how to apply parametric cost modelling to the attributes which are associated with Technology Insertion. The case study applies this modelling to an Unmanned Combat Air Vehicle (UCAV), although it is equally applicable to all generic system types... | 2007 | Software |
| Understanding the Sensitivity of PRICE and SEER to the Software Development Project CSCI Architecture | Abbey Turnau | Software development effort is often estimated using commercial tools such as PRICE and SEER. Obtaining accurate estimates is an important key to program planning and control. Size inputs are organized into Computer Software Configuration Items (CSCIs) and Computer Software Components (CSCs) typically based on the architecture designed by the systems engineering team. Often times when a third party is performing an independent cost estimate, access to information at the CSCI level is not provided. Without this information, the third party analysts must make assumptions about the architecture... | 2007 | Software |
| Making Risk Management Tools More Credible: Calibrating the Risk Cube | Richard L. Coleman, Jessica R. Summerville | The so-called Risk Cube is arguably the most widely used tool in Risk Management (RM) today. The method involves creating a risk register of all foreseeable risks. The attendant register is intended to give risk managers and program offices a tool to identify and manage risks to allow mitigation. The quantitative version of this method assigns a probability of failure (Pf) and a consequence of failure (Cf) to each risk. The qualitative version uses some measure of 'likelihood' and consequence which are usually low, and high or ordinal scales. This paper will discuss an improvement to the qualitative method. In reviewing the qualitative method the authors have observed three general shortcomings. First, many programs, despite having the means at their disposal, do not roll up the risk register to determine the expected outcome in total. This is easily remedied and if Monte Carlo methods are used (which will be briefly explained in the paper) a distribution of outcomes can be produced. Secondarily, many programs fail to allow for any 'unknown unknowns' which are well established as necessary in risk literature; accordingly, they yield an unreasonably low expected value... | 2006 | CEBoK Reference 2006 |
| How Age Affects Operations and Support Costs Differently Across Platforms | Sarah Grinnell, Richard L. Coleman, Jessica R. Summerville | This paper expands on work previously published in Ship Scheduled Overhaul Costs Over Time (J.R. Summerville, R.L. Coleman, M.E. Dameron, S. D. Leach, SCEA 2003) and the more recent installment, Operating & Support Cost Over Time (S.E. Grinnell, J.R. Summerville, M.E. Dameron, R.L. Coleman, SCEA 2004). The first paper was ground-breaking research that showed that age does indeed have an effect on ship Operations and Support (O&S) costs. In the latter paper, the research was further refined to show that the O&S cost for a ship goes up over time and then, because the number of overhauls and repairs decrease at the end of a ship's life, the cost decreases in the last few years of a ship's life. The discussion in this paper will focus on more recent research that demonstrates there is also an age effect in aircraft. This discovery is important for a couple of reasons. First of all, it confirms that age is a cost driver for platforms other than ships... | 2006 | CEBoK Reference 2006 |
| Analogies: Techniques for Adjusting Them | Richard L. Coleman, Jessica R. Summerville, Shishu S. Gupta | The three basic methods of cost estimation are generally accepted to be Parametric, Analogy and Build-up. Considerable attention is devoted to research in parametric methods, generally in higher-level and oversight organizations, and for early studies. Likewise, great expertise is resident in manufacturing sector in Build-up methods. The technique of analogy receives surprisingly little attention in research or in development of methods. Practitioners are enjoined to 'find the best match and adjust it if necessary. The issue of what exactly constitutes a best match and how to adjust are left to the practitioner. This paper will discuss the usual method, linear adjustment based on a single parameter such as weight or size (often called 'J-ing up'), and will recommend two other methods as preferable to straight linear adjustment: one which borrows a slope from a cost estimating relationship, when one exists, and the other that develops a factor from the geometry of correlation, and which arises from a previous paper by the authors. The reasons for these methods being preferable will be presented, and an illustrative case study with actual data will be shown. The problem of finding a best match is not trivial and is not solved in this paper, but it will be discussed briefly for completeness. | 2004 | CEBoK Reference 2004 |
| Rayleigh Curves: A Tutorial | H. Chelson, R. Coleman, J. Summerville, S. Van Drew | 2004 | CEBoK Reference 2004 | |
| How to Make Your Point Estimate Look Like a Cost-Risk Analysis | Stephen A. Book | Until recently, sponsors and managers of space-system and other high-technology systems under consideration expected cost estimators to provide "point" estimates of costs of various architectures and/or designs at each program decision point, from the initial trade-study stage to source selection and right through to the completion of the project. Unfortunately, the "point estimate" was never precisely defined as the term never meant the same thing to everybody. For example, was it the "most likely" cost, the 50% confidence cost, the "average" cost, or what? | 2004 | |
| Predicting Final CPI | R.L. Coleman, M.E. Dameron, J.R. Summerville, H. F. Chelson, S. L. Van Drew | Estimating the EAC based on current performance has traditionally been a point estimate or, at best, a range based on different EAC calculations (CPI, SPI, CPI*SPI, etc.). NAVAIR is in the midst of revising their EVM Toolkit, which incorporates the formation of an EAC. This paper provides the EVM analyst with a predicted Final CPI (and thus EAC), and also provides a confidence interval around the predicted value. This will enable analysts to determine where the program EAC falls on the cumulative probability distribution and to calculate the likelihood of achieving a favorable Final CPI (e.g., the probability of a Final CPI of 1.0 or better). The rule of thumb that EACs never improve over their values at 20% complete is analyzed and found to be generally true, but with some exceptions. ... | 2003 | CEBoK Reference 2003 |
| Schedule and Cost Growth | R.L. Coleman, J.R. Summerville, M.E. Dameron | Schedule and Cost Growth | 2002 | PMI 2002 National Conference |
| Relational Correlation: What to do when Functional Correlation is Impossible | Jessica R. Summerville, Megan E. Dameron, Cari L. Pullen, Richard L. Coleman, Donna M. Snead | This paper provides a significant breakthrough in the application of correlation in risk and cost estimation. The paper will supply the missing piece not handled by the procedure called Functional Correlation, put forward by the authors in 1994, and will show the unseen errors generated by other methods of correlation injection. The paper will show how to use the geometry of regression to determine the correct values of correlation and other variables in risk. Powerful and reliable methods of approximating the values of needed parameters will be suggested, and finally, possible applications to other areas of cost estimation will be suggested. | 2001 | CEBoK Reference 2001 |
| NAVAIR Cost Growth Study: A Cohorted Study of The Effects of Era, Size, Acquisition Phase, Phase Correlation and Cost Drivers | Jessica R. Summerville, Megan E. Dameron, Cari L. Pullen, Richard L. Coleman, Donna M. Snead | The goal of this study was to further analyze Navy programs to determine the cause of cost growth (technical or cost estimating) and to develop a concept for estimating future cost growth. This understanding will allow program offices to plan for cost growth (adjust cost estimates and budget with cost risk dollars) and/or mitigate the high risks in the program to minimize cost growth... | 2001 | CEBoK Reference 2001 |
| Do Not Sum Earned-Value-Based WBS-Element Estimates-at-Completion | Stephen A. Book | EACsare Random Variables, Not Deterministic Numbers, and Must be Handled Statistically 'Avoid Meaningless, Contradictory Outcome of Roll-up Procedure 'Get Mean, Median, Mode of Total-System Cost at Completion, as well as All Cost-at-Completion Percentiles | 2000 | CEBoK Reference 2000 |