Posted by Software Maintenance Data Collection and Estimating Challenges
Life Cycle Cost Track
LCC01_Presentation_SoftwareMaintenanceDataCollectionandEstChallenges_Welker
Abstract:
Entering a period of fiscal austerity, it becomes more important than ever to estimate and consider operating and support (O&S) costs, which represent the lion’s share of life cycle cost (LCC) for most platforms, during acquisition. Given the ubiquity of software in today’s complex programs, a key component of O&S is software maintenance. This paper presents the results of a research study co-sponsored by the Air Force Cost Analysis Agency (AFCAA) and the Naval Center for Cost Analysis (NCCA) to collect software maintenance data from government support activities and development contractors to enable high-fidelity cost estimates for software maintenance.
Software maintenance cost estimates have often resorted to apocryphal rules of thumb or “black-box” off-the-shelf models. Rules of thumb might be in the form of “Annual software maintenance cost is such-and-such a percentage of software development cost” or “Each full-time staff person can maintain so many thousands of source lines of code (KSLOC).” Neither method is defensible if they are not readily traceable to the source data (if any) upon which they’re based. It is this acute need for data that prompted the study team to undertake our effort. We collected data from multiple maintenance organizations, both government and contractor, across various platform types, yielding scores of data points, each constituting a maintenance release. We focused on the key parameters needed for estimating, primarily sizing, schedule, effort, and quality data. Where available, we also collected cost, capability, and complexity. We endeavored to follow best practices by: (a) developing a flexible Data Collection Form in Excel, which could be filled in from scratch or easily linked to an existing source; (b) working to understand each organization’s process to provide context to the data; (c) visiting the organization when possible to speak directly with software managers, engineers, and estimators; and (d) documenting which activities were included in effort data.
The paper will present the summary metrics derived from the collected data and discusses important issues that arose, including (1) whether organizations focused on the base code being maintained or the code being changed with each release (more like a “mini-development”); (2) whether organizations focused on defects as a driver of maintenance effort or took a broader view of software trouble reports (STRs) as requested changes; and (3) how organizations accounted for infrastructure costs and sustaining engineering activities. It was evident that maintenance metrics were of increasing importance across all the organizations surveyed, and occasionally that meant that measurement systems themselves were in flux, often due to mergers or benchmarking against sister organizations. Finally, the paper will make recommendations as to the improvement of ongoing software maintenance data collection.
Author(s):
Vanessa V. Welker
Technomics, Inc.
Vanessa Welker (née Virtudazo) is a Cost Analyst at Technomics, Inc. and has ten years experience in the cost analysis and estimation associated with development, production, and operations and support (O&S) of weapons systems. Since joining Technomics in August 2009, she has supported various DoD and Government clients such as the US Navy, Defense Advanced Research Projects Agency (DARPA), Department of Homeland Security (DHS), and Government Accountability Office (GAO). She currently supports Naval Center for Cost Analysis (NCCA) on various research projects. Ms. Welker started her cost analysis career in August 2002 at the Naval Surface Warfare Center (NSWC) Carderock Division through the Naval Acquisition Intern Program where she gained practical experiences in cost estimating through the rotational assignments with various Navy organizations. Throughout her career with the Navy, she primarily supported major defense acquisition program offices. She served as the cost lead for the DDG 1000 composite deckhouse and deputy cost lead to the Sea-Based Strategic Deterrent (SBSD) Analysis of Alternative (AoA).
Ms. Welker earned her Bachelors of Science degree in Industrial Engineering from North Carolina State University in 1999 and her Masters of Engineering degree in Systems Engineering from the University of Maryland in 2004. She is Defense Acquisition Workforce Improvement Act (DAWIA) Level III certified in Systems Planning, Research, Development, and Engineering (SPRDE) and a member of the Society of Cost Estimating and Analysis (SCEA).
Peter J. Braxton
Technomics, Inc.
Peter J. Braxton holds an AB in Mathematics from Princeton University and an M.S. in Applied Science (Operations Research) from the College of William and Mary. A Senior Cost Analyst and Technical Officer at Technomics, Inc., he is a Certified Cost Estimator/Analyst (CCEA) and currently serves as Director, Body of Knowledge, for the Society of Cost Estimating and Analysis (SCEA). He was named SCEAs 2007 Estimator of the Year for contributions in Education and received both a TASC Presidents Award for Operational Excellence and a Northrop Grumman Corporate Contracts and Pricing Award in 2008. He served as SCEAs Training Chair from 2004 to 2009 and as a Northrop Grumman Technical Fellow from 2006 to 2009.
He currently supports the Naval Center for Cost Analysis (NCCA) on data collection and cost research efforts, and Defense Acquisition University (DAU) on curriculum development. Throughout more than a dozen years in the community, he has worked to advance the state of knowledge of cost estimating and risk analysis, Cost As an Independent Variable (CAIV), and Target Costing on behalf of the Navy Acquisition Reform Office (ARO), the DD(X) development program, and other ship and intelligence community programs. He has conducted independent cost evaluations (ICEs) on all of Northrop Grummans major shipbuilding programs (now Huntington Ingalls Industries), and on several IT systems and services programs. He served as managing editor for development and maintenance of the acclaimed Cost Estimating Body of Knowledge (CEBoK(R)) and its predecessor, Cost Programmed Review Of Fundamentals (CostPROF), and as Training Track chair for the last nine SCEA international conferences. He has taught extensively at government, corporate, and society training events throughout the United States, Europe, and Australia. He is lead author or co-author of over a dozen professional papers on cost, risk, and CAIV, including two SCEA Best Paper winners.
Joe Dean
Air Force Cost Analysis Agency (AFCAA)
Joe Dean is the Air Force Cost Analysis Agency (AFCAA) Operating Location Chief at Hanscom AFB in Massachusetts. He spent 19 years with Tecolote Research Inc., as a Senior Cost Analyst and Technical Expert. His focus is on cost estimating Software intensive systems and he applies his skills on everything from submarines to satellites. He is a former member of the Software Engineering Institutes Software Acquisition Metrics Working Group. Joe was the SCEA National Vice President and is now the current SCEA Region 1 VP. He is also retired from the Air Force where he spent the last seven of his 22 year career at ESC/FMC, Hanscom AFB as a Cost Analyst and Cost Research Analyst where he developed the ESC Cost per Line of Code Model. Before that he was a Software Systems Analyst for the Communications Computer Programming Center at Tinker AFB in Oklahoma. Among many other things Joe has a Bachelor of Science degree in Math, a Masters Degree in Business Administration, is a Co-author of, Practical Software Measurement, Objective Information for Decision Makers and enjoys being a reenactor of the Revolutionary War era.
Wilson Rosa
Corinne Wallshein