2021 ICEAA Online Workshop – International Cost Estimating and Analysis Association

ICEAA’s live 2021 Online Workshop was a tremendous success! Registered attendees can log on to the Workshop platform until November 15, 2021 to view recordings of the sessions they missed or sessions they’d like to watch again using the same links used to log in during the live event.

Sponsors Best Papers

Breakout Sessions Keynote Presentations Speaker Bios

Thanks to our sponsors and exhibitors!

2021 Best Papers

ICEAA’s Best Paper process begins when authors are selected to present at the ICEAA Professional Development & Training Workshop, who may choose to submit a narrative-format paper along with the presentation they plan to give during the event. Those narrative papers submitted by the Best Paper Judging deadline are broken into theme-based categories and the best paper of each category is chosen by a panel of judges who name the best paper per category. Those winners are then rated by all of the judges from the various categories and the highest scoring paper among the winners is named Best Paper Overall.

Soft Skills & Communication Track:
Does Cost Team Leadership Matter
Christina N. Snyder

An anonymous survey of 150+ cost analysts unanimously reported that a cost-team lead’s effectiveness ultimately impacts the team’s products. However, there has been minimal guidance as to what defines good leadership. Using the ten behaviors identified by Google’s Project Oxygen, this paper seeks to understand what skills are necessary for successful cost leadership. The findings lead to a simple conclusion that mirrors that of Project Oxygen: improving our soft skills will improve cost leader efficacy.

Machine Learning & Data Science Track:
Machine Learning and Parametrics for Software Joint Confidence Level Analysis
Sara Jardine
Christian B. Smart
Kimberly Roye

Joint Confidence Level analysis has proven to be very successful for NASA. It is typically conducted using bottom-up resource-loaded schedules. However, the use of high-level parametrics and machine learning has been successfully used by one of the authors. This approach has some advantages over the more detailed method. In this paper, we discuss the approach and provide an example of the application of machine learning and parametric analysis to software programs.

2021 Best Paper Overall:
How Green Was My Labor: The Cost Impacts of Manufacturing Personnel Changes
Brent Johnstone

Estimators are frequently confronted with manpower increases or decreases and asked to calculate shop performance impacts. However, existing learning curve literature offers little guidance how to do so. This paper identifies issues associated with both new hires (so-called “green labor) and workforce reductions and offers an analytical format. Based on a study of a large workforce expansion on a mature aircraft program, a model to analyze future manpower changes is presented as well as several example cases.

Keynote Presentations

A Fool with a Tool is Still a Fool: What’s Your Value Proposition?
The ‘best’ cost analysts recognize that their potential value extends well beyond the confines of a particular program. Sure there is a tremendous sense of satisfaction in contributing to creation of realistic budgets via a credible program or independent cost estimate. Sure you are proud of a proposal evaluation resulting in a contract value that maximizes the probability of stakeholder success. Sure you feel great about your contract cost and schedule performance analysis translating to the best estimate at completion (EAC) this side of the Mississippi. But there are larger ‘problems’ that cost professionals are capable of solving if willing and able to leave their comfort zone and grow!

Panelists:

Cheri Cummings, DISL
Business Analysis and Optimization Chief
United States National Security Agency (NSA)

Ms. Cummings is the Business Analysis and Optimization Chief (B4) within the Business Management & Acquisition (BM&A) Directorate. Her organization provides cost analysis, data analytic, lean six sigma and change management support to the entire Agency. Specifically, she leads the organization that develops the Agency’s cost position on Major System Acquisition (MSA) programs and Life Cycle Cost Estimates (LCCEs) for all Tier I/II and Special Interest programs. She serves as the Agency’s focal point for the Basis of Estimate (BOE) and Earned Value Management (EVM) programs. She serves as the BM&A data steward, oversees reporting and data modeling efforts and works closely with the BM&A TD to facilitate the advancement of Business IT solutions. She is also responsible for conducting special studies, financial modeling and simulation, predictive analysis, and other data science driven efforts which underpin the agency’s fact-based resource and investment decision making process. Finally, her organization provides structured methodologies, training and expert knowledge to support and sustain continuous process improvement throughout the agency. As a member of the BM&A leadership team, Ms. Cummings has served on the agency’s DISL board for the last four years and currently co-chairs one of the Director’s Big 6 Focus Areas (Developing a Pipeline of Diverse Senior Candidates).

Ms. Cummings began her career as a participant in the Navy Sea Systems Command Cost Analysis Intern Program within the Surface Ship Combat System Division. She progressed up the managerial chain and eventually accepted a position at the Naval Center for Cost Analysis (NCCA). After leaving NCCA, Ms. Cummings held positions at both the National Geospatial-Intelligence Agency (NGA) and the Missile Defense Agency (MDA) where she gained further commodity expertise in geospatial imagery, satellite and missile defense systems.

Ms. Cummings received the Meritorious Professional Presidential Rank Award for her work in BM&A, the Director’s Joint Meritorious Unit Award while in PA&E and the Navy Superior Civilian Service Award.

Ms. Cummings is Defense Acquisition Workforce Improvement Act (DAWIA) Level III certified in Business, Cost Estimating and Financial Management and is a member of the Acquisition Corp. She completed the Senior Leadership Development Program and the Leading the Intelligence Community Development program. Ms. Cummings holds a bachelor’s degree in Industrial Engineering from North Carolina State University.

Ms. Cummings resides in Lanham, Maryland, with her husband Wayne. They have a daughter and a son.

Show less

Katie Geier Noreiga
Director, Cost Analysis Division, Office of the Chief Financial Officer, Management Directorate
United States Department of Homeland Security (DHS)

Katie Geier Noreiga is the Director of the Cost Analysis Division within the Office of the Chief Financial Officer. In this role Ms. Noreiga oversees the cost analysis function for the Department of Homeland Security. She is responsible for providing objective cost estimates and cost analyses to enable DHS acquisition and resource decision making. This includes principally advising the Under Secretary for Management, Chief Financial Officer, and Component Acquisition Executives on cost estimating and analysis issues; leading the Department in cost estimating and analysis policy, policy implementation, and guidance; developing Independent Cost Estimates (ICEs) and Assessments (ICAs) for major acquisition programs to inform the Acquisition Decision Authority; and advising the Homeland Security Acquisition Institute (HSAI) on career field certification, training, and education for the cost estimating and analysis community. Ms. Noreiga is also responsible for the development and implementation of a department-wide business intelligence tool that provides a centralized and authoritative view of the DHS major acquisition program portfolio.

Ms. Noreiga has been with DHS since 2009 serving in various roles throughout the acquisition community. Prior to joining DHS, Ms. Noreiga worked for the Navy Engineering Logistics Office for six years where she obtained vast experience estimating weapon systems for various platforms. Ms. Noreiga holds a B.S. in Industrial Engineering from The Pennsylvania State University (2002) and a M.B.A. from The George Washington University (2007). She is certified Level III for the Business, Cost Estimating and Financial Management career field. Ms. Noreiga resides in Davidsonville, MD with her husband and two daughters.

Show less

Ranae P. Woods
Technical Director for Cost and Economic Analysis Research
Air Force Cost Analysis Agency, Deputy Assistant Secretary for Cost and Economics, Office of the Assistant Secretary of the Air Force for Financial Management and Comptroller

Ranae P. Woods, a Senior Level executive, is the Technical Director for Cost and Economic Analysis Research, Air Force Cost Analysis Agency, Deputy Assistant Secretary for Cost and Economics, Office of the Assistant Secretary of the Air Force for Financial Management and Comptroller, Washington, D.C.

As the senior Air Force cost and economics technical leader, she provides strategic and professional leadership in the development of highly complex major weapon system program cost estimates, studies, and models, influencing multi-billion dollar Air Force acquisition, budget and programming decisions. She directs the Air Force wide cost research program to improve cost and economic analysis databases, methods and tools, and is responsible for the creation of policy, regulations, standards, guidance, and training to enhance workforce skills. Ms. Woods began her career as a cost analyst and industrial engineer in the Navy Cost Analysis Intern Program for the Naval Center for Cost Analysis where she supported the development of cost estimates for a wide variety of Navy weapon systems. Since 1993, she has held a wide variety of cost analysis positions within the Air Force as well as a career broadening position within the Office of the Deputy Chief of Staff, Strategic Plans and Programs. Ms. Woods has a Bachelor of Science in industrial engineering from Pennsylvania State University, a MBA from Virginia Polytechnic Institute and State University, and a Master’s degree in National Resource Strategy from the Industrial College of the Armed Forces (ICAF). She is a Department of Defense certified acquisition level III professional in business cost estimating.

Show less

Moderator:

Rick Collins
Technical Director
Technomics, Inc.

Rick Collins is Technical Director of Technomics, Inc., an employee-owned consulting firm offering a range of decision analytics services to U.S. and foreign government clients. He assumed this role in 2017 after serving as President and/or CEO for 13 years.

Rick has 40+ years cost analysis experience, including 29 years in the private sector (nine at SAIC in the 80s and nearly 20 at Technomics) and 11 years in the Department of Navy (DON), specifically the Naval Center for Cost Analysis (NCCA). His experience encompasses a varied mix of applied cost estimating and cost research projects covering most DOD weapon system commodities and all Military Departments.

Since establishing Technomics’ Arlington, VA office in June 2000, Rick has strived to create an organization recognized for technical excellence, integrity and innovation. He is particularly proud of Technomics’ and his personal efforts over the last 17 years to assist OSD CAPE in: a) compiling and validating contract cost data for all of DOD’s Major Defense Acquisition Programs and b) making cost, technical and programmatic data and visualization of the data readily accessible to the DoD acquisition community via the Cost Assessment Data Enterprise (CADE). The principal product of this work – reliable cost data for completed contracts – will provide future generations of cost analysts with the key ingredient for evidence-based cost analysis.

During his tenure at NCCA in the 90s, Rick served in various positions, including: Technical Director, Head of the Ships/Ship Systems Division and Head of the Cost Analysis Tools Division. He was awarded the DON Superior Civilian Service Award for his accomplishments coordinating DON cost research. While at Science Applications International Corp. (SAIC) in the 80s, Rick had the opportunity to learn the art and science of cost analysis from talented senior cost analysts in the course of performing a wide variety of cost research projects for DON clients.

Rick has served as President of the International Cost Estimating and Analysis Association (ICEAA) since July 2019. He served ICEAA twice in two year At-Large and Regional Board Director positions. He was the principal architect of ICEAA’s Strategic Plan that was adopted and serves as a compass for BOD decision-making. He has played an active role in Annual ICEAA/SCEA Workshops since 2008 by chairing paper tracks, serving on the best paper evaluation committee, presenting technical papers and teaching the Cost and Software Data Reporting (CSDR) s at the 2008-2013 conferences. He co-authored and presented “Competition in DOD Systems Acquisition: Past Lessons and Future Considerations,” judged a best paper in 2010. He was awarded the SCEA National Estimator/Analyst of the Year Award for Contributions in Management in 2010.

Rick has a M.A. and B.A. in Economics from Virginia Tech and Wake Forest University, respectively.

Show less

An Interview with the Estimator
I started off with the aim of understanding what makes a cost estimating professional. How did you enter the field, how do you help decision-makers, and where do you see the profession going in the future? The format is simple, not a webinar, but more personable. Now, more than 20 episodes on, and with participants from Australia to the United States, I have learned a few things about the cost estimator’s experience. In this presentation I will talk about some of the common themes from the interviews so far.

Dr. Amritpal Singh Agar
Host of The Cost of Everything Podcast

Amritpal S. Agar hosts the Cost of Everything Podcast and YouTube channel. He has worked for more than 10 years in the water and nuclear industries. He is currently the Cost & Estimating Manager for The Canal And River Trust, a charity which manages over 2000 miles of waterways in England and Wales. Amritpal has a passion for understanding how decisions are made using cost information, holding an engineering doctorate from Loughborough University. In his spare time Amritpal likes to spend time running long distances, and walking his dog, Sonny.

Show less

Implications of Accelerated Acquisition for Cost Analysis: What Are We Learning?
There have been many flavors of DoD acquisition reform initiatives over the years. Some have achieved success, others haven’t. Some have been declared dead and risen from the ashes under a different name in hopes that the second (or perhaps third) time is a charm. Regardless of a particular initiative’s eventual success or failure in relation to acquisition program outcomes, the cost analysis community will have done its due diligence to understand and quantify the potential cost implications of the initiative. Accelerated acquisition has provided yet another ‘opportunity’ for our community to temper decision maker expectations via objective, data-driven analysis!

Panelists:

George Barbic
Director, Pricing and Estimating, Program Assessment & Evaluation
Lockheed Martin Corporation

George Barbic is the Director of Pricing and Estimating for Lockheed Martin Corporation. Primary areas of responsibility include current cost estimating processes, practices and tools along with the corporate wide, future transformation of estimating. He is also the principal government, customer, audit agency and industry interface associated with estimating and estimating transformation initiatives.

Mr. Barbic is responsible for the integrated Lockheed Martin Corporate Estimating Council (CEC). In this capacity, he provides direction and day to day management of key elements associated with processes, products, and tools including capability, development and implementation as it relates to current and future estimating states. The objective of CEC is to share best practices and lessons learned, reduce resources and cycle time, increase accuracy and improve products in support of proposal and program cost estimating. He is also responsible for affordability initiatives associated with a data driven / fact-based approach to estimating.

Mr. Barbic was the driving force for industry in the establishment of several joint industry and government cost working groups such as Joint Space Cost Council (JSCC), Aviation CIPT, OSD CADE, NASA CIPT and Navy CIPT. In addition, he has established working relationships with the OSD CAPE, ODNI CAIG, NRO CAIG, AFCAA, SMC, NCCA, NASA, DCAA and DCMA.

Mr. Barbic has been with Lockheed Martin for over 37 years and has been in his current role for the past 12 years. He is the recipient of several Team Excellence, Business and Process Improvement, and New Business awards along with multiple individual achievement awards.

Show less

Dave Burgess
Corporate Director of Independent Cost Evaluations
Northrop Grumman Corporation

Dave Burgess joined Northrop Grumman Corporation in July of 2014. In his role as Corporate Director of Independent Cost Evaluations, Dave is responsible for leading all major independent cost and business management evaluation reviews across Northrop Grumman: identify and evaluate proposal risk issues in cost estimates, terms and conditions, and all other business management related functions; approve and present Independent Cost Estimate (ICE) analyses submitted in support of risk reviews; facilitate company-wide sharing of ICE best practices, process improvements, lessons learned, and consistent risk quantification.

Prior to joining Northrop Grumman, Dave worked for the Department of the Navy for over 37 years, in various capacities mainly related to cost estimating and analysis. His last position was as the Director of the Naval Air Systems Command’s Cost Analysis Department. In this role, Dave was responsible for the planning and execution of all life cycle cost estimating, cost analysis, and Integrated Project Management (IPM) activities for Naval Aviation programs.

Specific Accomplishments:
• Revolutionized the way the Navy views and tracks program performance
• Clearly defined the linkage between program planning and program success
• Successfully staffed a cross-functional team, referred to as the Program Performance Team, with highly-skilled analysts from across the Command to examine acquisition program data and relate to future actions that would improve program execution
• Examined root causes of program performance problems and provided factual data for PEO/Program Management interaction with industry to improve program performance
• In support of ASN RDA, lead the development of the executive education course at UNC Chapel Hill entitled “Understanding the Government-Industry Relationship in Acquisition”

Show less

Frederick “Fred” Janicki
Director, Space and Strategic Systems Cost Analysis Division
Cost Assessment and Program Evaluation

Mr. Fred Janicki, a member of the Senior Executive Service, currently serves as the director of the Space and Strategic Systems Cost Analysis Division, OSD Cost Assessment and Program Evaluation (CAPE). He leads a team of senior operations research analysts in developing independent cost estimates, resource analyses, and special studies for space, strategic and various classified weapon systems. In addition, Fred directs the Cost Assessment Data Enterprise (CADE), which provides cost estimating databases, tools, and cost estimating research to the Department’s cost analysis, contracting, and program management community.

Prior to his current position, Fred was the director of the Weapon Systems Cost Analysis Division, OSD CAPE, where he directed the development of independent cost estimates and special studies for large and complex weapon systems including submarines, ships, helicopters, ground vehicles, tactical and bomber aircraft, weapons and various classified systems. In 2008 and 2009, Mr. Janicki lead the first F-35 Joint Estimating Team (JET), which included 28 technical experts from the Navy, Air Force, and OSD, tasked with assessing the F-35 program to support decision making by the most senior leadership of DoD. As a direct result of Mr. Janicki’s JET recommendations, the Secretary of Defense announced a major restructuring of the F-35 program in conjunction with the submission of the 2011 President’s Budget. As a result of his work, Mr. Janicki was invited to testify at a closed hearing of the Senate Armed Services Committee on the F-35 program schedule and costs.

Before joining OSD CAPE in 2003, Fred spent 12 years at the Naval Air Systems Command (NAVAIR) as a senior cost team leader responsible for cost and schedule estimates, resource analyses for a myriad of aircraft weapon systems including the F-14, A-6, EA-6B, H-1, and E-2D aircraft. In his last assignment for NAVAIR, as the affordability director responsible for technical direction of all cost and economic activities for the F-35, he led the cost evaluation for the largest source selection (in terms of resources) in the Department’s history.

Fred holds an undergraduate degree in Mechanical Engineering from the University of Rhode Island and a Masters in Engineering Administration from Virginia Tech. Mr. Janicki has received numerous honors from government and industry including the Presidential Rank Award, the DoD Medal for Distinguished Civilian Service, the Secretary of Defense Meritorious Civilian Service Award (2002 & 2013), and the Society of Cost Estimating and Analysis (SCEA) National Cost Estimator of the Year for technical achievement.

Show less

C. Grant McVicker
Deputy Assistant Secretary for Cost and Economics
Office of the Assistant Secretary of the Air Force for Financial Management and Comptroller

Mr. C. Grant McVicker III is the Deputy Assistant Secretary for Cost and Economics, Office of the Assistant Secretary of the Air Force for Financial Management and Comptroller, Washington, D.C. Mr. McVicker is responsible for directing all Air Force cost and economic analysis programs. He is also the Executive Director of the Air Force Cost Analysis Agency, Chair of the Air Force Cost Review Board and serves as the Chair of the Investment Subcommittee of the Air Force Personnel Centers Board of Directors.

Mr. McVicker retired from active duty in October 2006. He initially served as a Missile Combat Crew Commander and then transitioned into the Acquisition Financial Management Career field in 1990. There he served in a variety of Program Office and staff positions. Specifically, Mr McVicker led the F-22 Production Financial Management Branch and was the Director of Affordability for the F-35 Joint Strike Fighter. Prior to Mr. McVicker’s placement as an SES, he served as the Chief of the Aircraft and Weapons Division, Air Force Cost Analysis Agency. In this role, he was accountable for the development and defense of the Air Force’s cost estimates for major defense acquisition aircraft and weapon programs presented at Defense Acquisition Executive reviews.

EDUCATION
1985 Bachelor of Science, Corporate Finance, West Virginia University, Morgantown, WV
1989 Master of Science, Business Administration, University of Montana, Missoula, MT
1993 Master of Science in Cost Analysis, Air Force Institute of Technologies (AFIT), Dayton, OH
2001 Master of Military Operational Art and Sciences, Air Command and Staff College, Montgomery, AL

CAREER CHRONOLOGY
1. 1986 – 1990, Missile Combat Crew Commander, 490th SMS, Malmstrom AFB, MT
2. 1990 – 1991, Financial Manager, Transports, Systems Division, ASC, Wright-Patterson AFB, OH
3. 1991 – 1992, Financial Manager, Executive Transports, Systems Division, ASC, Wright-Patterson AFB, OH
4. 1992 – 1993, Graduate Student in Cost Analysis, Air Force Institute of Technology, WPAFB, OH
5. 1993 – 1995, Weapon System Cost Analyst, Air Force Cost Analysis Agency, Arlington, VA
6. 1995 – 1997, Senior Weapon Systems Cost Analyst, Deputy Assistant Secretary of the Air Force, Cost and Economics, Pentagon, Washington, D.C.
7. 1997, Chief, F-117A Financial Management, F-117A SPO, Wright-Patterson AFB, OH
8. 1997 – 2000, Chief, F-22 Production Financial Management Branch, F-22 SPO, Wright-Patterson AFB, OH
9. 2000 – 2001, Masters Candidate in Military Operational Art and Science, Air Command and Staff College, Maxwell AFB, AL
10. 2001 – 2004, Director, JSF Affordability IPT, JSF Program Office, Crystal Gateway 4, Arlington, VA
11. 2004 – 2013, Chief, Aircraft & Weapons Division, AFCAA, Crystal Gateway North, Arlington, VA
13. 2013 – 2019, Associate Deputy Assistant Secretary for Cost and Economics, Office of the Assistant Secretary of the Air Force for Financial Management and Comptroller, Washington, D.C.
14. 2019- Present, Deputy Assistant Secretary for Cost and Economics, Office of the Assistant Secretary of the Air Force for Financial Management and Comptroller, Washington, D.C.

AWARDS AND HONORS
Meritorious Civilian Award
Special Acts and Services Annual Award for SDB I & II Financial Management Team
Special Acts and Services Award
Legion of Merit
Defense Meritorious Service Medal
Meritorious Service Medal with two oak leaf clusters
Air Force Commendation Medal with one oak leaf cluster
Joint Service Achievement Medal
Air Force Achievement Medal with two oak leaf clusters
Combat Readiness Medal with one oak leaf cluster
Notable Achievement Award for Global Hawk Should Cost Team
Secretary of the Air Force for Acquisition (SAF/AQ) Acquisition Hero Award
The Maxwell/Gunter Volunteer Excellence and Angel Award
F-22 Senior Level Professional Military
Department of Defense Value Engineering Team Award
F-22 Team of the Year Award
American Society of Military Comptrollers Analysis and Evaluation Award
PROFESSIONAL CERTIFICATIONS
DoD Financial Management/Comptroller, Level III
Business – Cost Estimating, Level III, Acquisition Professional
Financial Management and Comptroller, Level III, Acquisition Professional

Show less

Moderator:

Rick Collins
Technical Director
Technomics, Inc.

Rick has a M.A. and B.A. in Economics from Virginia Tech and Wake Forest University, respectively.

Show less

Cost Estimating in the AI Age
Driven by increases in computing power, Artificial Intelligence and Machine Learning are revolutionizing data-driven industries. This presentation will explore the application of Artificial Intelligence to Cost Estimating with a focus on leveraging data not just to predict costs more accurately, but to provide decision makers with additional insights for developing their strategies. Presenter Eric Druker will provide his unique perspective derived from his decade spent as a cost analyst and current role as a Principal in Booz Allen’s Artificial Intelligence practice.

Eric Druker
Principal
Booz Allen Hamilton

Eric Druker is a Principal at Booz Allen Hamilton where he serves as lead for the St. Louis Business Center and director for Artificial Intelligence in support of the United States Intelligence Agencies. Prior to taking on his current role, he spent 8 years as a cost estimator, creating the Polaris software for Integrated Cost & Schedule Risk Analysis and winning two ICEAA Analyst of the Year Awards. In his spare time, he enjoys hanging out with his wife and three kids and spending time outside practicing Deep Space Astrophotography.

Show less