Posted by Simulation-based Experimentation: How to Scope and Estimate Development Projects
Models Track
Downloadable Files:
Abstract:
Simulation-based Experimentation consists of designing, developing, and executing an experiment using computer simulations representing real systems, with or without operators-in- the-loop. Estimating simulation-based experimentation differs from estimating an acquisition program which includes distinct development, production, and operations and support phases. The end result of simulation-based experimentation is an experimentation environment and results of analysis to address both internal and external customer concerns. To improve estimation of the cost of a simulation-based experimentation project, we developed an experimentation cost estimating methodology and toolkit.
Several approaches to analyze the experimentation project data were considered based on the type and amount of data collected. Each approach was evaluated and noted with Pros and Cons, as well as limiting factors of the approach. In particular, we focused on Linear Regression and Random Forests. We developed models with each approach, evaluated the fit and prediction capabilities, as well as cross-validated the models with test results.
This presentation introduces Simulation-based experimentation, describes the methodology employed, and presents highlights, concerns and results of the analytical approaches.
Author(s):
Karen Mourikas
The Boeing Company
Karen Mourikas is an Operations and Systems Analyst at The Boeing Company in Huntington Beach, CA. Currently, she supports the Strategic Development and Experimentation (SD&E) group in Phantom Works focusing on Experimentation. In the Experimentation group, Karen manages Space Situational Awareness experiments, from initial development through execution and analysis, in the Space Experimentation & Analysis Center (SEAC) in Seal Beach. Previously, Karen worked as an Affordability Analyst in the Systems Engineering Affordability group. In that role, she worked on various programs estimating Life Cycle Costs, analyzing Cost Uncertainty, integrating Cost Risk with Risk Management processes, and performing Cost Effectiveness Trades. Combining Experimentation and Affordability, Karen proposed and manages the development of the simulation-based experimentation estimating toolkit. Karen holds a BA in Mathematics and Computer Science from Connecticut College, and two MS degrees from the University of Southern California, one in Applied Mathematics and the other in Operations Research Engineering.
Denise Nelson
The Boeing Company
Denise graduated from Cal Poly Pomona with a BS in Statistics and a MS in Pure Math.
Upon graduation she taught college math. Denise initially became employed with the Boeing Company as a Parametric Estimator and later joined Boeing’s Research and Technology (BR&T) division as a System Engineering specializing in Affordability. She works multiple projects and programs with the bulk of her experiences being GPS related. Denise applies her teaching experience by developing and/or instructing Boeing designed courses. She actively participates on the training committee which is implementing a standard Affordability curriculum within across all Boeing sites.