Vicky Nilsen is an Operations Research Analyst within the Office of the Chief Financial Officer (OCFO) at NASA Headquarters in Washington, DC. In this role, Vicky serves as OCFO's point of contact for cost and schedule analysis, research, and model development. Vicky began her tenure as a civil servant at NASA in March 2022. Prior to this, she has been affiliated with NASA in various ways. In 2017, she was a systems engineer on the George Washington University CubeSat project, sponsored by NASA's CubeSat Launch Initiative. From 2018-2020, she performed academic research for the Mission Design Lab at NASA's Goddard Space Flight Center and Team X at the Jet Propulsion Lab. In 2019, she was an intern in OCFO's Portfolio Investment Analysis branch working on cost and schedule analysis for the Human Landing System of the Artemis mission. Most recently, she worked as a contractor supporting the development of NASA's 2022 Strategic Plan and Evidence Act. She is extremely passionate about the work that she has done and continues to do at NASA and aims to drive change and innovation within NASA's Project Planning & Control and Cost Estimating communities.
When conducting probabilistic cost analysis, correlation assumptions are key assumptions and often a driver for the total output or point estimate of a cost model. Although the National Aeronautics and Space Administration (NASA) has an entire community dedicated to the development of statistical cost estimating tools and techniques to manage program and project performance, the application of accurate and data-driven correlation coefficients within these models is often overlooked. Due to the uncertain nature of correlation between random variables, NASA has had difficulty quantifying the relationships between spacecraft subsystems with specific, data-driven correlation matrices. Previously, the NASA cost analysis community has addressed this challenge by either selecting a blanket correlation value to address uncertainty within the model or opting out of using any correlation value altogether. One hypothesized method of improving NASA cost estimates involves deriving subsystem correlation coefficients from the residuals of the regression equations for the cost estimating relationships (CERs) of various spacecraft subsystems and support functions. This study investigates the feasibility of this methodology using the CERs from NASA's Project Cost Estimating Capability (PCEC) model. The correlation coefficients for each subsystem of the NASA Work Breakdown Structure were determined by correlating the residuals of PCEC's subsystem CERs. These correlation coefficients were then compiled into a 20x20 correlation matrix and were implemented into PCEC as an uncertainty factor influencing the model's pre-existing cost distributions. Once this correlation matrix was implemented into the cost distributions of PCEC, the Latin Hypercube Sampling function of the Microsoft Excel add-in Argo was used to simulate PCEC results for 40 missions within the PCEC database. These steps were repeated three additional times using the following correlation matrices: (1) a correlation matrix assuming the correlation between each subsystem is zero, (2) a correlation matrix assuming the correlation between each subsystem is 1, and (3) a correlation matrix using a blanket value of 0.3. The results of these simulations showed that the correlation matrix derived from the residuals of the subsystem CERs significantly reduced bias and error within PCEC's estimating capability. The results also indicated that the probability density function and cumulative distribution function of each mission in the PCEC database were altered significantly by the correlation matrices that were implemented into the model. This research produced (1) a standard subsystem correlation matrix that has been proven to improve estimating accuracy within PCEC and (2) a replicable methodology for creating this correlation matrix that can be used in future cost estimating models. This information can help the NASA cost analysis community understand the effects of applying uncertainty within cost models and perform sensitivity analyses on project cost estimates. This is significant because NASA has been frequently critiqued for underestimating project costs and this methodology has shown promise in improving NASA's future cost estimates and painting a more realistic picture of the total possible range of spacecraft development costs.
Session Materials: dataworks.testscience.org/wp-...
16 май 2024
