Reducing Systemic Errors in Cost Models
Covert, R
MCR, LLC

Cost models often rely on cost estimating relationships (CERs) derived through regression techniques that are traditionally limited to curve fitting of vectors of discrete dependent variables (costs) with vectors of discrete independent variables (cost drivers). The result of these regressions is a set of CERs and their fit statistics such as the standard error or standard percent error based on regression of discrete variables. When using these CERs in a cost uncertainty analysis, different systemic errors may be inadvertently produced by the cost model. Some of these systemic errors are identified as "functional correlations", which is just another term for systemic error. Systemic errors depend almost entirely on the cost drivers selected for the CERs and can be reduced or partially prevented. The first type of systemic error is due to the use of cost drivers that are themselves estimates, such as the estimate of an item's weight at an early design phase. The second error is due to the use of shared cost drivers, such as a common weight of a particular item. The third type of systemic error is due to the use of computed costs from one or more CERs as a cost driver, commonly referred to as a "cost-on-cost function". A new technique called errors-in-variables (EIV) regression can be used to measure the ensemble uncertainty of a cost model thus limiting the amount of systemic error.

In previous presentations, the author introduced the technique of EIV regression - a method that treats both the independent and dependent variables in a regression as random variables. EIV regression techniques can be employed to find appropriate CERs with errors in either the dependent or independent variables or even when both are random variables. Since cost modelers assume the regression variables are discrete and non-random in nature, even when computing cost-on-cost functions, systemic errors are neglected when choosing the appropriate CERs for a cost model. A cost model developed using EIV regression techniques can provide a minimum ensemble error for the cost model.

In this presentation, the author describes where and how systemic errors occur and shows examples of models containing CERs derived using the EIV technique that reduce the systemic errors in the model.