Institute for Software Research
School of Computer Science, Carnegie Mellon University


An Empirical Comparison of Field Defect Modeling Methods

Paul Luo Li, Mary Shaw, Jim Herbsleb, P. Santhanam*, Bonnie Ray*

May 2005


Keywords: Empirical Studies, Metrics, Reliability Engineering Management, Measurement, Reliability, Experimentation, Defect Modeling, Empirical Research, COTS, Maintenance Resource Planning, Software Insurance

In this study, we report empirical results from forecasting field defect rates and predicting the number of field defects for a large commercial software system. We find that we are not able to accurately forecast field defect rates using a combined time-based and metrics-based approach, as judged by the Theil forecasting statistic. We suggest possible conditions that may have contributed to the poor results. Next, we use metrics-based approaches to predict the number of field defects within the six months after deployment. We find that the simple ratios method produce more accurate predictions than more complex metrics-based methods. Our results are steps toward quantitatively managing the risks associated with software field defects.

17 pages

*Center for Software Engineering, IBM T.J. Watson Research Center, Hawthorne, NY 10532

Return to: SCS Technical Report Collection
School of Computer Science homepage

This page maintained by