Self-Starting Control Charts in Software Development Projects

Abstract

To improve processes, generally some metrics are defined, data are collected and these are analyzed by appropriate methods. Statistical process control (SPC) is one of the methods used to improve processes. Maybe the most common tool of SPC is control charts, which are used to monitor changes in a process or characteristic over time. Control charts were originally developed for the mass production industry where production amounts are large. In such environments, a Phase I study is carried out with sufficient amount of data to estimate in-control process parameters and to determine control limits. Nevertheless, in some environments data is not abundant as in mass production and alternative approaches are needed to estimate process parameters and design control charts. For short production runs, where there is few data available to estimate parameters, self-starting control charts were developed. Recently, the SPC has expanded beyond manufacturing and has started to be used for process improvement in software development processes. Just like in short production runs, there may not be enough data available to estimate parameters for designing control charts in software development processes. In this study, self-starting methods and change point models are considered for monitoring performance of software development processes over time. Main performance metrics used in software development processes are generally counts, which have discrete probability distributions. Performance measures can be used to monitor software development processes as well as to monitor project scope. Examples of these metrics could be the number of defects calculated for different scenarios or the number of requests from customers. In this thesis, a simulation study was conducted based on the assumption that the data follows a Poisson distribution. Different project durations, change points, and shift magnitudes in the parameters were compared using self-starting methods Q and Exponentially Weighted Moving Average (EWMA) Q control charts, and a change point method Generalized Likelihood Ratio (GLR) control chart. Comparisons were made based on the probabilities of giving an out-of-control signal during the project duration, and after a change point in the case of a shift from the mean of a software quality metric. The selected control charts were applied to monitor real software development process metrics and obtained results were compared. In addition to comparing the performance of Q, EWMA Q, and GLR control charts, robustness was evaluated by taking into account the responses to different change points. The thesis demonstrates the applicability of self-starting and change point methods designed for Poisson distributed data in software development processes and presents comparisons of these methods under different scenarios.