Conclusions

Cost effectiveness within the simulated scenario is strictly related to the quality of prediction when measured with recall (\(Rec\)). Based on the simulation presented, it is possible to calculate the \(NetReturn\) of using a proposed quality assurance effort allocation strategy for a series of defect prediction recall values (solid line on Figure \ref{fig:qa_vs_recall}). In such a way, we can observe how \(NetReturn\) depends on \(Rec\) in terms of the proposed QA effort allocation strategy. However, any \(NetReturn\) coefficient values for \(Rec\) lower than 0,5 should not be considered, as below that point, the efficiency of defect prediction is similar to that of random guessing.
One can argue, that it is improper for one to assume, that it is possible for programmers to predict and fix 78% of all detectable defects while still in the developmental phase, only due to the fact that code parts actually responsible for these defects are known the developers, thanks to defect prediction. Please notice, that in our simulation we considered only bugs actually fixed by programmers. The only difference is that in the actual project (not in simulation), more defects were found by testers and end-users during the second and third project phases, which caused higher average bug fixing costs in these phases. Nevertheless, the defects were finally fixed by the developers. Moreover, we could go a step further and modify our effort allocation strategy to use DePress for prediction model creation, aimed at finding code responsible for 100% of all detectable defects. In such a case, the relation between the expected number of defects fixed in the developmental stage, and prediction recall would be:
\begin{equation} H_{1}^{\prime}{}=Rec\times H_{total}\\ \end{equation}
Theoretically, in such a case we can expect that the \(NetReturn\) value should be even higher than calculated based on the proposed strategy (dotted line on Figure \ref{fig:qa_vs_recall}). However, as the proposed effort allocation strategy is based on the Pareto principle (only 20% of code is responsible for 80% of defects), we should consequently expect that to find and remove the remaining 20% of defects, an additional 80% of code would need to be inspected, respectively. In such a scenario, by covering even 100% of code by all necessary precautions to remove every detectable defect may cost significantly more than the actual, real quality assurance in the considered project release (4.0.0). We expect that the remaining approximately 20% of detectable defects can be effectively found and removed during further phases of the project, such as in testing phase, with better cost effectiveness, than with comprehensive quality assurance works during coding phase, focused on finding and eliminating 100% of detectable defects.