Summary
Multiple imputation of missing variance measures can be expected to become a standard feature to increase the quality and trustworthiness of future meta-analyses, as advocated by Gurevitch et al. and Nakagawa et al. Our results clearly show that complete-case and unweighted analyses, although frequently applied, can potentially lead to deviation in the grand means and thus biased conclusions and should therefore be replaced with or (at least) compared to the results of multiple imputation analyses. The same imputation methods might also be applied re-evaluate the robustness of already published meta-analyses.
With our simulation study, we aim to raise more awareness on the problem of incompletely reported study results and their frequent omission in ecological meta-analyses. Our results discourage the use of complete-case, unweighted and sample-size weighted meta-analyses since all three options could result in deviation of the grand means and confidence intervals. Even in the absence of valid predictors for the imputation of missing SDs or SSs, their imputation has the advantage of including all incompletely reported effect sizes while at the same time preserving the weights of the reported ones.
In summary, our study provides compelling evidence that future meta-analyses would benefit from a routine application of imputation algorithms to fill unreported SDs and SSs in order to increase both, the amount of synthesized effect sizes and the validity of the derived grand mean estimates. The provided R-script number three could thereby be used to quickly assess to what degree the results of one’s own meta-analysis might be affected by the different options to treat missing SDs and SSs.