Fig. 1. Results of our systematic review on ecological
meta-analyses and their treatment of missing variances and sample sizes
in primary studies summarized by 505 ecological meta-analyses that were
published until March 23, 2018 (cf. Supplement S1 and S2, Supporting
Information).
Fig. 2. Workflow to compare the effects of the multiple
imputation of deleted standard deviations (SDs) and sample sizes (SSs)
with a meta-analysis of a complete dataset. (1) We deleted between 10%
and 90% of the SDs, SSs or both in the control and treatment groups of
an artificial dataset. (2) The deleted values were imputed (in green)
via multiple imputations (100 times), all done with the same imputation
method. (3) Each of the 100 datasets was analysed with a separate linear
mixed-effects meta-analyses (imputed values in green). (4) The resulting
100 grand means, and confidence intervals were averaged according to
Rubin’s rules42 in order to obtain single estimates.
(5) These estimates were compared to the results of an analysis of the
complete data set (i.e. without missing values).
Fig. 3. Effects of imputing SDs and SSs that arem issing c ompletely a t r andom (MCAR)
on the grand mean (coloured line) and confidence interval (shaded area)
with respect to the results of fully informed weighted meta-analyses.
Rows shows results for the 14 methods to treat missing values (c.f.
Table 1). Columns show result for the log response ratio, Hedges’d and Fisher’s z effect sizes with 10% (top) up to 90%
(bottom) of standard deviations (SDs) and/or sample sizes (SSs) removed.
Each panel shows the deviation of the grand mean and its approximated
95% confidence interval (divided by two for better visibility) from the
results obtained with a fully informed weighted meta-analysis.
Deviations to the right indicate lower values and deviations to the
right indicate higher values.
Fig. 4. Effects of imputing SDs and SSs that arem issing a t r andom (MAR) on the grand mean
(coloured line) and confidence interval (shaded area) with respect to
the results of fully informed weighted meta-analyses. Rows shows results
for the 14 methods to treat missing values (c.f. Table 1). Columns show
result for the log response ratio, Hedges’ d and Fisher’sz effect sizes with 10% (top) up to 90% (bottom) of standard
deviations (SDs) and/or sample sizes (SSs) removed. Each panel shows the
deviation of the grand mean and its approximated 95% confidence
interval (divided by two for better visibility) from the results
obtained with a fully informed weighted meta-analysis. Deviations to the
right indicate lower values and deviations to the right indicate higher
values.
Fig. 5. Effects of imputing SDs and SSs that arem issing n ot a t r andom (MNAR) on the
grand mean (coloured line) and confidence interval (shaded area) with
respect to the results of fully informed weighted meta-analyses. Rows
shows results for the 14 methods to treat missing values (c.f. Table 1).
Columns show result for the log response ratio, Hedges’ d and
Fisher’s z effect sizes with 10% (top) up to 90% (bottom) of
standard deviations (SDs) and/or sample sizes (SSs) removed. Each panel
shows the deviation of the grand mean and its approximated 95%
confidence interval (divided by two for better visibility) from the
results obtained with a fully informed weighted meta-analysis.
Deviations to the right indicate lower values and deviations to the
right indicate higher values.
Fig. 6. Effects of imputing SDs and SSs that arecor related with effect sizes and m issingc ompletely a t r andom (corMCAR ) on
the grand mean (coloured line) and confidence interval (shaded area)
with respect to the results of fully informed weighted meta-analyses.
Rows shows results for the 14 methods to treat missing values (c.f.
Table 1). Columns show result for the log response ratio, Hedges’d and Fisher’s z effect sizes with 10% (top) up to 90%
(bottom) of standard deviations (SDs) and/or sample sizes (SSs) removed.
Each panel shows the deviation of the grand mean and its approximated
95% confidence interval (divided by two for better visibility) from the
results obtained with a fully informed weighted meta-analysis.
Deviations to the right indicate lower values and deviations to the
right indicate higher values.