The use of large ensembles of model simulations is growing due to the need to minimize the influence of internal variability in evaluation of climate models and the detection of climate change induced trends. Yet, exactly how many ensemble members are required to effectively separate internal variability from climate change varies from model to model and metric to metric. Here we analyze the first three statistical moments (i.e., mean, variance and skewness) of detrended precipitation and sea surface temperature (interannual anomalies for variance and skewness) in the eastern equatorial Pacific from observations and ensembles of Coupled Model Intercomparison Project Phase 6 (CMIP6) climate simulations. We then develop/assess the equations, based around established statistical theory, for estimating the required ensemble size for a user defined uncertainty range. Our results show that — as predicted by statistical theory — the uncertainties in ensemble means of these statistics decreases with the square root of the time series length and/or ensemble size. Further to this, as the uncertainties of these ensemble-mean statistics are generally similar when computed using pre-Industrial control runs versus historical runs, the pre-industrial runs can sometimes be used to estimate: i) the number of realizations and years needed for a historical ensemble to adequately characterize a given statistic; or ii) the expected uncertainty of statistics computed from an existing historical simulation or ensemble, if a large ensemble is not available.

Large ensembles of model simulations require considerable resources, and thus defining an appropriate ensemble size for a particular application is an important experimental design criterion. Utilizing the recently developed CLIVAR ENSO Metrics Package (Planton et al., 2021), we estimate the ensemble size (N) needed to assess a model’s ability to capture observed ENSO behavior. Using the larger ensembles available from CMIP6 and the CLIVAR Large Ensemble Project, we find that larger ensembles are needed to robustly capture baseline ENSO characteristics (N > 65) and physical processes (N > 50) than the background climatology (N ≥ 12) and remote ENSO teleconnections (N ≥ 6). While these results vary somewhat across metrics and models, our study highlights that ensembles are required to robustly evaluate simulated historical ENSO behavior, and provide initial guidance for designing model ensembles to reliably evaluate and compare ENSO simulations.