Confidence intervals for forecast verification measures in meteorology using a three-dimensional block-bootstrap
In forecast verification, it is common to report verification measures without any estimate of uncertainty. Bootstrapping is a powerful data-driven method to construct probability distributions for arbitrary parameters, however, care has to be taken to address the issue of temporal and spatial correlation, as well as non-stationarity, of meteorological data. The block bootstrap method addresses the former issue, but is still unsuitable for non-stationary time series in its usual form. Here we present the use of two multi-dimensional block bootstrap strategies for inferring the confidence interval of measures of skill for categorical thunderstorm forecasts, which have both a diurnal and annual cycle. The first method respects the annual cycle by simply bootstrapping entire years in the temporal dimension, while using a suitable spatial block width. The second is based on reconstructing the annual cycle by using spatially bootstrapped blocks which are sampled and placed sequentially in time in blocks which span integer multiples of the diurnal period. As expected, the use of larger blocks in the first method leads to a lower estimated variance. A computationally fast implementation of these algorithms is provided in Julia.