loading page

The fractions skill score for ensemble forecast verification
  • +3
  • Tobias Necker,
  • Ludwig Wolfgruber,
  • Lukas Kugler,
  • Martin Weissmann,
  • Manfred Dorninger,
  • Stefano Serafin
Tobias Necker
Institut für Meteorologie und Geophysik, Universität Wien

Corresponding Author:[email protected]

Author Profile
Ludwig Wolfgruber
Institut für Meteorologie und Geophysik, Universität Wien
Lukas Kugler
Institut für Meteorologie und Geophysik, Universität Wien
Martin Weissmann
Institut für Meteorologie und Geophysik, Universität Wien
Manfred Dorninger
Institut für Meteorologie und Geophysik, Universität Wien
Stefano Serafin
Institut für Meteorologie und Geophysik, Universität Wien

Abstract

The Fractions Skill Score (FSS) is a neighbourhood verification method originally designed to verify deterministic forecasts of binary events. Previous studies employed different approaches for computing an ensemble-based FSS for probabilistic forecast verification. We show that the formulation of an ensemble-based FSS substantially affects verification results. Comparing four possible approaches, we determine how different ensemble-based FSS variants depend on ensemble size, neighbourhood size, and forecast event frequency of occurrence. We demonstrate that only one ensemble-based FSS, which we call the probabilistic FSS (pFSS), is well-behaved and reasonably dependent on ensemble size. Furthermore, we derive a relationship that allows one to predict how the pFSS behaves with ensemble size. The proposed relationship is very similar to a known result for the Brier Skill Score. Our study uses high-resolution 1000-member ensemble precipitation forecasts from a high-impact weather period. The large ensemble enables us to study the influence of ensemble and neighbourhood size on forecast skill by deriving probabilistic skilful spatial scales.