loading page

Eleven Ways to Measure the Immeasurable and Count the Incalculable
  • +1
  • Dwight Owens,
  • S. Kim Juniper,
  • Kathryn Moran,
  • Benoît Pirenne
Dwight Owens
Ocean Networks Canada

Corresponding Author:[email protected]

Author Profile
S. Kim Juniper
Ocean Networks Canada
Author Profile
Kathryn Moran
Ocean Networks Canada
Author Profile
Benoît Pirenne
Ocean Networks Canada
Author Profile

Abstract

In 2017 Ocean Networks Canada (ONC), a research infrastructure operator, sought to redefine its core reporting metrics. We asked, “which metrics should we hold as key, essential metrics to drive our organizational priorities and decision making?” This question helped us define a collection of eleven sets of yardsticks, some inward-looking, others squarely focused on societal outcomes. Here, we introduce the individual metrics adopted, insights they are helping us glean and some of their inherent challenges. ONC’s core funding agency, the Canada Foundation for Innovation (CFI), continues to emphasize scientific output as a primary criterion. We measure this by counting peer-reviewed presentations and publications resulting from use the facility and ONC’s data archives. But this seemingly clear-cut metric has been a thorny one to define, track and grow. Training and support for post-secondary students is another core reporting metric, however this measurement is also fraught with ambiguities. Some of the easier metrics to track are those specifically related to facility operations, such as reliability and user satisfaction. But we were perplexed by the question of how to measure “optimal use” of the facility, as mandated by CFI. Optimal use is hard to define for an underwater infrastructure design like ONC’s, which can be flexibly extended with no hard limits on hardware capacity, archive volume or data access. When it comes to societal benefit, our approach has been twofold. One set of metrics examines technology transfer, grants and contracts. Another set focuses on our engagements and active collaborations with governmental, indigenous and non-governmental organizations. However, some outcomes remain challenging to measure. While it is straightforward to count up our external interactions and collaborations, how can we quantify their current and future societal impact? These and related questions will be explored.