loading page

Common Data and Metadata Models for Geophysical Data in the Cloud
  • +10
  • Ted Habermann,
  • Chad Trabant,
  • Tim Ronan,
  • Manoch Bahavar,
  • Christopher Crosby,
  • Timothy Dittman,
  • Jerry Carter,
  • David Mencin,
  • Yazan Suleiman,
  • Lloyd Carothers,
  • Bruce Beaudoin,
  • Matt Briggs,
  • Garrett Bates
Ted Habermann
Metadata Game Changers

Corresponding Author:[email protected]

Author Profile
Chad Trabant
IRIS Data Management Center
Author Profile
Tim Ronan
IRIS Data Management Center
Author Profile
Manoch Bahavar
IRIS Data Management Center
Author Profile
Christopher Crosby
UNAVCO Inc
Author Profile
Timothy Dittman
UNAVCO
Author Profile
Jerry Carter
Incorporated Research Institutions For Seismology
Author Profile
David Mencin
UNAVCO Inc
Author Profile
Yazan Suleiman
IRIS Data Management Center
Author Profile
Lloyd Carothers
IRIS/PASSCAL
Author Profile
Bruce Beaudoin
IRIS/PASSCAL
Author Profile
Matt Briggs
IRIS/PASSCAL
Author Profile
Garrett Bates
IRIS/PASSCAL
Author Profile

Abstract

UNAVCO and IRIS, major repositories for global geodetic and seismic data, are in the process of joining their operations to form a unified facility for supporting the broad spectrum of geophysical observations and science required to help understand and predict the behavior of Earth Systems. This process would be complicated in a static data management environment, but both repositories are also migrating archives and services to the cloud as part of the merger. To simplify and unify archive data management, the organizations are collaborating to create common data and metadata models for observations from a wide variety of instruments and disciplines. For data, the initial focus has been on the xArray data model, already used in the geodetic and magnetotelluric communities, which can be implemented with several disc- and cloud-native approaches (HDF5, netCDF4, and Zarr). For metadata, the SensorML Standard developed by the Open Geospatial Consortium is being explored because 1) SensorML accommodates the large parameter space associated with instrument metadata required to use and trust complex observations and 2) the ability to extend the standard when required. The merger of two large repositories combined with migration to the cloud requires careful identification and on-going testing of a wide variety of assumptions about data management systems. This presentation will focus on lessons learned so far.