Andreas Colliander

and 47 more

NASA’s Soil Moisture Active Passive (SMAP) mission has been validating its soil moisture (SM) products since the start of data production on March 31, 2015. Prior to launch, the mission defined a set of criteria for core validation sites (CVS) that enable the testing of the key mission SM accuracy requirement (unbiased root-mean-square error <0.04 m3/m3). The validation approach also includes other (“sparse network”) in situ SM measurements, satellite SM products, model-based SM products, and field experiments. Over the past six years, the SMAP SM products have been analyzed with respect to these reference data, and the analysis approaches themselves have been scrutinized in an effort to best understand the products’ performance. Validation of the most recent SMAP Level 2 and 3 SM retrieval products (R17000) shows that the L-band (1.4 GHz) radiometer-based SM record continues to meet mission requirements. The products are generally consistent with SM retrievals from the ESA Soil Moisture Ocean Salinity mission, although there are differences in some regions. The high-resolution (3-km) SM retrieval product, generated by combining Copernicus Sentinel-1 data with SMAP observations, performs within expectations. Currently, however, there is limited availability of 3-km CVS data to support extensive validation at this spatial scale. The most recent (version 5) SMAP Level 4 SM data assimilation product providing surface and root-zone SM with complete spatio-temporal coverage at 9-km resolution also meets performance requirements. The SMAP SM validation program will continue throughout the mission life; future plans include expanding it to forested and high-latitude regions.

Jerry Bieszczad

and 4 more

Newer satellite platforms, such as NISAR, are poised to produce huge amounts of data that require large computational resources. Currently, researchers typically download datasets for analysis on local computer resources. This paradigm is no longer practical given the volumes of data from new sensing platforms. While cloud computing services offer a potential solution for accessing and managing large computational resources, there remains a significant barrier to entry. Levering cloud services requires users to: navigate new terminology without appropriate documentation; optimize settings for services to reduce costs; and maintain software dependencies, upgrades, and allocated hardware resources. A more accessible approach for migrating earth scientists to the cloud is needed. To address this problem, we are developing the open source Python library PODPAC (Pipeline for Observational Data Processing Analysis and Collaboration), with the goal of helping to address NASA’s rapidly growing observational data volume and variety needs. PODPAC enables earth scientists to seamlessly transition between processing on a local workstation (their current paradigm) to distributed remote processing on the cloud. It does this by leveraging a text-based JSON format automatically generated for any plug-and-play algorithm developed using PODPAC (e.g., in a Jupyter Notebook). This text format describes data provenance, and is used in RESTful web requests to preconfigured PODPAC cloud deployments, allowing scalable, massively distributed processing. We demonstrate the seamless transition to the cloud by developing a simplified soil moisture downscaling algorithm in Python using PODPAC. Data for this algorithm uses NASA Soil Moisture Active Passive (SMAP) sensor retrieved from the National Snow and Ice Data Center using OpenDAP, and fine-scale topographic data retrieved via Open Geospatial Consortium (OGC) Web Coverage Service (WCS) calls. We then use a serverless AWS Lambda function to run the same algorithm using the automatically-generated text format. Our generic preconfigured environment can handle a wide variety of processing pipelines, and scale up to 1024 parallel processes. This approach enables incremental adoption of cloud services by researchers, significantly lowering the barrier to entry.