Since June 2017, the Stratospheric Aerosol and Gas Experiment III instrument on the International Space Station (SAGE III/ISS) has been providing vertical profiles of upper tropospheric to stratospheric water vapor (WV) retrieved from solar occultation transmission measurements. The goal of this paper is to evaluate the publicly released SAGE III/ISS beta version 5.1 WV retrieval through intercomparison with independent satellite- and balloon-based measurements, and to present recommendations for SAGE III/ISS data quality screening criteria. Overall, we find that SAGE III/ISS provides high quality water vapor measurements. Low quality profiles are predominately due to retrieval instabilities in the upper stratosphere that cause step-like changes in the profile, and aerosol/cloud-related interferences (below ~20 km). Above 35 km, the retrieved uncertainty and noise in the data rapidly grow with increasing altitude due to relatively low extinction signal from water vapor. Below the tropopause, retrieved uncertainty increases with decreasing altitude due to enhanced molecular scattering and aerosol extinction. After screening low-quality data using the procedures described herein, SAGE III/ISS WV is shown to be in good agreement with independent satellite and balloon-based measurements. From 20 – 40 km, SAGE III/ISS WV v5.1 data exhibit a bias of 0.0 to -0.5 ppmv (~10 %) relative to the independent data, depending on the instrument and altitude. Despite its status as a beta version, the level of SAGE III/ISS WV agreement with independent data is similar to previous SAGE instruments, and therefore the data are suitable for scientific studies of stratospheric water vapor.