loading page

Real Effect or Bias? Best Practices for Evaluating the Robustness of Real-World Evidence through Quantitative Sensitivity Analysis for Unmeasured Confounding
  • +7
  • Douglas Faries,
  • Chenyin Gao,
  • Xiang Zhang,
  • Chad Hazlett,
  • James Stamey,
  • Shu Yang,
  • Peng Ding,
  • Mingyang Shan,
  • Kristin Sheffield,
  • Nancy Dreyer
Douglas Faries
Eli Lilly and Company

Corresponding Author:[email protected]

Author Profile
Chenyin Gao
North Carolina Central University College of Health and Sciences
Author Profile
Xiang Zhang
CSL Behring Canada Inc
Author Profile
Chad Hazlett
University of California Los Angeles
Author Profile
James Stamey
Baylor University
Author Profile
Shu Yang
North Carolina Central University College of Health and Sciences
Author Profile
Peng Ding
University of California Berkeley Department of Statistics
Author Profile
Mingyang Shan
Eli Lilly and Company
Author Profile
Kristin Sheffield
Eli Lilly and Company
Author Profile
Nancy Dreyer
IQVIA Deerfield
Author Profile

Abstract

The assumption of ‘no unmeasured confounders’ is a critical but unverifiable assumption required for causal inference yet quantitative sensitivity analyses to assess robustness of real-world evidence remains underutilized. The lack of use is likely in part due to complexity of implementation and often specific and restrictive data requirements required for application of each method. With the advent of sensitivity analyses methods that are broadly applicable in that they do not require identification of a specific unmeasured confounder – along with publicly available code for implementation – roadblocks toward broader use are decreasing. To spur greater application, here we present a best practice guidance to address the potential for unmeasured confounding at both the design and analysis stages, including a set of framing questions and an analytic toolbox for researchers. The questions at the design stage guide the research through steps evaluating the potential robustness of the design while encouraging gathering of additional data to reduce uncertainty due to potential confounding. At the analysis stage, the questions guide researchers to quantifying the robustness of the observed result and providing researchers with a clearer indication of the robustness of their conclusions. We demonstrate the application of the guidance using simulated data based on a real-world fibromyalgia study, applying multiple methods from our analytic toolbox for illustration purposes.