INTRODUCTION
A central aspect of scientific inference is distinguishing coincidence from cause and effect. David Hume – one of the greatest philosophers of all times – maintained that this is empirically impossible.1 Methodological development during last 50-70 years have identified randomization as a necessary condition to establish the relationship between cause and effect2-4, such as ascertaining treatment effects tested in randomized controlled trials (RCTs).5 Since the early 1960’s 3,6, regulators such as Food and Drug Administration (FDA) and later on, the European Medicines Agency (EMA) have typically required RCTs to approve new treatments for use in medical practice. However, developments in basic science and signals can be identified without testing in RCTs. 7An unintended consequence of these developments is that a formal mechanism for rigorous cause-effect inferences is removed and, by and large, cannot be reliably compensated by using techniques based on non-randomized comparisons.8 Nevertheless, under some circumstances, the therapeutic signal observed in non-RCTs testing is accepted as “truthful”.9 This, for example, occurs when effects of treatments are so large (“dramatic”) that they are believed to override the combined effects of biases and random errors that potentially affect the study’s results.9 The mechanism for accepting such results as “true” resides in the way our minds distinguish these signals as “true” from “false”. A theoretical framework for such inferences can be postulated within the heuristic theory of decision-making10,11 linked with signal detection theory (SDT).12,13 Indeed, when formal methods such as RCTs are not available people resort to heuristics.10 Here we argue that inference about dramatic effects is driven by two heuristics: first, the large effects must be recognized (recognition heuristic) and second, the magnitude of that effect to cross the decision threshold reflects the heuristic according to the Weber-Fechner law.
In this paper, we propose how SDT-related heuristics can be used to interpret the results of non-RCT comparisons of drug approvals by the FDA and EMA. We believe that wider familiarity with these principles has important educational values for trainees, practicing physicians, researchers, and policy-makers.