CNODES Q&A:
How does external validity come into play when planning and
conducting projects in the database? Just as with Sentinel, there is no
explicit consideration of external validity during research. Still, the
query refinement process helps establish whether limiting to specific
groups of people (e.g. true new users of drugs, patients at high risk of
the outcome) would reduce the relevance of the analysis, and whether
some specific provinces and data partners like CPRD or MarketScan differ
enough from the others in covariates and follow-up distribution to
ultimately limit their use of the research question. External validity
also comes up indirectly when combining the results, identifying
outliers, and conducting meta-analyses.
What target populations, if any, underlies most analyses?Typically, the main target population of interest is the overall
Canadian population. As a result of the fact that 97% of Canadian
citizens reside in the provinces contributing to CNODES, the population
represented in the Canadian portion of CNODES analyses and the Canadian
target population are very similar; restrictions on drug coverage can
change this for some analyses, however.
Are there ways to generalize the findings of the nodes to the
network? The main way that findings are generalized from site-specific
estimates to the broader network is typically by random-effects
meta-analysis using inverse variance weighting.30 When
using exclusively the Canadian portions of the network, this means that
provinces with more events (and likely more individuals) tend to
contribute more to the overall effect estimates. No attempt is made to
generalize the results of each site, however, and effects are generally
assumed to be constant across sites unless there is substantial
heterogeneity.
How easily can node-specific estimates be transported between
nodes or to external populations? Differences in demographics, the
services and medications covered by each province, and the calendar time
intervals each data source contributes can make it difficult to directly
transport effect estimates between provinces. Because all these
variables are measured, however, analytic methods like inverse odds
weights or G-computation may be used to obtain more precise
within-province estimates.26 Similar approaches could
be used for researchers interested in using CNODES to estimate treatment
effects in European or US populations, provided a target population was
provided in the scientific and analytic protocols.
Are choices ever made to maximize target validity, rather than
internal validity or precision? Internal validity is the core focus
when preparing the scientific and analytic protocols. When choosing the
provinces and data that will contribute to a given study, however,
attention is paid to populations that are may differ greatly from the
Canadian target population.