Statistical Analysis
The comparison measure used was statistical agreement, defined as the
number of correctly classified images divided by the total number of
images submitted to the test. Cohen’s Kappa coefficient was used to
compare artificial intelligence algorithms, and Fleiss’ Kappa
coefficient was used to compare different evaluators with the CaRDIA-X
algorithm (9).
The sample size was calculated based on the results of Jay Chudow (10),
using the formula proposed by Lachenbruch in 1992 (11), implemented in
the biostatUZH R package by the sampleSizeMcNemar function. For an
estimated overall accuracy of 80%, an alpha of 0.05, the minimum number
of images was 395.
The pre-specified primary outcome was an analysis of the agreement in
the identification of the cardiac device manufacturer of the PIDa®,
PIDw® and PMMnn® applications and the CaRDIA-X® algorithm performed by 4
evaluators with different levels of medical training (operator 1:
medical student, operator 2: internal medicine resident, operator 3:
cardiology resident, operator 4: electrophysiology resident). The
standard of comparison was the interrogation of the device and recorded
in the clinical history. A p -value calculation was performed to
adjust for the effect of chance on the observed proportion of agreement.
For all calculations, R Core Team (2020) was used (12).