2.6 GI complication risk score
The derivation cohort was randomly divided into internal training (70%) and internal hold-out validation (30%) datasets [14]. Candidate risk predictors with p>0.10 in the univariate logistic regression using the internal training dataset were included in the multivariate analysis. We performed three repeated five-fold cross-validations for predictor variable selection [14]. This was based on the principle of randomly dividing the dataset into five equal subsamples and using four subsamples for training and the remaining subsample for the test [13]. As each of the five subsamples was used once during the cross-validation process, the analysis was performed five times per cross-validation. Fifteen analyses were performed when this was repeated three times. This process has also been used for internal cross-validation [15].
After selecting the final predictors through model optimization, the regression coefficient for the risk factors in the final model was used to calculate the integer assigned to the risk prediction score [13]. The integer points closest to each regression coefficient ×10 was chosen for each risk factor [6]. Individual risk was based on the sum of the weighted scores for each assigned risk factor score [13]. For use in clinical decision-making, we derived a cut-off value for the risk prediction score to distinguish between high-risk and non-high-risk groups based on the Youden index [16].
We used an internal hold-out validation and an external validation dataset to validate the final model. The discrimination of the prediction score was evaluated by calculating the Area under the receiver operating characteristic (ROC) curves (AUROC) [13]. Calibration of the model for comparing the predicted and observed risks was evaluated using the Hosmer–Lemeshow goodness-of-fit test and calibration plots [13].
After applying the prediction model to the external validation dataset, we classified patients with a risk score greater than or equal to the cut-off value as high-risk cases. We then evaluated predictors of high-risk cases.