2. Comprehensive evaluation index
This paper uses a random 10×2K-Folding cross-validation method to
classify the samples based on the ARF-OOBEE ensemble model. Among them,
after testing, the number of ensemble learning base classifiers is 70,
the depth is 12, and it is compared with the prediction results of 5
common machine learning algorithms. According to the prediction index
analysis in Table 3, compared with the other five algorithms, the
ARF-OOBEE algorithm has improved the accuracy of G-Mean and AUC
parameters by nearly 2%. It can be seen that for the AR samples with
clinical imbalance characteristics, the ARF-OOBEE model has good
generalization performance and comprehensive classification ability.
Precision, sensitivity, specificity, G-Mean= sqrt
(Sensitivity×Specificity), F1-Score, area under ROC curve AUC and other
parameters together were used as predictive evaluation
indicators[22]. In Table 3 and Figure 4, 7
classification models are selected for comparison, covering probability
model, tree model, linear model, ensemble model and neural network
model.It comprehensively reflects the performance of the research
objects in different classification models and the ensemble model has
the best and most stable effect, in this paper. The comprehensive
classification evaluation index is lower than the ensemble
classification algorithms ARF-OOBEE and GCForest. The GCForest algorithm
is composed of two RF and two extreme random tree(ERT) in parallel
structure, and its multiple comprehensive evaluation indicators are
better than the single structure RF algorithm, but the classification
calculation is relatively large. The structure of the ARF-OOBEE model
has adaptive characteristics, which can dynamically change the number of
ensemble learning base classifiers, and train the component classifier
model parameters separately. It has good comprehensive classification
characteristics for massive large data and unbalanced samples.
Table 4 gives the independent classification evaluation indicators of
the 8 types of rhinitis symptoms data for the original sample. Data
analysis shows that the prediction accuracy of AR, RS, CS, SD, URI, AH,
NAR and OTH for the binary classification of rhinitis is higher, while
the classification of degree and types in multi-class rhinitis is lower.
The reason is that the classification of the four binary classification
rhinitis is based on data rebalancing and is determined by the
dynamically ensemble RF weighted voting algorithm in the ARF model.
Output prediction of AR classification were estimated using an
ERTensemble algorithm with multi-category classification. ARF-OOBEE
ensemble model converts the compound label classification problem into a
four-label classification problem as and two multi-class classification
problems.Multi-label classification were used in classification of AR,
RS, URI, OTH, and multi-category classification were used in
classification of AR’sdegree and type respectively,and it can avoid two
or more AR classification labels in the same patient at the same time
The evaluation method in this paper uses a calculation method based on
sample weights. Sensitivity represents the model’s ability to identify
patients with real illnesses, while specificity represents the model’s
misdiagnosis rate, and the Hamming loss is a common way of evaluating
multiple classifications. The data in the table uses weighted scores.
Compared with evaluating the performance of the model itself, it more
reflects its performance in actual use. Avoid the rare cases of
diagnosis in reality that reduce the overall evaluation of the model.
For the few cases of missed diagnosis in the auxiliary diagnosis model
designed in this paper, it can be ruled out by the doctor’s secondary
review and other methods.
Discussion
In recent years, the prevalence of AR has increased significantly, and
its diagnosis is more based on symptom evaluation and allergen
detection, but due to the lack of effective and reliable diagnostic
tests, the diagnosis requires experts to verify the final results based
on experience[23,24]. In order to help junior physicians and
clinicians diagnose allergic diseases, this work uses AI methods to
extract new information from previous data for training[25,26].
Through the dynamic verification of the rule base and rule inference
method, make the clinical diagnosis support system more adaptable. By
introducing meta-heuristic data preprocessing technology and ensemble
classification method, the systemefficiency can be further improved.
Therefore, junior clinicians can strengthen clinical decision-making by
more accurately diagnosing allergic diseases, can diagnose and treat AR
earlier, can control the appearance of patients’ symptoms to the
greatest extent, and thus improve the quality of life of patients with
AR.
The diagnosis of AR is mainly based on the symptoms and the detection of
allergens[27]. However, due to the complex and variable nature of
nasal inflammation, it is often combined with other diseases, such as
rhinosinusitis and nasal tumors. Imaging examination helps to diagnose
other diseases. Turbinate hypertrophy is also a characteristic change of
AR. Our selected cases have also been found to have rhinosinusitis and
nasal polyps. Therefore, the use of CT imaging can better assist the
diagnosis of AR.
AI technology, without human intervention, can learn tasks from a series
of training examples. Moreover, they aim to produce output that is
simple enough to be easily understood by humans. The difference is that
the characteristics of classical statistical methods are usually a clear
probability model, and it is assumed that in most cases, they require
expert intervention in variable selection and transformation of the
problem and overall structure. The general method of data analysis
usually includes four stages, namely (a) collecting and coding clinical
data in an electronic form suitable for further processing; (b) Useing
feature extraction and dimensionality reduction techniques (principal
component analysis) for data processing to select the most predictive
parameters; (c) Schema-model selection AI model; (d) Extract knowledge
by evaluating accuracy, sensitivity and specificity[28]. At present,
the most common calculation models include: artificial neural network
(ANN), SVM, Bayesian network (BN) and fuzzy logic (FL),etc.
In recent years, ensemble learning can organically combine multiple
prediction results obtained by multiple single learning models to obtain
more accurate, stable and strong final results. For exampleensemble
learning models such as Boosting, Bagging and RF have been proposed one
after another and applied to various types of data sets[29,30]. In
this study, through the deep learning of the ensemble learning model,
six common machine learning classification algorithms have been selected
for comparative experiments, including RF, multi-label naive Bayes (NB),
and multi-label SVM (SVM), multi-label logistic regression (LR),
GCForest. The single-classifier RF algorithm is a base classification
evaluation standard, and also constitutes the base classifier component
of other algorithms, with good classification specificity, but the
comprehensive classification evaluation index is lower than the ensemble
classification algorithms ARF-OOBEE, GCForest. The GCForest algorithm is
composed of two RF and two ERT in parallel structure, and its multiple
comprehensive evaluation indicators are better than the single structure
RF algorithm, but the classification calculation is relatively
large[31].
There are two types of output for AR diseases, degree and types, which
belongs to the multi-class classification problem. This article uses the
OOB (out-of-bag) EE ensemble classification algorithm and uses all
samples as training data. And the Extra-Tree (ET) model is used as the
base classifier to balance all training data to realize the prediction
of unbalanced small samples. OOBEE extracts the data equal to the
minority class from the majority class, and combines the reused minority
class data to build a multi-group base classifier, and obtains the
ensemble classifier through the weighted voting method to reduce the
impact of sample data imbalance on classification. The structure of the
ARF-OOBEE model has adaptive characteristics. It can dynamically change
the number of ensemble RF and ERTbaseclassifiers, and train the
component classifier model parameters separately. It has good
comprehensive classification characteristics for massive large data and
unbalanced samples. The results show that compared with the other five
algorithms, the ARF-OOBEE algorithm has improved the accuracy of G-Mean
and AUC parameters by nearly 2%. It can be seen that for the AR samples
with clinical imbalance characteristics, the ARF-OOBEE model has good
generalization performance and comprehensive classification ability.
There are some deficiencies in this study. First of all, the diagnosis
of AR is mainly based on the symptom score and allergen detection, but
some patients still have obvious symptoms while the test is negative,
and need to be identified by such as nasal provocation test. However,
this test cannot be widely used in the outpatient diagnosis and
treatment, therefore, there will be individual cases of diagnostic
errors. The artificial intelligence system is designed to help
diagnosis, but it cannot completely replace the rhinologist. This study
is a dual-center study conducted at Tongji Hospital of Tongji University
and Anting Branch Hospital. There may be a selection bias. In the
future, a multi-center study should be conducted to improve the database
required for training artificial intelligence systems and improve their
diagnostic capabilities. Finally, through the self-learning of the
system, it can help junior doctors complete the diagnosis of AR and
improve their diagnosis ability.
Ethical disclosures Confidentiality of data
The authors declare that no patient data appears in this article. Right
to privacy and informed consent. The authors declare that no patient
data appears in this article. Protection of human subjects and animals
in research. The authors declare that no experiments were performed on
humans or animals for this investigation.
Funding
This work was supported by the National Science Foundationof China
(grant no. 8187040043,81973749), Western Medicine GuideProject of
Shanghai City (grant no. 17411970500), ShenkangMedical Development
Center Clinical Science and Technology Innovation Project of Shanghai
City (grant no. SHDC12019X07), Health Commission Advanced Technology
Promotion Project of Shanghai City (grant no. 2019SY071).
Conflicts of interest
The authors report no conflicts of interest. The authors alone are
responsible for the content and writing of this manuscript.