Key Points
INTRODUCTION
Atelectasis and attic retraction pocket occur because of tympanic membranes architectural deformity and bad ventilation, then tympanic membranes collapse toward to the tympanic cavity. Tympanic membrane retraction is the most common change of tympanic membrane in pediatric otitis media with effusion (OME) patients 1. Atelectasis and attic retraction pocket also could be sequela of OME, and it’s more frequently in surgery cases 2,3. Patients with mild atelectasis or attic retraction pocket may have no symptoms, however, tiny attic retraction pocket may conceal attic cholesteatoma 4. Severe atelectasis and attic retraction pocket may cause erosion of ossicular chain, outer wall of tympanic cavity and hearing loss. Moreover, whether there is atelectasis or attic retraction pocket is important part in the OME diagnosis procedure 5. Cholesteatoma and adhesive otitis media are common severe sequela of atelectasis and attic retraction pocket 6. Although surgery serves as an effective method to treat severe atelectasis and attic retraction pocket, but surgery is associated with economic burden, and surgery risk, such as sensorineural hearing loss, facial palsy. And some patients may cease to retract and returned to normal condition, so prophylactic surgery would not be recommended 7. However, early diagnosis with appropriate follow up is a reasonable policy to manage atelectasis and attic retraction pocket, irrespective of surgery or not 8,9.
Otoscope is widely accepted for diagnosing and managing OME, atelectasis and attic retraction pocket 5,10. And many types of smartphone adaptable otoscopes can be used to acquire tympanic membranes images by no-specialty or no-clinicians 11-13. However, Diagnosis of ear disease only with manual examination shows low accuracy, which may lead to improper referral, delayed or improper treatment and pointless follow-up.
The progressive use of telemedicine and artificial medicine in the otologic setting may gradually change the procedure of disease management. Wu et. al 14 developed a deep learning model to diagnosis pediatric otitis media using otoscopic images and tested in a smartphone-enabled otoscope set. Shie et. al15 extracted color, geometric and texture features to develop a classification system for differentiating most type of otitis media, achieving an accuracy of 88.06% in 865 otoscopic images. Cha et. al 16 developed a deep learning model to detect 6 common ear diseases acquiring an accuracy of 93.67%. All the previous artificial intelligence studies didn’t classify retraction of tympanic membranes as atelectasis and attic retraction pocket.
The purpose of this study was to develop a deep learning model to diagnose atelectasis and attic retraction pocket using otoscopic images and assess the performance of deep learning model. It may be used to improve the procedure of OME diagnosis and management, such as saving time and improving diagnostic accuracy.
METHOD
Participant selection and otoscopic images acquisition
Otoscopic images from inpatients and outpatients were collected retrospectively from 3 hospitals between year 2015 to 2019. Otoscopic images were taken with 4 mm (KARL STORZ, Germany) or 2.7 mm (TIAN SONG, China) 0-degree otoscope by otolaryngologists. OME cases were confirmed with criteria of clinical guideline5, including disease history, medical examination and auditory test. 1 to 3 best quality otoscopic images from different angles with complete pars tensa and pars flaccida were adopted from each ear with at least 500 × 500 pixels. White light, eardrum size more than 50% in the otoscopic images and light reflection without overexposure and underexposure were optimal. Otoscopic images with tympanostomy tube, secretion and earwax more than 25% of tympanic membranes were excluded in this study.
Clinical labelling of otoscopic images
Only a few parts of otoscopic images have been recorded the presence of attic retraction pocket and atelectasis in the electronic medical record systems. To achieve a consistent ground truth label, we didn’t adopt these records as ground truth label. Firstly, JBZ with more than 3 years clinical experience in otology were assigned to address the presence of attic retraction pocket and atelectasis according to the first widespread standard independently 17,18. Because attic retraction pocket and atelectasis may present in the same otoscopic images, these two lesions were labeled separately. All otoscopic images were labeled the presence of attic retraction pocket and atelectasis without region annotation. Then, two otologists with more than 10 years clinical experience in otology were assigned to review labels independently, and any discrepancy will be discussed with another otologist with more than 20 years clinical experience in otology until consensus was reached. As in actual clinical practice, the prevalence of different stage of attic retraction pocket and atelectasis was heavily skewed in our dataset, stage III and IV attic retraction pocket and atelectasis with less than 5%. To ensure that there was sufficient data to develop and assess the performance of this model, we only address the presence of attic retraction pocket and atelectasis without stage classification. Other clinical demographic data wasn’t used to develop deep learning model, such as acoustic test results, age and gender.
Deep learning model development
3-fold random cross validation was adopted to divided dataset into training set and validation set. The output of this model was a standard two-class task for determining whether the input otoscopic image contained attic retraction pocket or atelectasis. We used a CNN model pretrained on the ImageNet dataset (http://www.image-net.org), then otoscopic images of this dataset were used to fine-tune the hyperparameters of the pretrained CNN model. During the process of training, online data was used for data expansion, including random vertical and horizontal flip, and constant aspect ratio scaling. Considering our previous experience, Google Inception-V3 were suitable for developing deep learning model based on otoscopic images. So, Google Inception-V3 CNN model was adopted as the backbone network and trained, tuned and evaluated19. All the otoscopic images were turned into 299 × 299 pixels as input data. CNN model consisted of a convolutional neural network to implicitly recognize characteristics of attic retraction pocket and atelectasis from otoscopic images.
To evaluate the CNN model performance in clinical practice, we compared the predicted diagnosis with the labeled standard diagnosis using the 3-fold average classification accuracy, sensitivity, specificity of the model (normal pars flaccida vs attic retraction pocket, normal pars tensor vs atelectasis). We also used receiver operating characteristic (ROC) curve and corresponding area under ROC curve to show the diagnostic ability of the deep learning model in identifying the presence of attic retraction pocket and atelectasis.
Class Activation Mapping
Class Activation Mapping (CAM) was employed to visualize the discriminative region in the otoscopic images. CAM used different colors to show different values of deep learning model ranging from blue (no specific region) to red (most discriminative region). Right identification of lesion region with red color in the otoscopic images are essential for clinician to trust the deep learning model. All experiments were operated with Python 3.6 in Keras using Python programming language. The diagnostic model was developed based on TensorFlow and carried out with 4 Titan XP 256 GB GPU.
RESULTS
We collected an image dataset consist of 6393 OME otoscopic images, of which 3564 (55.74%) otoscopic images were assessed for attic retraction pocket, and atelectasis was diagnosed in 2460 (38.48%) otoscopic images. Each otoscopic images were reviewed by at least 3 expert otologists. We used 3-fold cross-validation for developing and testing the deep convolutional neural network (DCNN) model to detect OME referable attic retraction pocket and atelectasis.