Applying Deep Learning to X-rays in a clinical environment: A project proposal


Abdominal and chest X-rays are amongst the most commonly ordered imaging modality in any modern hospital system. (Practitioners) While other imaging modalities such as computed tomography or ultrasound can offer increased resolution and diagnostic accuracy, X-rays being widely available and accessible after hours, are often ordered for either monitoring or diagnosis. However, being so commonly ordered, these tests are often interpreted by junior staff on the wards and hence may not have a formal report by a radiologist until significant time has elapsed. (Pinto 2010) Analysis of junior staff interpretation of suggest that classic appearances of common conditions are commonly not identified, particularly where chest X-rays are concerned. (Eisen 2006) While these studies are primarily performed in the US medical system, the results are nevertheless applicable to the Australian setting. While most diagnoses on X-rays, particularly abdominal X-rays, are straightforward, certain diagnoses may be more elusive to junior staff. In the context of X-rays, we can separate the aetiology of missed lung cancer into detection or decision errors, and perform visual search experiments, concluding that missed lung cancers are often decision rather than detection error. (Manning 2004) Diagnoses such as bowel obstruction on abdominal X-ray are also often subjective and lacking in objective decision rules - these lead to wide discrepancies in accuracy rates between senior and junior staff as senior staff tend to be consistently more confident in these decisions as compared to junior staff. (Thompson 2007)

Beyond junior staff on the wards, X-rays are often ordered in the Emergency Department as a front line, non invasive and cheap test. Given the logistical pressures on Emergency Departments within Australia, emergency physicians often do not have the luxury of waiting for a formal radiology report. For instance, at the Alfred ED - 36% of formal posteroanterior chest X-rays within the last 7 years have been reported within 4 hours, while 21% of chest X-rays done outside of normal working hours are reported within 4 hours. However, while emergency physicians are relatively accurate at identifying common radiographic abnormalities such as consolidation and congestion, the literature suggests discover that less common findings such as coin lesions suggestive of lung malignancy are often missed, with 10 out of 13 lesions missed in this study. (Gatt 2003) Furthermore, acute medical conditions such as pneumonia or exacerbations of congestive heart failure tend to be treated on the basis of their clinical presentation in addition to their radiological findings, reducing the number of potentially improperly treated patients.

While this is good news for such acute conditions, sub-acute conditions such as lung malignancies may be lost to follow up if good communication is not maintained between the radiology department and emergency department following the discharge of a patient. A retrospective study found that out of 58 patients with histologically proven lung cancer, 14 (28%) had prior chest X-ray abnormalities which were not recognised or followed upon. (Turkington 2002) Chest X-rays up to 5 years prior to the diagnostic X-ray were assessed by a respiratory physician and radiologist blinded to the site of the abnormality to identify missed abnormalities. The study further points out that out of the reasons for missing abnormalities, the most common was the lack of diagnosis rather than the lack of follow up or ambiguity within the report.

Study rationale

With the use of radiology in the hospital setting increasing, demands on radiology services to interpret X-rays swiftly and accurately will grow, particularly for after hours and emergency use. To aid junior staff as well as other non-radiologists, we propose to develop a computer aided diagnosis system capable of autonomously diagnosing common and clinically relevant conditions on chest and abdominal X-rays. To maximise the utility of this system for medical staff, we intend to use recent advances in deep learning to illuminate the thought processes of the computer aided system, allowing medical staff to examine the reliability of the system’s prediction and logic.

Furthermore, we intend to show that deep learning approaches in medical imaging, particularly in X-rays, are practically achievable from existing data in clinical data warehouses. To the authors knowledge, no other group has demonstrated both training and evaluation of a deep learning system on chest X-rays from pre-existing clinical data warehouses. By using existing clinical data instead of specially prepared research datasets, we hope to optimise the accuracy of the resulting model to the particular image characteristics and patient population of this institution as well as leverage the large dataset available to us.

In addition to augmenting radiographic diagnosis, a fully autonomous system can have several other applications, such as enabling the construction of future early warning systems designed to detect complications of care, such as that proposed in a prior study at the Alfred - “Project 429/09: Towards an automated surveillance system for invasive fungal infections using existing hospital information systems”. Furthermore, the proposed system can also act as an image recognition and retrieval system, enabling content based image retrieval in addition to existing techniques for searching through report texts. This can aid in case discovery and audit, contributing to teaching as well as quality control.