Introduction
Human neuroscience has entered an era of “Big Data” \cite{Van_Horn_2013}: the assembly of large aggregated datasets, or large datasets containing measurements from many individuals, collected through consortium efforts, such as the Human Connectome Project promise to enhance our understanding of the relationship between brain anatomy and brain activity. The field is experiencing a “paradigm shift” \cite{Fan_2014}, where our once established scientific procedures are morphing, as dictated by the new challenges posed by large datasets. We’ve seen a shift from desktop computers to cyberinfrastructure \cite{Horn2013}, from small studies siloed in individual labs to an explosion of data sharing initiatives \cite{Ferguson_2014,Poldrack_2014}, and an overall shift in statistical thinking and computational methods \cite{Fan2014} that can accommodate large datasets. But one, often overlooked, aspect of our scientific protocols in neuroimaging has not yet evolved to the needs of Big Data: expert decision making.
Specifically, neuroimaging expert decisions made through visual inspection of the data that are not (yet) automated cannot be reliably scaled to large datasets. These are decisions made by those with experience in imaging and neuroanatomy, like tracing of regions of interest (ROIs), the editing of fascicle models from streamline tractography \cite{Jordan_2017}, the evaluation of cross-modality image alignment, and quality control of images at each stage of processing. On large datasets, especially longitudinal multisite consortium studies, these expert decisions cannot be reliably replicated because the timeframe of these studies is long, and individual experts get fatigued.
One solution to this issue is to train machines to emulate experts’ decisions, but there are many cases in which even where automated algorithms exist, expert decision-making is still required. For example, a variety of image segmentation algorithms have been developed to replace manual ROI editing, with Freesurfer \cite{fischl2012freesurfer}, FSL \cite{Patenaude_2011}, ANTS \cite{Avants_2011}, and SPM \cite{Ashburner_2005} all offering automated segmentation tools for standard structures. But these algorithms were developed on a specific type of image (T1-weighted) and on a specific type of brain (those of healthy controls). Pathological brains, or those of children or the elderly may violate the assumptions of these algorithms, and their outputs often require manual expert editing anyway. Similarly, in tractography, a set of anatomical ROIs can be used to target or constrain streamlines to automatically extract fascicles of interest \cite{CATANI_2008}. But again, abnormal brain morphometry resulting from pathology would still require expert editing \cite{Jordan_2017a}. The delineation of retinotopic maps in visual cortex is another task that has been recently automated \cite{Benson2014,Benson2012}, but these procedures are limited to only a few of the known retinotopic maps and substantial expertise is still required to delineate the other known maps \cite{Winawer2017,Wandell2011}. Finally, one of the first steps of image processing pipelines is quality assurance/quality control: there are several automated methods to quantify image quality, based on MRI physics and the statistical properties of images, and these have been collected under one umbrella in an algorithm called MRIQC \cite{Esteban2017}, but these methods are specific to T1-weighted images, and cannot generalize to other image modalities. To address all of these cases, we need a more general-purpose framework that can train machines to emulate experts for any purpose.
Deep learning via convolutional neural networks (CNNs) have shown promise in a variety of biomedical image processing tasks. Modeled loosely on the human visual system, CNNs can be trained for a variety of image classification and segmentation tasks using the same architecture. For example, the U-Net \cite{ronneberger2015u} which was originally built for segmentation of neurons in electron microscope images, has also been adapted to segment macular edema in optical coherence tomography images \cite{Lee_2017}, to segment breast and fibroglandular tissue \cite{Dalm__2017}, and a 3D adaptation was developed to segment the Xenopus kidney \cite{cciccek20163d}.