4.1 Intra- and inter-slice HAN
Our intra- and inter-slice HAN (hereafter patch-HAN) was designed for abnormality detection using a collection of slices from a single MRI sequence (Fig. 2). The model treated each scan as being hierarchically composed of a number of slices, Nslice, with each slice itself composed of a number of local regions (patches). The number of patches, Npatch, was a model hyper-parameter, with lots of small patches improving abnormality localisation, but increasing the computational cost. A CNN looped through all patches in a given slice, processing each sequentially before passing its output to a bidirectional LSTM unit which built up an internal representation of what it had seen in that slice. The LSTM had Npatchoutputs per slice, and these were passed to an attention network to calculate the importance of each patch. The resulting weighted sum of patches become the representation for that slice. Following (Yang et al., 2016) the attention weights were computed as follows:
\(u_{\text{it}}\ =\ \tanh(W_{w}h_{\text{it}}\ +\ b_{w})\)
\begin{equation} \alpha_{\text{it}}\ =\ \frac{exp(u_{\text{it}}^{T}u_{w})}{\sum_{t}{exp(u_{\text{it}}u_{w})}}\nonumber \\ \end{equation}
\(s_{i\ =\ \sum_{t}{\alpha_{\text{it}}h_{\text{it}}}}\) (Eqn. 1)
where si is the representation for thei ’th slice, and αit is the weighting of the t ’th patch representation in slice i , hit. The new parameters to learn were therefore a context vectoruw , a matrix Ww and a biasbw for each attention module. This procedure was repeated for all slices, with a second bidirectional LSTM taking these slice representations, outputting Nslice hidden states (the number of slices) which were sent to another attention module to compute the importance of each slice. Finally, a weighted combination of slices (i.e., a weighted combination of a weighted combination of local patches) was sent to a single layer classifier which was trained by minimizing the binary cross-entropy at the level of series labels (i.e., all slices in a series have the same label; 1 for abnormal, and 0 for normal)