4.1 Intra- and inter-slice HAN
Our intra- and inter-slice HAN (hereafter patch-HAN) was designed for
abnormality detection using a collection of slices from a single MRI
sequence (Fig. 2). The model treated
each scan as being hierarchically composed of a number of slices,
Nslice, with each slice
itself composed of a number of local regions (patches). The number of
patches, Npatch, was
a model hyper-parameter, with lots of small patches improving
abnormality localisation,
but increasing the computational cost. A CNN looped through all patches
in a given slice,
processing each sequentially before passing its output to a
bidirectional LSTM unit which
built up an internal representation of what it had seen in that slice.
The LSTM had Npatchoutputs per slice, and these were passed to an attention network to
calculate the importance
of each patch. The resulting weighted sum of patches become the
representation for that
slice. Following (Yang et al., 2016) the attention weights were computed
as follows:
\(u_{\text{it}}\ =\ \tanh(W_{w}h_{\text{it}}\ +\ b_{w})\)
\begin{equation}
\alpha_{\text{it}}\ =\ \frac{exp(u_{\text{it}}^{T}u_{w})}{\sum_{t}{exp(u_{\text{it}}u_{w})}}\nonumber \\
\end{equation}\(s_{i\ =\ \sum_{t}{\alpha_{\text{it}}h_{\text{it}}}}\) (Eqn. 1)
where si is the representation for thei ’th slice, and αit is the weighting of the t ’th patch
representation in slice i , hit. The new parameters to learn were
therefore a context vectoruw , a matrix Ww and a biasbw for each attention module. This procedure was
repeated for
all slices, with a second bidirectional LSTM taking these slice
representations, outputting
Nslice hidden states (the number of slices) which were
sent to another attention module
to compute the importance of each slice. Finally, a weighted combination
of slices (i.e., a
weighted combination of a weighted combination of local patches) was
sent to a single layer
classifier which was trained by minimizing the binary cross-entropy at
the level of series
labels (i.e., all slices in a series have the same label; 1 for
abnormal, and 0 for normal)