Figure 5: Sequence scores for two test set studies. Also
included are the most informative slices for each sequence. In both
cases the sequence attention weights closely match the visibility of the
abnormality on the corresponding sequence.
Given the discriminative power of T2-weighted images for this task, we
trained the patch-
HAN using only these images to additionally localise abnormalities
within MRI slices; the
results also appear in Table 1. The hierarchical model outperforms all
single-sequence
baseline architectures. Figure 6 displays the slice and patch attention
weights for two test set
examples. In both cases, the top left plot displays the distribution of
attention weights over
each slice and the top right plot shows the most informative patch for
the most informative
slice. For reference the raw slices are shown as well. Again, the slice
attention scores
broadly agree with the spatial distribution of the abnormality across
slices. The classifier seems able to classify and localise abnormalities
across the entire brain volume (Fig 7), and
even gave outputs with multi-modal distributions in cases where
multiple, spatially separate
abnormalities were present.