Figure 5: Sequence scores for two test set studies. Also included are the most informative slices for each sequence. In both cases the sequence attention weights closely match the visibility of the abnormality on the corresponding sequence.
Given the discriminative power of T2-weighted images for this task, we trained the patch- HAN using only these images to additionally localise abnormalities within MRI slices; the results also appear in Table 1. The hierarchical model outperforms all single-sequence baseline architectures. Figure 6 displays the slice and patch attention weights for two test set examples. In both cases, the top left plot displays the distribution of attention weights over each slice and the top right plot shows the most informative patch for the most informative slice. For reference the raw slices are shown as well. Again, the slice attention scores broadly agree with the spatial distribution of the abnormality across slices. The classifier seems able to classify and localise abnormalities across the entire brain volume (Fig 7), and even gave outputs with multi-modal distributions in cases where multiple, spatially separate abnormalities were present.