Data Preparation

All images were then converted into a set of 2D axial slices using the NiBabel python library and uploaded to https://braindr.us. Two images of the 724 were corrupted, so the total image count became 722 images. Five slices, spaced 40 slices apart from each other, were selected from each brain, where the first slice was one that had over 10,000 non-zero pixels. All slices were padded to 256x256 or 512x512 depending on original image size. One subject (sub-NDARVJ504DAA) had only 4 slices because the last slice did not meet the 10,000 pixel threshold. The total number of slices uploaded to https://braindr.us was 3609. 

The Braindr web application

The braindr application was written in Javascript using the Vue.js (https://vuejs.org) framework. Google Firebase (https://firebase.google.com/) was used for the realtime database. The axial brain slices were hosted on Amazon S3 and served over the Amazon CloudFront content delivery network. Figure \ref{186072} shows the braindr interface, which simply shows a 2D slice. On a desktop, a user may click the "pass" or "fail" button or use the right or left arrow keys to rate the image. On a touchscreen device (tablet or mobile phone), users can swipe right or swipe left to pass or fail the image. The user receives a point for each rating, unless they rate against the majority, where the majority is defined as any image with more than 5 ratings, where the average rating is below 0.3 or above 0.7. The user receives a notification of the point they earned (or did not earn) for each image after each swipe. All users electronically signed a consent form through the University of Washington IRB. Images were initially served randomly, and then images with fewer ratings were preferentially served.