Abstract

In this summer project, we perform an analysis of the intermediate layer neurons primarily through the use of CLIP-dissect, a powerful model to describe neurons, and its Soft-WPMI similarity function. Specifically, we first validate the relative accuracy of CLIP-dissect when analyzing intermediate layer neurons using ground-truth labeling, before using the large pretrained model and the Soft-WPMI similarities to perform analysis of convolutional neural networks (CNNs) such as ResNet-50. We then create an interactive GUI with the visualizations of model, layer, and neuron-level analysis to allow anyone to access the intermediate neurons of these CNNs using output from CLIP-dissect and images from Broden. We allow users to directly search for specific concepts within any network so that they can better understand the inner workings of the models that increasingly define our daily life. Our website with an interactive version of this project can be found at https://dr4nx.github.io/clip-search/index.html.