This study proposed a method for interpreting immunohistochemistry (IHC) images based on a graph convolutional network (GCN). Self-supervised transfer learning was employed to obtain cell nucleus segmentation masks, providing effective strong cues for a cell nucleus graph (CN-G). This study applys a GCN to end-to-end diagnostic classification tasks for IHC images, fully considering global distribution features and local details in images. We believe that our study makes a significant contribution to the literature because the proposed approach ensures high accuracy in the relevant tasks while addressing the challenges of the lack of labeled datasets and high number of sample pixels.