Automatic modulation recognition (AMR) plays an important role in various communications systems. It has the ability of adaptive modulation and can adapt to various complex environments. Automatic modulation recognition is also widely used in orthogonal frequency division multiplexing (OFDM) systems. However, because the recognition accuracy of traditional methods to extract the features of OFDM signals is very limited. In order to solve these problems, many deep learning based AMR methods have been proposed to improve the recognition performance. However, most of these AMR methods neglect the harmful effect by carrier phase offset (PO) which often appears in real communications systems. Hence it is required to consider the PO effect for designing the OFDM system. Unlike conventional methods, we propose a convolutional neural network (CNN) based AMR method for considering PO in the OFDM system. The proposed method is used to eliminate the PO to achieve the high classification accuracy. Experiment results are provided to confirm the proposed method when comparing to conventional methods.