In this paper, we study the use of the angle of arrival (AoA) as a feature for performing robust, machine learning (ML)-based physical layer authentication (PLA). In fact, whereas most previous research on PLA relies on physical properties such as channel frequency/impulse response or received signal strength, the use of the AoA in this context has not yet been studied in depth as a means of providing resistance to impersonation (spoofing) attacks. In this study, we first prove that an effective impersonation attack on AoA-based PLA can only succeed under very stringent conditions on the attacker in terms of location and hardware capabilities, and thus, the AoA can in many scenarios be used as a robust feature for PLA. In addition, we exploit machine learning in our study to perform lightweight, model-free, intelligent PLA. We show the effectiveness of the  proposed AoA-based PLA solutions by testing them on experimental outdoor massive multiple input multiple output data.