Unmanned Aerial Vehicle (UAV) detection in real-time is an emerging field of study that focuses on computer vision and deep learning algorithms. However, the increasing use of UAVs in numerous applications has generated worries about possible risks and misuse. The purpose of this research is to detect UAVs, under adverse weather conditions (such as rain) and image distortions (such as motion blur and noise). The goal is to examine how these adverse conditions affect UAV detection performance and to provide techniques to increase model robustness. To achieve this, a custom training dataset was constructed by combining multiple existing datasets, supplementing them with complex backgrounds. In addition, a custom testing dataset was generated containing UAV images affected by adverse conditions.  On the proposed dataset, the performance of well-known object detection algorithms including YOLOv5, YOLOv8, Faster-RCNN, RetinaNet, and YOLO-NAS was investigated. In comparison to clean images, the results demonstrated a considerable performance decrease under adverse conditions. However, training the models on the augmented dataset containing samples of distorted and weather-affected images significantly enhanced the models' performance under challenging settings. These findings highlight the importance of taking adverse weather conditions into account during model training and underscore the significance of data enrichment for improving model generalization. The work also accentuates the need for further research into advanced techniques and architectures to ensure reliable UAV detection under extreme weather conditions and image distortions. (Note: This is a pre-print of a paper submitted to IEEE for potential journal publication and final version may vary upon acceptance and publication)