In current scenario islanding in smart grid a big challenge that results, in various uncertainties in the system parameters, also leads to degrade the power quality (PQ), and can also become threat to the maintenance workers. In this study a new passive islanding detection technique for grid-connected distributed generation (DG) units is proposed. The presented method employs an Unscented Kalman filter (UKF) to extract and filter the harmonic contents of the voltage signal measured at DG end. A three-phase voltage signal is measured at PCC (point of common coupling) and then it be taken as a test signal for analysis of islanding event detection. Here, at first a residual signal is produced using UKF to detect the changes occurring in the power system. Then, at second step, a total harmonic distortion (THD) is estimated by the UKF). The variation of THD classifies between islanding events and normal events. The IEEE 9-bus test system simulated in Matlab/Simulink is used as a test bed to assess the performance of the proposed approach. The proposed method is enormously analysed under various islanding and non-islanding scenarios. The results obtained demonstrated that the proposed method can successfully differentiate between the two events. Moreover, it also provides high reliability by eliminating the non-detection zone (NDZ) and stands robust against any mal-operation.