This study assessed species diversity, abundance, and habitat association of medium and large-sized mammals in the Gelama block of Arsi Mountains National Park, Southeast Ethiopia. The study covered two seasons extending between February and August 2020. Based on the vegetation cover and topographic variation, the study area was stratified into four habitat types. A total of 74 transect lines were systematically established, 14 for Afroalpine, 26 for Ericaceous, 18 for Natural forest, and 16 for Mixed plantation. The survey for mammals was done by walking on foot along each transect. A total of 25 medium and large-sized mammals were recorded, including globally threatened and endemic mammals like Canis simensis and Tragelaphus buxtoni. Results showed a significant difference in the abundance of medium and large-sized mammals among the four habitat types: dry (χ2 =126.256, df =3, P=0.000) and wet (χ2 =74.822, df =3, P=0.000) seasons. During the dry season, the highest and lowest species diversity was recorded in the natural forest (H’=2.488) and mixed plantations (H’=2.154). During the wet season, ericaceous vegetation sheltered the highest diversity of mammalian species (H’=2.224), while mixed plantation forests inhabited the lowest species diversity (H’=1.788). The presence of food, cover, and water availability were the main factors for the differences in species diversity and abundance across the habitats. Phacochoerus africanus was the most abundant species during the dry (16.89%) and wet (18.81%) seasons. However, Felis caracal (0.14 during dry) and Panthera pardus (0.36 during wet season) were the least abundant species recorded. Occurrence of abundant species was associated with their feeding habits and physiological adaptation. Although the study area is home to different species of mammals, including endangered and endemics, livestock grazing and human encroachments are evident, putting strain on flora and fauna. Hence, conservation measures should be taken to ensure long-term conservation of the area.