Human pose estimation has a variety of applications in action recognition, human-robot interaction, motion capture, augmented reality, sports analytics, and healthcare. There is a substantial stream of datasets and deep learning-based models to attain robust human pose estimation within the visible domain. Nonetheless, there are certain obstacles in this domain, including insufficient illumination and privacy concerns. These issues can be addressed using thermal cameras. However, only a limited number of annotated thermal human pose datasets are available to train data-hungry deep learning models. In this regard, we introduce a novel open-source thermal human pose dataset named OpenThermalPose. The dataset contains 6,090 thermal images and 14,315 annotated human instances. The annotations include bounding boxes and 17 anatomical keypoints, following the annotation format of the MS COCO dataset. The dataset covers various fitness exercises, multiple-person activities, and outdoor walking in different locations and weather conditions. As a baseline, we trained and evaluated YOLOv8-pose models on our dataset. We have made the dataset, source code, and pre-trained models publicly available at https://github.com/IS2AI/OpenThermalPose to bolster research in this area.

According to the World Health Organization (WHO), hundreds of million people have some type of visual disability. Vision impairment has a personal impact with lifelong consequences because more than 80% of our perception, cognition, learning, and daily activities are mediated through vision. Moreover, in the era of rapid advancements in artificial intelligence (AI), visually impaired and blind people face challenges at work and in education because of inaccessibility to AI technologies. In this regard, we present an assistive mobile application with an intuitive user interface (UI) for visually impaired and blind people to interact with ChatGPT via natural conversation. The app employs automatic speech recognition (ASR), text-to-speech (TTS), keyword spotting (KWS), voice activity detection (VAD), and a convenient UI to interact with ChatGPT effortlessly. We have made the source code, pre-trained models, and UI publicly available at https://github.com/IS2AI/talk-llm to stimulate the development of assistive mobile applications.

Visually impaired and blind people often face a range of socioeconomic problems that can make it difficult for them to live independently and participate fully in society. Advances in machine learning pave new venues to implement assistive devices for the visually impaired and blind. In this work, we combined image captioning and text-to-speech technologies to create an assistive device for the visually impaired and blind. Our system can provide the user with descriptive auditory feedback in the Kazakh language on a scene acquired in real-time by a head-mounted camera. The image captioning model for the Kazakh language provided satisfactory results in both quantitative metrics and subjective evaluation. Finally, experiments with a visually unimpaired blindfolded participant demonstrated the feasibility of our approach.

Speech Command Recognition (SCR) is rapidly gaining prominence due to its diverse applications, such as virtual assistants, smart homes, hands-free navigation, and voice-controlled industrial machinery. In this paper, we present a data-centric approach to creating SCR systems for low-resource languages, particularly focusing on the Kazakh language. By leveraging synthetic data generated by Text-to-Speech (TTS) and data extracted from a large-scale speech corpus, we successfully created the Kazakh language equivalent of the Google Speech Commands dataset. Moreover, we also compiled the Kazakh Speech Commands dataset with data collected from 119 participants. This dataset was used to benchmark the performance of the Keyword-MLP model trained using our synthetic dataset. The results showed that the model achieves 89.79% accuracy for the real-world data demonstrating the efficacy of our approach. Our work can serve as a recipe for creating customized speech command datasets, including for low-resource languages, obviating the need for laborious and costly human data collection.

Face detection is a mandatory step in many computer vision applications, such as face recognition, emotion recognition, age detection, virtual makeup, and vital sign monitoring. Thanks to advancements in deep learning and the introduction of annotated large-scale datasets, numerous applications have been developed for human faces. Recently, other domains, such as animals and cartoon characters, have started gaining attention but still lag far behind human faces. The biggest challenge is the limited number of annotated face datasets in these domains. The manual labeling of large-scale datasets is tedious and requires substantial human labor. In this regard, we present an input-agnostic face detector to ease the annotation of various face datasets. We propose a simple but effective data-centric approach instead of building a specific neural network architecture. Specifically, we trained a face detection model, YOLO5Face, on human, animal, and cartoon face datasets. The experiments show that the model can achieve accurate results in all domains. In addition, the model achieved decent results for animals and cartoon characters different from the ones in the training set. This implies that the model can extract agnostic facial features. We have made the source code and pre-trained models publicly available at https://github.com/IS2AI/AnyFace to stimulate research in these fields.

This paper addresses sequential indoor localization using WiFi and Inertial Measurement Unit (IMU) modules commonly found in commercial off-the-shelf smartphones. Specifically, we developed an end-to-end neural network-based localization system integrating WiFi received signal strength indicator (RSSI) and IMU data without external data fusion models. The developed system leverages the advantages of WiFi and IMU modules to locate finer-level sequential positions of a user at 150 Hz sampling rate. Additionally, to demonstrate the efficacy of the proposed approach, we created the IMUWiFine dataset comprising IMU and WiFi RSSI readings sequentially collected at fine-level reference points. The dataset contains 120 trajectories covering an aggregate distance of over 14 kilometers. We conducted extensive experiments using deep learning models and achieved a mean error distance of 1.1 meters on an unseen evaluation set, which makes our approach suitable for many practical applications requiring meter-level accuracy. To enable experiment and result reproducibility, we made the developed localization system and IMUWiFine dataset publicly available in our GitHub repository.

Face detection and subsequent localization of facial landmarks are the primary steps in many face applications. Numerous algorithms and benchmark datasets have been introduced to develop robust models for the visible domain. However, varying conditions of illumination still pose challenging problems. In this regard, thermal cameras are employed to address this problem, because they operate on longer wavelengths. However, thermal face and facial landmark detection in the wild is an open research problem because most of the existing thermal datasets were collected in controlled environments. In addition, many of them were not annotated with face bounding boxes and facial landmarks. In this work, we present a thermal face dataset with manually labeled bounding boxes and facial landmarks to address these problems. The dataset contains 9,982 images of 147 subjects collected under controlled and uncontrolled conditions. As a baseline, we trained the YOLOv5 object detection model and its adaptation for face detection, YOLO5Face, on our dataset. In addition to our test set, we evaluated the models on the external RWTH-Aachen thermal face dataset to show the efficacy of our dataset. We have made the dataset, source code, and pre-trained models publicly available at https://github.com/IS2AI/TFW to bolster research in thermal face analysis.