In machine learning, feature selection is of utmost importance for augmenting the predictive capabilities of ensemble models. This paper presents an innovative hybrid framework for selecting features in ensemble models, which combines Rough Set Theory (RST) with Recursive Feature Elimination (RFE), complemented by Association Rule Mining, to enhance interpretability. The suggested method considerably improves ensemble models’ prognostic accuracy and comprehensibility, particularly Random Forests and Gradient Boosting Machines. The framework starts with the RFE process, meticulously eliminating less influential features, and then applies RST to refine the feature set further by eliminating redundancies. This two-phase approach results in a feature set that is optimally reduced yet highly influential. By implementing this hybrid method on ensemble models, significant improvements in predictive accuracy are demonstrated across three diverse datasets: cancer, Pima Indians Diabetes, and a weather dataset from Underground. The accomplished accuracies for these datasets were 0.9663, 0.8793, and 0.8427, respectively, highlighting the proposed approach’s effectiveness. This article also proposes the incorporation of association rule mining to analyze the outcomes of the models. This technique improves the understandability of the models, offering more profound insights into the connections and patterns, thus tackling the difficulty of interpretability in intricate ensemble models. Our empirical analysis confirms the effectiveness of the proposed hybrid feature selection model, representing a significant advancement in the field. The integration of RFE and RST optimizes the feature selection process and bridges the gap in interpretability, offering robust solutions for applications where accuracy and understanding of model decisions are crucial.

Recently, the ever-increasing complexity of datasets has necessitated the development of sophisticated techniques to uncover meaningful patterns and interactions within the data. This paper investigates the synergy between Rough Set Theory and Association Rule Mining, which is a potent approach to detecting interactions and enhancing the prediction capabilities of machine learning models. The proposed framework leverages the Greedy Heuristic Method for reduct generation, an established technique in Rough Set Theory, to efficiently identify relevant features and reduce the dimensionality of the dataset. Furthermore, Association Rule Mining extracts association rules from the data, revealing interesting relationships and dependencies among the features. These association rules are transformed into binary values, representing the detected interactions, to create a concise yet informative representation of the data’s intrinsic relationships. This binary representation is ideal for integration into machine learning models, enabling them to exploit the discovered interactions and gain a more comprehensive understanding of the underlying patterns. To assess the effectiveness of our proposed framework, we propose a comprehensive experiment involving a weather dataset scraped from www.wunderground.com for Kariki farm in the Juja sub-county, Kiambu County, Kenya. Using detected interactions, we modelled them to base machine learning models, including Naive Bayes, Decision Trees, Support Vector Machines (SVM), and Logistic Regression models. We compared the performance of these models while using the detected interactions versus not using the detected interactions. Through extensive experimentation, we demonstrate that our proposed approach is more effective than traditional machine learning models without interaction detection. Our results indicate that our interaction detection method framework significantly improves the prediction accuracy of the tested models on the benchmark datasets. This enhancement in accuracy highlights the practical relevance and potential benefits of adopting our approach to uncover valuable insights from datasets.

AbstractFeature interaction is a crucial concept when it comes to machine learning interpretability. These interactions make it easier for the person using a machine learning tool to know how the model made its prediction. Intrinsic models such as the Generalized linear models pull their interpretability aspects from detecting feature interactions on the data. However, these models do not search the whole sample space for interactions and assume all interactions to the target feature are identical. A hybrid model combining Rough Set and Generalized linear models is proposed. Rough Set uses the concept of information granulation and approximations of regions with meaningful information to find feature interactions in the whole sample space. The detected features will then be modeled onto a Generalized linear model for prediction purposes. The dataset used in the experiment was drawn from wunderground.com online weather site, specifically the Kariki Farm online weather station. The proposed methodology for the research is the CRISP-DM method.Keywords: Machine interpretability, Rough Set, Generalized linear model, weather prediction.

Sentiment analysis of social media posts and texts can provide information and knowledge that is applicable in social settings, business intelligence, evaluation of citizens’ opinions in governance and mood triggered devices in Internet of Things. Feature extraction and selection is a key determinant of accuracy and computational cost of machine learning models for such analysis. Most feature extraction and selection techniques utilize bag of words such as N-grams and frequency-based algorithms especially Term Frequency-Inverse document frequency (TF-IDF). However, these approaches suffer shortcomings such as; they do not consider relationships between words, they ignore words’ characteristics and they suffer high feature dimensionality. In this paper we propose and evaluate an approach that utilizes a fixed hybrid N-gram window for feature extraction and Minimum Redundancy Maximum Relevance feature selection for sentence level sentiment analysis. The approach improves the existing feature extraction techniques specifically the N-gram by generating a tri-gram vector from words, Part of speech tags and word semantic orientation. The N-gram vector is extracted by employing a static 3-gram window identified by a lexicon where a sentiment word appears in a sentence. A blend of the words, POS tags and the sentiment orientations of the 3N-gram are used to build the feature vector. The optimal features from the vector are then selected using Minimum Redundancy Maximum Relevance (MR2) algorithm. Experiments were carried out with a publicly available yelp tweets dataset to evaluate the performance of four supervised machine learning classifiers (Naïve Bayes, K-Nearest Neighbor, Decision Tree and Support Vector Machines) when augmented with the proposed model. The results showed that the proposed model had the highest accuracy (86.85%), recall (86.85%) and precision (86.96%).

In Sub-Saharan Africa, Professionals visually analyse the plants by looking for disease markers on the leaves to diagnose cassava infections, however, this method is extremely subjective. Automating the identification and classification of crop diseases may improve the accuracy of professional disease diagnosis and enable farmers in remote areas to monitor their crops without the assistance of experts. Algorithms for machine learning have been used in the early detection and classification of crop diseases. Motivated by the current developments in the field of Gaussian Processes, this study proposes to integrate the transfer learning approach with a deep Gaussian convolutional neural network model (DGCNN) for the detection and classification of cassava diseases. During this study, we used MobileNet V2 and VGG16 pre-trained transfer learning models and a hybrid kernel. Experiments with MobileNet V2 and a hybrid kernel revealed an accuracy of 90.11%. Also, experiments with VGG16 and a hybrid kernel revealed an accuracy of 88.63%. The major limitation of this study was computing resources since we used an ordinary computer in all our experiments. In our future work, we will experiment with the three kernel functions used in this study with kernel algorithms such as support vector machines and compare the results with those obtained during this study.