3.2.3. Gradient Boosting
The Gradient Boosting algorithm is a machine learning technique for regression and classification problems, which produces a prediction model in the form of a set of weak prediction models, usually decision trees. It builds the model in steps like other reinforcement methods, and generalizes them by allowing the optimization of an arbitrary differentiable loss function [41, 42]. The algorithm’s objective is to create a chain of weak models, where each one aims to minimize the error of the previous model through a loss function. The adjustments of each weak model are multiplied by a value called the learning rate. This value aims to determine the impact of each tree on the final model. The lower the value, the lower the contribution of each tree. Scikit-Learn has precisely and effectively implemented the Gradient Boosting algorithm for solving classification and regression problems. In this work, the Gradient Boosting algorithm is analyzed in a classification model. The data is provided by the library itself, which has a Dataset package. Table 6 shows the results of Gradient Boosting technique. The precision and accuracy of Gradient Boosting technique is tested by taken random data from Table 3. Table 6 analyzes the model’s accuracy metrics in relation to the experimental data. The first row presented in Table 6 is the behavior of the explained variance. The explained variation measures the proportion for which a mathematical model is responsible for. Randomly taken values from Table 3 are written in Table 7 and calculated also by the model. The accordance between the model’s and experimental values are approximately 92%, 95%, and 86% for KD, E and Z values, respectively. In practice, these percentages show how accurate is the model. The second row presented in Table 6 is the mean absolute error (MAE). In statistics, the MAE is a measure of errors between observation pairs that expresses the same phenomenon. A lower MAE value is desirable for an accurate model.