3.2.3. Gradient Boosting
The Gradient Boosting algorithm is a machine learning technique for
regression and classification problems, which produces a prediction
model in the form of a set of weak prediction models, usually decision
trees. It builds the model in steps like other reinforcement methods,
and generalizes them by allowing the optimization of an arbitrary
differentiable loss function [41, 42]. The algorithm’s objective is
to create a chain of weak models, where each one aims to minimize the
error of the previous model through a loss function. The adjustments of
each weak model are multiplied by a value called the learning rate. This
value aims to determine the impact of each tree on the final model. The
lower the value, the lower the contribution of each tree. Scikit-Learn
has precisely and effectively implemented the Gradient Boosting
algorithm for solving classification and regression problems. In this
work, the Gradient Boosting algorithm is analyzed in a classification
model. The data is provided by the library itself, which has a Dataset
package. Table 6 shows the results of Gradient Boosting technique. The
precision and accuracy of Gradient Boosting technique is tested by taken
random data from Table 3. Table 6 analyzes the model’s accuracy metrics
in relation to the experimental data. The first row presented in Table 6
is the behavior of the explained variance. The explained variation
measures the proportion for which a mathematical model is responsible
for. Randomly taken values from Table 3 are written in Table 7 and
calculated also by the model. The accordance between the model’s and
experimental values are approximately 92%, 95%, and 86% for
KD, E and Z values, respectively. In practice, these
percentages show how accurate is the model. The second row presented in
Table 6 is the mean absolute error (MAE). In statistics, the MAE is a
measure of errors between observation pairs that expresses the same
phenomenon. A lower MAE value is desirable for an accurate model.