Linear regressions |
Regression |
The simplest form of regression
models. Attempts to minimize the mean squared error (MSE) between actual
target values and the target values estimated by fitting a linear
equation to the training set. |
[114, 120, 121, 123] |
k-nearest neighbors (KNN)
|
Classification
Regression
|
A simple ML algorithm that stores k nearest neighbor samples in a
dataset (k=1, 2, 3, …) based on the feature similarities.
|
[81, 108, 116]
|
Support vector machines (SVMs)
|
Classification
Regression
|
The objective of SVM is to transform each data in an n-dimensional space
(n: number of features) and separate data points into two categories in
a manner that maximizes the width of the gap between the nearest
observations. SVMs are divided into two main groups: Support vector
regressors (SVRs) and support vector classifiers (SVCs).
|
[103, 104, 115, 116, 119]
|
Decision trees and random forest
|
Classification
Regression
|
Decision trees provide a tree-shaped structure, including nodes and
branches. The algorithm makes decisions based on a hierarchy of if/else
questions. Each node classifies the input depending on the question, and
the branches are the final decisions representing the output.
Random forest is an ensemble method that collects many randomly made
decision trees. This algorithm aims to overcome the limitations of each
individual decision tree and averages their results.
|
[106, 115, 116]
|
Gaussian process (GP)
|
Classification
Regression
|
GP is a probabilistic non-parametric method that aims to make
predictions and provide uncertainty information on the estimations based
on Bayes’ rule. Gaussian process classification (GPC) and Gaussian
process regression (GPR) are two main groups of GP.
|
[107]
|
Fuzzy logic (FL) |
Regression |
FL resembles the pattern of human
reasoning for solving problems considering all available possibilities
between Yes and No. |
[105] |
Artificial neural networks (ANNs)
|
Classification
Regression
|
ANNs are one of the most popular ML algorithms which are inspired by the
human brain information process. ANNs are consist of interconnected
neurons arranged in an input layer, a series of hidden layers, and an
output layer. In these algorithms, each neuron makes decisions and gives
it to other neurons in the next layer. The weighted sum of the inputs is
calculated by an activation function. Then, the procedure tends to
update the weights in order to minimize the prediction errors.
|
[99-102, 106, 115, 117, 122]
|