3.3 Comparing machine learning algorithms
Figure 2 shows boxplots of MSEs for the training data
(MSEtrain), crossed validation
(MSEtest), and doubled validation
(MSEval) between different machine learning
architectures. Leaning calculations were carried out ten times in each
machine learning. For estimating cell yields, the
MSEtrain of PLS, RF, NN, and DNN recorded
1.10×10−2 ± 9.9×10−4,
1.62×10−3 ± 1.12×10−3,
2.05×10−3 ± 1.40×10−3, and
7.30×10−4 ± 9.00×10−4 as the means ±
standard deviations, respectively. The MSEtest of PLS,
RF, NN, and DNN recorded 7.58×10−2 ±
2.09×10−2, 9.78×10−2 ±
7.10×10−2, 3.67×10−2 ±
2.33×10−2, and 4.90×10−3 ±
5.30×10−3, respectively. The MSEval of
PLS, RF, NN, and DNN recorded 9.84×10−1 ±
5.57×10−1, 5.70×10−1 ±
8.87×10−2, 1.16×10−2 ±
1.29×10−2, and 3.43×10−3 ±
3.37×10−3, respectively. For estimating the GFP
yields, the MSEtrain of PLS, RF, NN, and DNN were
1.66×10−2 ± 2.85×10−3,
1.08×10−2 ± 6.70×10−4,
2.84×10−2 ± 3.43×10−2, and
6.26×10−3 ± 1.31×10−2, respectively.
The MSEtest of PLS, RF, NN, and DNN were
7.04×10−1 ± 1.23×10−1,
6.96×10−2 ± 4.48×10−2,
4.12×10−2 ± 2.30×10−2, and
8.70×10−3 ± 7.70×10−3, respectively.
The MSEval of PLS, RF, NN, and DNN were
8.28×10−1 ± 5.25×10−1,
3.55×10−1 ± 4.36×10−2,
6.10×10−2 ± 2.67×10−2, and
9.69×10−3 ± 1.31×10−2, respectively.
To summarize the results of the model fitting, MSEtrain,
MSEtest, and MSEval were observed as the
smallest values in DNN in the calculated machine learning architectures,
and were observed higher values in the others.
Figure 3 shows plots of the measured and predicted values of
the best model for each machine learning analysis. For the PLS model,
the coefficients of determination for the training data
(R2train) were 0.961 and 0.958 for
cell growth and GFP yields, respectively. The coefficients of
determination for the test data
(R2test), which can be also defined
as Q2 in a metabolomics analysis, were 0.815 and
0.852, respectively (Figures 3A and 3E ). The coefficients of
determination in the cross-validation seemed to be sufficient in general
metabolome analyses.[16] However, the predicted
values were severely varied in the test data and the validation data. RF
showed similar R2train values to PLS,
and higher R2test values than PLS,
with lowered MSEtrain and MSEtest values
but large MSEval values (Figures 3B and 3F ).
This indicates that RF overfit the train data and test data, which
suggests a poor forecasting ability for the extrapolation data. The NN
fit the train data and test data similar to RF, and the
MSEval values were one order of magnitude smaller than
those of RF (Figures 3C and 3G). This means that the NN model can
forecast extrapolation data. DNN demonstrated very high coefficients of
determination and low MSE values using all data (Figures 3D and
3H ). In the case of multivalent outputs using time course data, the
data were excellently fitted to DNN (Figures 3J and 3L ), which
were preferred to those of RF (Figures 3I and 3K ).