Methodology

2.1 ANOVA sensitivity analysis techniques

In order to use the same terminology to present each sensitivity technique, a generalized model is defined as:
Where represent the independent variable (such as model parameters, or model structure) and Y represents the response (such as the model performance). Variance-based methods use a variance ratio to estimate the importance of parameters. According to the ANOVA theory, the total sum of the squares (SST) can be divided into the sum of squares due to individual model parameters and their interactions as follows (Saltelli et al., 2008., Saltelli et al., 2010).
where represents the squares due to the individual effect of and to represent the squares due to interactions among k factors. In this model, we summarize all interaction terms into the term.
Then, for each effect, the variance fractions are derived as follows:
where:
The symbol “o” indicates averaging over the particular index. Values of 0 and 1 for the variance fraction correspond to a contribution of an effect to the total ensemble variance (uncertainty) of 0% and 100%, respectively. Obviously:

Subsampling

To diminish the effect of the sample size on contribution quantification in ANOVA, (Bosshard et al., 2013) proposed a subsampling scheme. Assume that there are elements for each parameter, the vector can be represented as. In each subsampling iteration, two elements are selected out of the total Ti elements which results in a total of (specify that C is the combination symbol) possible element pairs for. Therefore, for element, the is replaced by, which is a matrix as Formula (9). Here h means the row number and j means the column number. The total number of columns is defined as J. Therefore, h=1 or 2 and j=1, 2, 3,……,J. For more details of subsampling scheme, please refer to the literature (Bosshard et al., 2013).

Single-subsampling ANOVA

Single-subsampling ANOVA means that only one parameter from the parameter vector ( ) is subsampled before the ANOVA. Assuming that the is subsampled, which mean the two elements selected from vector are used for . As for all other parameter, there are still elements for each. We estimate the terms in equations (2) and (3) using the subsampling procedure introduced in section 2.2 as follows.
For :
For :
The symbol indicates averaging over the particular index. Then, for each effect, the variance fraction is derived as follows:

Multiple-Subsampling ANOVA

The single-subsampling ANOVA may lead to biased results if different parameters are chosen for subsampling. As an extension of the single-subsampling ANOVA, a multiple-subsampling scheme is introduced to ANOVA, leading to a multiple-subsampling ANOVA approach. Multiple-subsampling ANOVA means that more than one parameter from the parameter vector () are going to be subsampled at the same time before the ANOVA is calculated. Assume that are subsampled then are replaced by respectively. We estimate the terms in equations (2) and (3) using the subsampling procedure as follows:
For :
For :
Then, for each effect, the variance fraction is derived as follows:

Full-Subsampling ANOVA

Moreover, a full-subsampling approach can be formulated when all parameters are going to be subsampled. In detail, the full-subsampling ANOVA means that all parameters are subsampled before ANOVA is calculated. Consequently, are replaced by respectively. We estimate the terms in equations (2) and (3) using the subsampling procedure as follows:
Then, for each effect, the variance fraction is derived as follows:
To evaluate the performance of the above different subsampling ANOVA approaches, two test cases are applied in the following.