During the design process, several tools are used to iterate and perform mathematical operations to optimise the design. To proof that these tools deliver the correct results they have to be verified and validated. For this, the following definitions are utilised \cite{VandV_definition}. Verification is to determine if a simulation model accurately represents the chosen physical model. Validation is to determine if the simulation results accurately represent the physical problem. The following tools are used to perform important calculations or aid in crucial design steps.
Excel tool to analyse the weight distribution and the resulting centre of gravity
Excel tool to analyse the stability and controlability of the design configurations
Python tool to perform the trade off of subsystems and the configurations
Python tool to simulate the track time around a race track
The first three tools presented are already used at this stage of the project. Several approaches are planned for the track analysis that is to be designed.
Unit tests These tests are part of the calculation verification. Whenever a tool can be split up into different units that have an individual input and output, it is useful to analyse for the particular part of the tool whether it delivers the correct intermediate results. This can be done for the excel tools and the python tools. In the case of the first excel tool, the total weight as well as the centre of gravity location are calculated by hand and by the tool and the results are compared. For the second tool, the formulas presented in \ref{cha:stability} are calculated by hand using simple, rounded input values. Again, the results are compared. Both tools work correctly. The same approach is taken for the first python tool. Simple inputs and pairwise comparisons are performed by the tool and by hand. The resulting matrices are investigated and the eigenvectors are calculated. The comparison shows that the result of the tool matches the analytically found result. Also, the puts and inputs of different units are compared to see that the program transfers the correct intermediate results to the next units, verifying the correct working of the entire system. All unit tests are performed for several inputs to verify that the tool works for different cases.
The same approach will be taken for the track simulation. Separate units that determine intermediate results mathematically will be tested against analytical results.
Code verification Python has a built in error detection. This assists the engineers in correcting faulty coding already before the tool is finished. Errors that were detected could immediately be corrected for.
This will also happen when the track simulation is performed.
Validation is performed by comparing the results of the tools to real life behaviour of a representative test case or another validated tool to make sure that the tool not only delivers mathematically correct values but also serves to solve the problem at hand. The excel tools cannot be validated since no representative data is provided. It could be validated as soon as the plane under consideration is built.
The python tools can be validated. The first one, the trade-off tool, is used with inputs for which the outputs are known. Thus, the output matrices and vectors are validated. The test cases are taken from \cite{AHP_example}. No discrepancies were found and the tool was deemed to work correctly. Therefore, it is validated.
It will be attempted to validate the track simulation by considering a real life track, for which the track characteristics such as distances, speed limits and track requirements are known. By investigating the times real pilots achieve when racing on the track it will be judged whether the tool delivers realistic ideal times. For example, when the tools ideal times are slower than real life times around the track the tool does not deliver correct results and does not solve the problem under consideration. The tool should yield ideal times around the track that are slightly faster than the ones achieved by pilots, since human errors as well as non-ideal conditions are not accounted for in the tool.