Nicolas Saunier edited section_Experimental_Results_subsection_The__.tex  almost 9 years ago

Commit id: d06d28138472e9443abb7b4f0076cc99731ca247

deletions | additions      

       

\subsection{Correlation of Tracking Parameters with Tracking Accuracy}  Each unique MOTA from the various optimizations had associated parameters from which Spearman's rank correlation coefficient could be extracted. The parameters were individually compared to the MOTA and presented in Table~\ref{tab:spearman}. The winter camera for Site 1 had many results in which no objects were tracked and thus had a MOTA of 0. These sets of parameters were removed from the analysis to obtain a more representative correlation. The number of samples also varied based off of computation time since the higher resolution cameras took over 12 hours for the 20 samples of each generation of the genetic algorithm.  \begin{table}  \caption{Spearman's Rank Correlation Coefficient by Parameter and Site}  \label{tab:spearman}  \begin{tabular}{llllllllll}  \hline  \textbf{Site} & Number of Samples & \textbf{window-size} & \textbf{feature-quality} & \textbf{min-feature-distanceklt} & \textbf{mm-connection-distance} & \textbf{mm-segmentation-distance} & \textbf{min-tracking-error} & \textbf{min-nfeatures-group} & \textbf{min-feature-time}\\  \hline  S1S & 224 & -0.050 & -0.779 & -0.162 & 0.118 & -0.183 & 0.023 & -0.077 & 0.262\\  S1W & 210 & -0.101 & -0.879 & -0.189 & 0.092 & 0.049 & 0.257 & 0.195 & 0.359\\  S2 & 347 & 0.303 & -0.599 & 0.189 & 0.415 & 0.106 & 0.222 & 0.043 & 0.137\\  S3V1 & 115 & 0.639 & -0.577 & -0.011 & 0.451 & -0.186 & 0.114 & -0.296 & 0.003\\  S3V2 & 215 & -0.032 & 0.373 & 0.311 & -0.282 & -0.058 & 0.058 & -0.030 & 0.253\\  \end{tabular}  \end{table}    Performance of the low resolution IP-camera was largely dependent on the feature-quality. In fact, in the case of the winter camera, a MOTA above 0.150 was not found unless the feature-quality was below 0.10 and the best performance came from a feature-quality of approximately 0.01. This suggests that the winter camera should be calibrated separately from any other camera since the most important parameter is directly related to computation time. The range of the feature-quality should also be adjusted for such a calibration. Another point of mention is that the GoPro camera had no strong correlation towards any of the parameters. The implication is that the high resolution allows for a wide range of parameters to have similar performances. The third implication is that certain parameters such as min-feature-distanceklt and min-feature-time could potentially be eliminated from the optimization process due to the lack of correlation on any camera.  \subsection{Over-fitting Analysis}  The second phase of the validation process was to evaluate the performance of optimized parameters compared to the default parameters. The genetic algorithm was run for the first five minutes of each video file for between 10 and 30 generations over the course of one to three days depending on the site. Table (insert table about optimized parameters) is a summary of the optimal parameters found for these specific sequences.