Nicolas Saunier edited Results.tex  about 9 years ago

Commit id: c52f05d72282f5e6414a6dfda0d7efb563e7f25a

deletions | additions      

       

\subsection{Tracking Performance}  \label{tracking_calibration}  Tracking accuracy was evaluated by way of the CLEAR MOT metric Measures Of Tracking Accuracy (MOTA) \cite{Bernardin_2008} using as ground truth manually annotated trajectories with the annotation tool \href{http://www.jpjodoin.com/urbantracker/index.html}{Urban Tracker} \cite{Jodoin_2014}. Perfect tracking would yield MOTA of 100\%, while it can become negative if more false alarms are done than there are ground truth objects. Nearly 40,000 observations (instants) of 371 motor vehicles were annotated manually across two prototypical sample sites: one site where objects were tracked at a distance of between 20 and 50 meters away from the camera and one site where objects were tracked at a distance no greater than 20 meters away. An initial MOTA was calculated for default tracking parameters and offered modest accuracy performance  (in the neighbourhood of $70\%$). A tracking optimization was then performed using a genetic algorithm similarly to  \cite{ettehadieh15systematic} to search for parameters that maximized MOTA. After optimization, accuracy increased to $94\%$. $94\%$ (measured over the same ground truth used for optimization).  These statistics are summarized in Table~\ref{tab:track_calib_data}. Tracking parameter optimization converged in less than 24 hours. Of important note is the fact that this calibration was performed on the oldest and poorest quality video data (resolution of $800\times 600$, without software-assisted image stabilization, and before lens correction). Tracking results are expected to increase with increases in quality of video data and video pre-treatment. The optimized tracking parameters should be portable to other sites with similar view and camera characteristics, though this will be investigated further and be the subject of a future paper in depth.