Despite the extensive studies on the performance of video sensors and computer vision algorithms, calibration of these systems is usually done by trial and error using small datasets and incomplete metrics such as brute detection rates. There is a widespread lack of systematic calibration of tracking parameters in the literature. This study proposes an improvement in automatic traffic data collection through the optimization of tracking parameters using a genetic algorithm by comparing tracked road user trajectories to manually annotated ground truth data with Multiple Object Tracking Accuracy and Multiple Object Tracking Precision as primary measures of performance. The optimization procedure is first performed on training data and then validated by applying the resulting parameters on non-training data. A number of problematic tracking and visibility conditions are tested using five different camera views selected based on differences in weather conditions, camera resolution, camera angle, tracking distance, and camera site properties. The transferability of the optimized parameters is verified by evaluating the performance of the optimization across these data samples. Results indicate that there are significant improvements to be made in the parametrization. Winter weather conditions require a specialized and distinct set of parameters to reach an acceptable level of performance, while higher resolution cameras have a lower sensitivity to the optimization process and perform well with most sets of parameters. Average spot speeds are found to be insensitive to MOTA while traffic counts are strongly affected.