A System for 3D Vehicle Tracking Using Multiple Cameras

Abstract

In this thesis, we present a 3D vehicle tracking system that allows 3D vehicle dimension estimation using fusion of information from multiple cameras. Compared to traditional 2D vehicle tracking systems with a single camera, the 3D tracking system can potentially improve tracking accuracy in the presence of similar noise, shadow or vehicle occlusion. Taking synchronized images from multiple cameras as inputs, the developed 3D vehicle tracking system first processes image frames using image processing techniques to derive vehicle silhouettes. Using a square roadside pattern derived from parallel traffic lanes, multiple cameras are calibrated using an extended vanishing points based approach and post-optimization by minimizing the projection error. After segmentation, objects in image frames are projected to the real world, in which the overlap of the same object would ideally correspond to the chassis of the object, and this allows estimation of vehicle length and width. Once vehicle length and width are obtained, they can be further used in each image frame to estimate the vehicle height. The base of the vehicle can be represented as an oriented rectangle. The height of the object is sought in the image view that would create the top of the hyper rectangle of the 3D vehicle that has the edge furthest away from the vehicle base rectangle. In the 3D vehicle tracking process, Kalman filter is used to predict the state of the vehicle in the real world to allow data association of the vehicles. Experiment results using realistic traffic videos of an intersection from two cameras have shown that the 3D vehicle tracking is fully functional. Vehicle dimension estimation results are mostly consistent in spite of some error caused by camera calibration and vehicle segmentation.