Optical Traffic surveillance

Goal: Advanced Analysis of traffic data for monitoring, traffic control and emergency handling on the basis of optical camera images.

In traffic scenarios, where a number of cameras from different view angles is available, object tracking and 3D reconstruction can be carried out. In this context, we developed such an application that initially requires a fully calibrated camera system. Camera images are transmitted from single servers for each camera to a central client, using image compression algorithms, e.g. MPEG-4 or H.264. The camera sequences from all cameras from one scene are than used to provide information for traffic control and additionally provide a 3D scene reconstruction. The algorithm starts with a segmentation of moving scene objects from a static background. The segmentation is based on a Kalman filter formalism that is able to adapt to environmental changes, especially global illumination changes due to varying weather conditions.

After segmentation, a 3D scene model from the background images is constructed first by manually selecting the appropriate ground area in each view. In the traffic environments this includes the street areas of the image. Additional side areas can be selected further to model adjacent areas or buildings as perpendicular planes w.r.t. the ground plane. The following example in Fig. 1 shows the modeled background image. Interpolation between the views was achieved by assigning different plane vectors to each view and is than handled automatically by today's graphics hardware. By clicking on the image, the appropriate sequence should be opened in a suitable player.

Fig. 1:  Background model of the traffic scene

Moving objects are first tracked within each 2D view separately. During visibility an object ID or label is assigned to that object. Afterwards the objects are merged in the 3D fusion stage. By projecting the center of gravity of a 2D object contour into the other view, the associated object in all other views is selected. To provide a certain degree of error robustness due to possibly incorrect segmentation results, the nearest object's center of gravity is chosen. Fig. 2 shows the label assignment after fusion, where the same label number is associated with the same object in both views.

Fig. 2: Label assignment during the tracking process

In the final dynamic object reconstruction synthetic 3D models are selected from a database in a best-match approach. The model is properly positioned according to the 3D center of gravity, which was obtained by projecting all 2D centers of gravity into the scene background using projection matrices. Additionally the synthetic model is aligned in parallel to the motion trajectory and scaled to fit the original 2D object texture. Finally object textures are mapped onto the model and the 3D model is integrated into the scene. For further visual scene enhancement, trajectories of moving objects can be interpolated for each render cycle instead of each camera frame rate. The example in Fig. 3 shows the non-interpolated version with jerky object motion on the left and interpolated version on the right.

Fig. 3: Traffic scene with 3D object non-interpolated positions, i.e. temporal sparse positions according to camera frame rate left and 3D traffic scene with interpolated motion trajectories for moving objects, right.

Finally a 3D scene is reconstructed that allows free user navigation to better visualize traffic situations and provide guidance by setting up a virtual camera path or flight through the 3D scene.

Publications in Journals

2005
• Karsten Müller, Aljoscha Smolic, Michael Droese, Patrick Voigt, and Thomas Wiegand:
  Reconstruction of a Dynamic Environment with Fully Calibrated Background for Traffic Scenes,
  IEEE Transactions on Circuits and Systems for Video Technology, Vol. 15, No. 4, pp. 538-549, April 2005.
  
  

2003
• Karsten Müller, Aljoscha Smolic, Michael Droese, Patrick Voigt, and Thomas Wiegand:
  Multi-Texture Modelling of 3D Traffic Scenes,
  IEEE International Conference on Multimedia and Expo (ICME'03), Baltimore, MD, USA, July 2003.
  
  
• Aljoscha Smolic, Karsten Müller, Michael Droese, Patrick Voigt, and Thomas Wiegand:
  Multiple View Video Streaming and 3D-Scene Reconstruction for Traffic Surveillance,
  International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS'03), London, UK, April 2003.
  
  

2003
• Thomas Wiegand:
  TraVis 3D – Multiview Video for Traffic Surveillance,
  Internationale Funkausstellung (IFA 2003), Berlin, Germany, September 2003.