Simultaneous Association and Localization for Multi-Camera Multi-Target Tracking

Abstract

In this dissertation, we propose two approaches for three-dimensional (3D) localizing and tracking of multiple targets by using images from multiple cameras with overlapping views. The main challenge is to solve the 3D position estimation problem and the trajectory assignment problem simultaneously. However, most of the existing methods solve these problems independently. Unlike single camera multi-target tracking, it is much more complicated to solve both problems because the relationship between cameras is also taken into consideration in multi-camera. To tackle this challenge, we present two approaches: mixed multidimensional assignment approach and variational inference approach. In the mixed multidimensional assignment approach, we formulate the data association and 3D trajectory estimation problem as the mixed optimization problem with discrete and continuous variables and suggest an alternative optimization scheme which jointly solves the two coupled problems. To handle a large solution space, we develop an efficient optimization scheme that alternates between two coupled problems with a reasonable computational load. In this optimization formulation, we design a new cost function that describes 3D physical properties of each target. In the variational inference approach, we establish a maximum a posteriori (MAP) problem over trajectory assignments and 3D positions for given detections from multiple cameras. To find a solution, we develop an expectation-maximization scheme, where the probability distributions are designed by following the Boltzmann distribution of seven terms induced from multi-camera tracking settings.