Learning patterns of activity using real-time tracking

Our goal is to develop a visual monitoring system that passively observes m oving objects in a site and learns patterns of activity from those observat ions. For extended sites, the system will require multiple cameras. Thus, k ey elements of the system are motion tracking, camera coordination, activit y classification, and event detection. In this paper, we focus on motion tr acking and show how one can use observed motion to learn patterns of activi ty in a site. Motion segmentation is based on an adaptive background subtra ction method that models each pixel as a mixture of Gaussians and uses an o n-line approximation to update the model. The Gaussian distributions are th en evaluated to determine which are most likely to result from a background process. This yields a stable. real-time outdoor tracker that reliably dea ls with lighting changes, repetitive motions from clutter, and long-term sc ene changes. While a tracking system is unaware of the identity of any obje ct it tracks, the identity remains the same for the entire tracking sequenc e. Our system leverages this information by accumulating joint co-occurrenc es of the representations within a sequence. These joint cooccurrence stati stics are then used to create a hierarchical binary-tree classification of the representations. This method is useful for classifying sequences, as we ll as individual instances of activities in a site.