Monitoring human behavior from video taken in an office environment

In this paper, we describe a system which automatically recognizes human ac tions from video sequences taken of a room. These actions include entering a room, using a computer terminal, opening a cabinet, picking up a phone, e tc. Our system recognizes these actions by using prior knowledge about the layout of the room. In our system, action recognition is modeled by a state machine, which consists of 'states' and 'transitions' between states. The transitions from different states can be made based on a position of a pers on, scene change detection, or an object being tracked. In addition to gene rating textual description of recognized actions, the system is able to gen erate a set of key frames from video sequences, which is essentially conten t-based video compression. The system has been tested on several video sequ ences and has performed well. A representative set of results is presented in this paper. The ideas presented in this system are applicable to automat ed security. (C) 2001 Elsevier Science B.V. All rights reserved.