Visual rhythm and shot verification

Typical result of an automatic shot change detection algorithm expectedly i ncludes a certain number of undetected shots as well as falsely detected sh ots. Even though automatic shot change detection algorithms are continuing to improve, the ultimate goal of automatically detecting all shot changes w ithout false alarms may never be achieved. Thus, allowing a human operator to intervene-to review and verify the result of a shot change detection alg orithm, to delete falsely detected shots as well as to find undetected shot s-may be the most viable approach currently available for increasing the ac curacy of the overall shot detection process. For this exact purpose, we pr opose a shot verifier based on the visual rhythm. The visual rhythm, an abstraction of the video, is a single image, a sub-sa mpled version of a full video in which the sampling is performed in a pre-d etermined and in a systematic fashion. It is a representation of the video, which includes the overall content of the video. But most importantly, the visual rhythm contains patterns or visual features that allow the viewer/o perator to distinguish and classify many different types of video effects ( edits and otherwise): cuts, wipes, dissolves, fades, camera motions, object motions, flashlights, zooms, etc. The different video effects manifest the mselves as different patterns on the visual rhythm. Using the visual rhythm , it becomes possible, without sequentially playing the entire video, to fi nd false positive shots as well as undetected shots. Thus, inclusion of the visual rhythm in the shot verification process will aid the operator to ve rify detected shots as well as to find undetected shots fast and efficientl y. Our newly developed shot verifier based on the visual rhythm has been desig ned for operator efficiency. The design of our shot verifier presented and the usefulness of the visual rhythm during the shot verification process wi ll be demonstrated.