TOWARD AUTOMATIC ROBOT INSTRUCTION FROM PERCEPTION - TEMPORAL SEGMENTATION OF TASKS FROM HUMAN HAND MOTION

Our approach to program a robot is by direct human demonstration of th e grasping task in front of the system. The system analyzes the stream of perceptual data measured during the human execution of the task an d then produces commands to the robot system to replicate the observed task. In order to analyze the stream of perceptual data, it is easier to first segment this stream into meaningful and separate units for i ndividual analysis. This paper describes work on the temporal segmenta tion of grasping task sequences based on human hand motion. The segmen tation process results in the identification of motion breakpoints sep arating the different constituent phases of the grasping task. A grasp ing task is composed of three basic phases: pregrasp phase, static gra sp phase, and manipulation phase. We show that by analyzing the finger tip polygon area (which is an indication of the hand preshape) and the speed of hand movement (which is an indication of the hand transporta tion), we can divide a task into meaningful action segments such as ap proach object (which corresponds to the pregrasp phase), grasp object, manipulate object, place object, and depart (a special case of the pr egrasp phase which signals the termination of the task). We introduce a measure called the volume sweep rare, which is the product of the fi ngertip polygon area and the hand speed. The profile of this measure i s also used in the determination of the task breakpoints. The temporal task segmentation process is important as it serves as a preprocessin g step to the characterization of the task phases. Once the breakpoint s have been identified, steps to recognize the grasp and extract the o bject motion can then be carried out.