A GUI-INTERACTION AIDING SYSTEM FOR CUT-AND-PASTE OPERATION BASED ON IMAGE-PROCESSING FOR THE VISUALLY-IMPAIRED

In this paper we propose a method to aid a visually impaired person in the operation of a computer running a graphical user interface (GUI). It is based on image processing techniques, using images taken by a c olor camera placed over a Braille display. The shape of the user's han d is extracted from the image by analyzing the hue and saturation hist ograms. The orientation of the hand, given by an angle a with the vert ical axis, is calculated based on central moments. The image of the ha nd is then rotated to a normalized position. The number of pixels in e ach column of the normalized image is counted, and the result is put i n a histogram. By analyzing the coefficient of asymmetry of this histo gram, it can be determined whether the thumb is positioned along the p ointing finger, or whether it is far from the other fingers. These two positions define two states that correspond to a mouse button up or d own. In this way, by rotating the hand and moving the thumb, we can em ulate the acts of moving a scroll bar and depressing a mouse button, r espectively. These operations can be used to perform tasks in a GUI, s uch as cut-and-paste, for example. Experimental results show that this method is fast and efficient for the proposed application.