We are exploring the use of hand gestures when it comes to computer interaction. Rather than using a mouse, you will define what is called your own Personal Space in order to accurately control and manipulate the mouse cursor on the screen. The hope here is that gesture manipulation will replace using traditional keyboard and mouse therefore alleviating carpel tunnel syndrome and other ergonomic issues.
We start with a calibration phase, where the user defines the 4 edges of their personal space, and the system draws a straight line between them to build a Personal Space.
A Transformation Matrix is built to map the rectangular shape of the flat plane used as input to the display device. The image below shows how we skew the display to fit within the user’s Personal Space. In this image we virtually project the screen onto the user’s calibrated space to illustrate the process.
We can therefore imagine the display device being stretched and skewed to fit the users Personal Space.
We will also be exploring using gestures for keyboard input
We have so far been able to prove usability of gestural pointing (ie: navigation, or cursor manipulation) with Personal Space. In the future we intend to incorporate some gestural selection (click) and gestural manipulation (drag-and-drop). The following shows how Personal Space can be used on the desktop: