Typealike program turns hand gestures into commands

The prototype works through a laptop webcam with a mirror attached to it. According to its developers, the program recognises the user’s hands beside or near the keyboard and prompts operations based on different hand positions.

In use, a person could place their right hand with the thumb pointing up beside the keyboard, and the program would recognise this as a signal to increase volume. Different gestures and different combinations of gestures can be programmed to carry out a wide range of operations.

“It started with a simple idea about new ways to use a webcam,” said Nalin Chhibber, a recent master’s graduate from the University of Waterloo’s Cheriton School of Computer Science. “The webcam is pointed at your face, but the most interaction happening on a computer is around your hands. So we thought, what could we do if the webcam could pick up hand gestures?”

MORE FROM ELECTRONICS & COMMUNICATIONS

According to the university, this led to the development of a small mechanical attachment that redirects the webcam downwards towards the hands. The team then created a program capable of understanding hand gestures in variable conditions and for different users. The team used machine learning techniques to train the Typealike program.