Current upper limb prosthetics that can grip are controlled by myoelectric signals from the muscles in the stump, but it’s a skill that takes patience and time to master. Funded by the EPSRC, the Newcastle team created a computer vision system that enables prosthetics to ‘see’ with the assistance of an off-the-shelf camera. The work appears in the Journal of Neural Engineering.
“Responsiveness has been one of the main barriers to artificial limbs,” said Dr Kianoush Nazarpour, senior lecturer in Biomedical Engineering at Newcastle University. “For many amputees the reference point is their healthy arm or leg, so prosthetics seem slow and cumbersome in comparison.”
“Using computer vision, we have developed a bionic hand which can respond automatically - in fact, just like a real hand, the user can reach out and pick up a cup or a biscuit with nothing more than a quick glance in the right direction.”
The researchers trained the system using neural networks, showing it numerous pictures of various objects from multiple angles and in different light conditions. Over time, the AI learned which grasp pattern to use for different objects according to their shape, but without measuring specific dimensions or explicitly identifying them. Objects were categorised into four grasp classes: pinch, tripod, palmar wrist neutral and palmar wrist pronated.
“The computer isn’t just matching an image, it’s learning to recognise objects and group them according to the grasp type the hand has to perform to successfully pick it up,” said lead author Ghazal Ghazaei, who carried out the work as part of her PhD at Newcastle’s School of Electrical and Electronic Engineering.
“It is this which enables it to accurately assess and pick up an object which it has never seen before – a huge step forward in the development of bionic limbs.”
The research is part of a wider prosthetics project led by Newcastle, which also involves the universities of Leeds, Essex, Keele, Southampton and Imperial College London. Longer term, the aim is to develop a bionic arm which connects directly to the nerve networks in the amputated limb, and which could be controlled directly by the user’s brain. According to Dr Nazarpour, the camera-assisted prosthetic is an interim solution that can help pave the way.
“It’s a stepping stone towards our ultimate goal,” he said. “But importantly, it’s cheap and it can be implemented soon because it doesn’t require new prosthetics – we can just adapt the ones we have.”
Nanogenerator consumes CO2 to generate electricity
Whoopee, they've solved how to keep a light on but not a lot else.