Computer vision to aid surgery

Medical procedures could be rendered safer and faster using computers with a human-like ability to see, say researchers at UCLA’s Henry Samueli School of Engineering and Applied Science.

Brain surgery, endoscopies and other medical procedures could be rendered safer and faster using computers with a human-like ability to see, say researchers at UCLA’s Henry Samueli School of Engineering and Applied Science.

Stefano Soatto, assistant professor at UCLA’s computer science department and head of the Vision Lab, is reportedly drawing inspiration from the human visual system in order to pass the ability on to machines. ‘In practice, the human visual system is still by far the best around,’ said Soatto, ‘but this may not be so for long.’

Soatto’s research team is examining how people use vision to interact with others and with their surroundings, and is designing systems that will allow computers to perform in similar ways. ‘We use senses to build models of the world around us that allow us to walk through our environment and interact with it safely,’ said Soatto. ‘I want a machine to be able to do the same thing.’

Their work could have an impact on the growing field of image-guided surgery, where doctors use imaging technology to help them perform surgery.

Surgeons already use technology that merges multiple images to create a 3D map of a patient’s brain. But these images are often up to a day old, and if the procedure itself alters conditions, the images become unusable.

Though a handful of magnetic resonance imaging machines (MRIs) worldwide do provide updated images during surgery, they too have limitations, chief among them that humans must still grip or manipulate the surgical tools.

Soatto argues that a computer that can understand and act within its environment can not only recreate and constantly update a three-dimensional model of the brain, it can use what it ‘sees’ to perform tasks otherwise done by surgeons.

Instead of a person interpreting visual information from a computer to manually guide a catheter through the body or to gather a tissue sample, the computer would use its own imaging data to perform the task itself.

With the demand for surgeons fast outgrowing the time that experts have to train them in certain procedures, the benefits of having such a computer system could be significant. It would eliminate the need for surgeons to travel from afar to perform operations by placing a ‘virtual surgeon’ in the operating room.

The key, said Soatto, is in giving a computer ‘dynamic vision,’ the ability to study images as they change over time and use the information to perform assigned tasks.

‘The world has certain physical properties – shape, motion, material properties of objects and so forth,’ said Soatto. ‘Humans have developed, over the course of evolution, a particular way of representing their environment that has been crucial for humans to survive – detecting prey, recognising familiar objects, for example.’

Machines, especially computers, can also be made to interpret the physical world and interact with it, whether that environment is a nuclear reactor or the human body.

To interact with a changing environment, a computer needs to gain information about certain spatial properties – shape, motion, distances, angles – measurable properties it obtains by taking many photographs from multiple points of view, where either the scene or the viewer’s perspective changes from one time to the next. In this way a three-dimensional representation of the world can be created.

In July 2000, Soatto’s team was the first to demonstrate a computer system that could track an object’s movement and shape in real time.

By capturing and processing images in real time, a machine can act immediately based on what it sees, rather than later, after the visual data has been analysed by humans. It also means a computer could do more than just pre-assigned tasks based on data collected earlier. It could constantly update what it knows about its environment and truly interact with a changing world.

On the web