Machine vision system helps robots get picky

Known as Dense Object Nets (DON), the system uses a camera to create a visual roadmap of an object as a collection of points. These coordinates can then be referred to by the system from any angle, allowing it to identify specific objects and helping robots grab them in specific ways. Unlike other machine vision systems, DON is able to carry out tasks on objects without having seen them before or being trained on the task. The MIT team believes the technology could be applied in warehouses by logistics companies or online retailers such as Amazon.

"Many approaches to manipulation can't identify specific parts of an object across the many orientations that object may encounter," said PhD student Lucas Manuelli, who wrote a new paper about the system with lead author and fellow PhD student Pete Florence, alongside MIT professor Russ Tedrake. "For example, existing algorithms would be unable to grasp a mug by its handle, especially if the mug could be in multiple orientations, like upright, or on its side."