VR operated robots could cut out the commute for factory workers

The group, from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) used an Oculus Rift headset linked to a Rethink Robotics Baxter robot to place the user in a VR control room with multiple sensor displays, making it feel like they’re inside the robot’s head.

Using Oculus’ controllers, users can interact with controls that appear in the virtual space to open and close the hand grippers to pick up, move, and retrieve items. A user can plan movements based on the distance between the arm’s location marker and their hand while looking at the live display of the arm.

To make these movements possible, the human’s space is mapped into the virtual space, and the virtual space is then mapped into the robot space to provide a sense of co-location.

The system is also more flexible compared to previous systems that require many resources.

Other systems might extract 2-D information from each camera, build out a full 3-D model of the environment, and then process and redisplay the data.

In contrast, the CSAIL team’s approach bypasses all of that by simply taking the 2-D images that are displayed to each eye. (The human brain does the rest by automatically inferring the 3-D information.)

To test the system, the team first teleoperated Baxter to do simple tasks like picking up screws or stapling wires. They then had the test users teleoperate the robot to pick up and stack blocks. The team also showed that the system could pilot the robot from hundreds of miles away, testing it on a hotel’s wireless network in Washington, DC to control Baxter at MIT.

“A system like this could eventually help humans supervise robots from a distance,” said CSAIL postdoctoral associate Jeffrey Lipton, who was lead author on a related paper about the system. “By teleoperating robots from home, blue-collar workers would be able to tele-commute and benefit from the IT revolution just as white-collar workers do now."

The researchers even imagine that such a system could help employ increasing numbers of jobless video-gamers by “game-ifying” manufacturing positions.