Video game- and eye-inspired 3D camera can work outdoors

A device developed for video gaming has proved top be the key to a new, inexpensive type of cameras which can produce 3D images. The Micorsoft Kinect system, which uses optics to capture the movement and gestures of players to control the actions of their in-game avatars, provided the inspiration for Oliver Cossairt of Northwestern University in Illinois to develop a new imaging system that overcomes some of the drawbacks of the Kinect system, such as being able to work outdoors and producing higher-quality images.

Kinect works by projecting light patterns onto the player and detecting their reflections, with internal software estimating how far from the unit the reflection took place and using that information to build up a 3D image in its processor. This is fast, but much less precise than, for example, industrial laser scanners, which collect data by scanning across a whole scene and collecting data on single-point reflections.

Cossairt’s device, which he developed with graduate student Nathan Matsuda and Mohit Gupta of Columbia University in New York City with sponsorship from the US Department of Energy and Office of Naval Research, is called Motion Contrast 3D. It works more like the human eye; rather than scanning the whole scene all the time, it looks specifically for the part of the scene that changes between subsequent scans. “If you send the same signal to your eye over and over, the neurons will actually stop firing,” Cossairt said. “The neurons only fire if there is a change in your visual stimulus. We realized this principle could be really useful for a 3-D scanning system.” This makes the system faster and also more accurate than Kinect.

Motion Contrast 3D also uses a laser for scanning that is much brighter than ambient sunlight, which allows it to be used in outdoors. Kinect uses relatively low-intensity light projection that is overwhelmed by sunlight, and can only be used indoors in relatively dim lighting. “In order for a 3-D camera to be useful, it has to be something you can use in everyday, normal environments,” Cossairt said. ‘Outdoors is a part of that, and that’s something the Kinect cannot do, but Motion Contrast 3-D can.”

In a paper presented at the IEEE International Conference on Computational Photography last week, the team explained that the system has potential for applications in capturing 3D images for robotics, bioinformatics, augmented reality, and manufacturing automation. The team has received a grant from Google to integrate it in an experimental autonomous vehicle, where it will form part of the navigation system; a role it could also fulfil on motorized wheelchairs.