Sensitive display enhances photorealism

A ‘lighting sensitive’ computer display senses the illumination of it’s surroundings and then adjusts the appearance of the content to be displayed.

A team of researchers led by a Columbia computer scientist has developed a lighting sensitive computer display that achieves a high degree of photo-realism by sensing the illumination of the display’s surroundings and accordingly adjusting the appearance of the content to be displayed.

The new technology, called Lighting Sensitive Displays, or LSD, overcomes a major drawback among the current generation of computer displays: their inability to respond to a wide spectrum of lighting conditions.

‘Whether electronic billboards or laptop monitors, today’s displays produce ‘flat’ images that have no relation to the surroundings of the display,’ says Shree K. Nayar, professor of computer science and director of the Computer Vision Laboratory at Columbia’s Fu Foundation School of Engineering and Applied Science.

Nayar developed the LSD technology with two other computer scientists, Peter Belhumeur of Yale University and Terry Boult of Lehigh University. The new technology enables, for instance, a digital image of a museum sculpture on a computer display to have the colours, shadings, highlights, and shadows needed to make the sculpture look like it was lit by the surroundings of the display.

‘The light field around a display or, for that matter, any object, is complex and varies dramatically from one environment to another. To achieve a high degree of realism, you need to ensure that the displayed objects appear to be ‘lit’ by the illumination around the display,’ said Nayar.

Lighting Sensitive Displays use sensing devices that estimate in real time what Nayar, Belhumeur, and Boult describe as the ‘illumination field’ – the intensities and directions of light hitting the display.

The sensing technologies are embedded within the display device and can come in different forms – an arrangement of compact photo-sensitive detectors over and around the display device, optical fibres distributed around the device, or wide-angle camera systems placed close to the device. The measured illumination field can be used in several ways to modify the displayed content, depending on whether the content is a two-dimensional image, a three-dimensional scene, or a combination of both.

‘The idea is to explicitly use the measured light rays hitting your display to render the content you want to display, whether it’s a piece of art or a commercial product, such that it looks ‘real’ in your personal space,’ says Nayar. ‘You can embed the illumination sensors and the chips that perform the rendering computations in the display itself. You don’t need a computer to make a display lighting sensitive.’

The development of the technology was supported by the Information Technology Research initiative of the National Science Foundation.

The invention has implications for e-commerce. For example, a potential furniture buyer can view fabric or wood choices on a laptop placed in the physical space for which the furniture is intended, giving the buyer a more accurate representation of what the product would look like in the lighting of the space.

The invention was also motivated by the emergence of high quality digital displays as possible replacements for paintings, prints, and sculptures in homes of the future.

‘Displays give you the flexibility to look at a Picasso on one day and a Brancusi on another. The LSD technology puts us one step closer to making the displayed artwork look real whether it is in your sunlit living room or your halogen lit bedroom.’

Nayar’s research program at Columbia has previously developed several advanced digital imaging technologies, including, 360-degree immersive imaging techniques that allows internet or television viewers to see images in all directions at once, and high dynamic range imaging methods that allows cameras to capture a much broader range of light and colour variations.