Picture search

1 min read

A pair of Penn State researchers has developed a new system that one day could make it easier to search for photographs on the internet.

The system works by teaching computers to recognise the contents of photographs, such as buildings, people, or landscapes, rather than by searching for keywords in the surrounding text, as is done with most current image-retrieval systems.

They hope that one day, their so-called Automatic Linguistic Indexing of Pictures in Real-Time (ALIPR) system could be used in industry for automatic tagging or as part of internet search engines.

'Our basic approach is to take a large number of photos - we started with 60,000 photos - and to manually tag them with a variety of keywords that describe their contents,' said Jia Li, an associate professor of statistics at Penn State.

'We then teach the computer to recognise patterns in colour and texture among these photos and to assign our keywords to new photos that seem to contain similar content. Eventually, we hope to reverse the process so that a person can use the keywords to search the web for relevant images.'

Li said that most current image-retrieval systems search for keywords in the text associated with the photo or in the name that was given to the photo. This technique, however, often misses appropriate photos and retrieves inappropriate photos. Li's new technique allows her to train computers to recognise the semantics of images based on pixel information alone.

Li, who developed ALIPR with her colleague James Wang, a Penn State associate professor of information sciences and technology, said that their approach appropriately assigns to photos at least one keyword among seven possible keywords about 90 per cent of the time.

Although the team's goal is to improve the accuracy of the technique, Li said she does not believe the approach will ever be 100 per cent accurate.

'There are so many images out there and so many variations on image content that I don't think it will ever be possible for ALIPR to be 100 per cent accurate,' she said.