A researcher at Penn State University has created a new system that automatically sorts, classifies and retrieves digital images based on the way people look at and understand pictures. The system is said to offer the promise not only faster, more accurate image database searches but also better Web searches.
The new approach, created by Dr. James Z. Wang, Penn State assistant professor of information sciences and technology, does not consider any information other than the image itself.
Image retrieval techniques currently in commercial use mostly rely on keywords or descriptions. While this text-based approach can be accurate and efficient for limited databases of high value it can become prohibitively expensive to manually input descriptions of large-scale image databases.
The new approach not only reduces the need for textual information but also can handle, quickly and efficiently, the one billion images that can be found on the Internet.
Wang and colleagues have built an experimental image retrieval system, called SIMPLIcity, to validate and demonstrate their methods and have tested it on a database of about 200,000 general-purpose images and an archive of more than 70,000 pathology images.
Wang notes that the capability of existing CBIR systems is essentially limited by the fact that they rely on only primitive features of the image.
In his new approach, Wang matches the image features selected to classify the image to the type of picture. For example, a colour layout indexing method may be best for outdoor pictures while a region-based indexing approach may be better for indoor pictures.
A biomedical image database can be categorised into X-ray, MRI, pathology, graphs, micro-arrays and other features specific to the types of images in the collection.
For general-purpose image libraries and the Web, Wang has classified images into textured vs. non-textured, graph vs. photograph.
His approach is said to represent the first time that categories, such as textured vs. non-textured, have been used as a distinguishing feature in image retrieval.
In addition, besides using new image features as classification tools, SIMPLIcity uses a similarity measure based on information about the entire image rather than representative segments.
In traditional approaches, computer programs may segment one image of a dog into two regions: the dog and the background. The same program may segment another image of a dog into six regions: the dog’s body, the dog’s front legs, the dog’s rear legs, the dog’s eyes, the background and the sky. The inconsistent segmentation makes it harder to make a match.
In SIMPLIcity, an overall ‘soft similarity’ approach is reported to reduce the influence of inaccurate segmentation. The most similar region pairs are matched first and then the matching process is ‘softened’ by allowing one region of an image to be matched to several regions of another image. In this way all of the regions of the images are taken into consideration.