The ubiquitous mechanical computer mouse, Douglas Engelbart’s invention of the 1960’s, may be an endangered species. While it revolutionised and simplified the way we interact with computers, many users find its reliability problems and need for frequent cleaning aggravating. Because mechanical mice rely on the friction of a trackball rolling across a desktop, they can pick up grime and begin to skid.
One of the first non-mechanical mice, introduced in the 1980’s partially to address these issues, sensed printed grid lines on a special mouse pad and had only limited acceptance. Until the development of the latest optical mouse, the only significant advances in pointing devices were the mouse trackpad, trackballs, and tablets.
Thanks to advances in optical navigation pioneered by Agilent Laboratories, solid-state optical mice are now widely available from all the leading mouse manufacturers. This new type of computer mouse never needs cleaning, tracks precisely, works on almost any surface, and may even reduce repetitive stress injury. And, most importantly, users love the ‘feel’ so strongly that they say they would never go back to a mechanical mouse.
Silicon-based optical navigation has led to a better computer mouse and to a substantial new business for Agilent Technologies. All of the world’s leading manufacturers of computer mice, more than 50, are using the Agilent-Labs-developed optical sensor. First introduced in 1999, mice with Agilent chips made up more than 7.5 percent of the retail market that year. In 2000, this figure jumped to 33 percent.
And that just hints at the potential of the technology. In the future, optical navigation promises to change not just the type of mice people use with their computers, but the way in which they interact with a variety of information and entertainment appliances including cell phones, entertainment centres, video games, and digital cameras.
The optical mouse owes its existence to recent advances in application-specific integrated circuits (ASICs), imaging arrays, and embedded mathematics; technologies that have come together to lower the cost and increase the practicality of optical navigation. Optical navigation involves ultra-fast image capture and a suite of mathematical image-processing manipulations, including prediction, correlation, and interpolation. To make optical navigation practical for mice and other consumer devices, it must be implemented inexpensively. Generally this means that all these navigation functions need to be embedded in a single silicon integrated circuit.
For the optical mouse, optical navigation involves capturing an image and then analysing and tracking the motion of microscopic texture or other features on a surface. Because optical mice depend on tracking surface detail, a key discovery was that most work surfaces are microscopically textured. When these surface textures are illuminated (e.g., by a light-emitting diode), a pattern of highlights and shadows is revealed (see illustration). Optical mice ‘watch’ these surface details move by imaging them onto navigation integrated circuits (IC).
For the mouse, the Agilent optical-navigation IC captures images at the rate of 1,500 pictures per second, using a small 16-by-16-pixel image sensor. As each image is captured, it is transferred to the processing and computation section of the same IC, where the movement of the mouse is computed by comparing successive images.
The Future of Optical Navigation
While optical navigation dramatically improves the performance and reliability of computer mice, it can also be utilised in many new ways. Designers are taking advantage of the optical sensor’s small size to create new sizes and uses for cursor-control devices. A very small mouse could be stowed in the side of a laptop computer or a new more user-friendly shape could greatly reduce mouse-induced repetitive stress injury.
The uses for optical navigation also stretch well beyond desktop mice. The technology will find a host of other cursor-control applications in devices such cell phones, digital cameras, palm and laptop computers, ergonomic mice, and fingerprint-identification scanners. More and more, the increasing complexity and decreasing size of these appliances is forcing designers to rely on screen menu systems, which will require some attendant mechanism for pointing. For such small appliances, the cursor controller is likely to be built into the device. One solution is an ‘optical button’ that can identify skin whorls, thereby detecting the motion of a finger across it. A ‘button mouse’ — inserted near the ‘J’ key on a laptop, or placed anywhere on a camera, cell phone, or personal organiser– could be used to navigate menus and programmed to recognise text and numbers spelled out with a finger.
Optical navigation can also be applied to many types of computer and entertainment-centre controllers. Entertainment centres will increasingly leverage the synergy between television, the web, and computers. Imagine an entertainment system that learns a viewer’s preferences, searches online TV guides for candidate programs, speculatively records these programs on hard disk, and then suggests the programs for later viewing. Such a system would rely heavily on screen menus and would need the family-room equivalent of a mouse. By changing the focal length of the imaging lens to infinity, a navigation chip will be able to track hand motions made by the user holding the device and apply it to the cursor. This type of ‘flying mouse’ or controller enables users to control devices such as television sets, where operating a conventional mouse on a flat surface would be inconvenient.
New optical-navigation applications will continue to be discovered and commercialised. These developments promise to change the ways in which we interact with modern technologies and will likely enable a variety of new products never before imagined.
These technological and user-interface advancements will doom the mechanical mouse to the same fate as other mechanical contrivances, such as typewriters, rotary telephones, and vinyl records.
History of Optical Navigation
Optical navigation’s origins can be traced back to military target tracking. Tracking cameras can follow aeroplanes, missiles, and other targets by using image sensors to detect when a target is drifting out of the centre of the camera and then by applying small corrections to keep the camera centred on the target. Optical navigation on the other hand, tracks textures or features, and rather than continually re-centring them, measures their movement from the previous position. Both optical navigation and target tracking capture sequential images and use cross-correlation to determine any displacement between the images.
The roots of optical navigation at Agilent (then part of Hewlett-Packard Company) date back to early work on a sensor that tracked the motions of hand-held scanners. Independently, an engineer at Agilent Labs, Gary Gordon, was contemplating how one might interact with the screen menus of entertainment centres of the future, and concluded that one needed a hand-held controller that could optically track its own movements. In the process of transferring this idea to the Imaging Electronics Division for commercialising, a number of other very exciting applications became apparent, and the mouse sensor won out as the first to be marketed.