Members of the Editorial Advisory Board of PennWell Publishing’s ‘Vision Systems Design’ magazine provide their perspectives on vision and imaging technology advancements anticipated in the coming year.
Machine Vision: Back to the Future?
By David Dechow
ISRA Vision Systems/Insight Integration
Despite the nation’s current and hopefully short-lived economic crisis, the future for the machine-vision industry is still promising. However, future success faces several market obstacles, and one of them is user perception of product performance.
Although advancements in computing hardware and software continue unabated, machine vision as a technology seems poised, ready, and willing to repeat its marketplace mistakes of the past. No matter what technological improvements unfold, machine vision still must deliver performance or run the risk of continued skepticism by its target markets. The trouble is that there is a strong temptation to pitch the new generation of smart cameras, sensors, and configurable processors as a cure-all to the challenges of machine-vision implementation. Regrettably, current technology remains some distance from that goal.
Manufacturers of general-purpose systems are pouring resources into making machine-vision performance better, easier to use, and easier to configure. Aided by increasing computing power and the ubiquitous Windows operating system, they are producing proficient advancements in vision products. The additional processing power and speed are being used to attack challenging applications. Highly complex or brute force but compute-intensive algorithms are also being successfully applied. And, a myriad of operator-interface schemes have appeared. The current lineup of enhanced systems that are being configured are incorporating, where feasible, touchscreens, controllers, trackballs, buttons, mice, and joysticks.
Interestingly enough, the gyrations in operator interfacing are not particularly new or original. Older vision-system input devices included, for example, sometimes cranky but elegant light pens and sophisticated three-dimensional joysticks. Despite these and other efforts in usability, machine-vision systems in the general-purpose marketplace have developed, and continue to retain, a persistent reputation as a technology that does not always work.
The lesson of the past is that a slick operator interface does not make machine vision easy to use. The challenge of the future continues to be to deliver what is promised by the technology. With zealous marketing in a highly competitive environment and lower prices with lower margins, the market conditions are set to repeat the optimistic exaggerations of capability that soured the promotion of machine vision more than a decade ago. The popular ‘testimonial-type’ advertisements that show a lowly engineer solving all system-inspection problems should come with the kind of caveat familiar in weight-loss advertising: ‘Results may vary.’
There is no doubt that machine vision is improving. The products available on the market are almost without exception extremely high-quality tools, capable of being applied to an array of machine-vision tasks. However, no matter how easy the system is to configure, the underlying basics of imaging remain. The vision engineer or systems integrator must still select appropriate lighting and optics, must still understand the operating principles of various algorithms, and must still execute a robust integration of the vision hardware. The overall cost of system implementation is always the hardware cost plus application design and engineering, which easily could exceed hardware costs by twofold or more.
Machine vision is not yet a point-and-click technology. The industry needs to be diligent and self-policing in making sure that promises do not exceed capabilities. If we don’t, a new crop of end-user engineers will be saying, ‘No thanks – we tried machine vision once, but it didn’t work.’
Software ease of use needed
By Christian Demant, General Manager
The North American machine-vision market is changing dramatically. This expansion follows on the footsteps of the European market, with the core change in image processing coming from the center of the European vision market – Stuttgart, Germany. The key words in vision technology are ‘easy to use.’ This simple phrase must be adapted to all areas of machine vision, from a single smart-camera system to networked multicamera installations performing multiple tasks.
The key to ease of use is a standard platform, one system that can be adapted from a single-camera system to large solutions that are populated over an entire factory. End users and OEMs must solve the majority of image inspection tasks without in-depth vision knowledge and must be able to swap the knowledge from application to application.
The vision industry must continue to promote the use of icon-based, ready-to-run applications that provide rapid development and shorten the product time to market for OEMs, as well as appealing to unskilled end users. By supporting standard Windows architectures, vendors of machine-vision software can offer an out-of-the-box, plug-and-play environment that is favored by all engineers. Based on these considerations, image-processing tools must be designed as universal, extensible systems that can be used as a component and provide powerful, highly integrated image-processing functionality.
The future of image-processing systems involves the development of highly complex applications completely inside a point-and-click environment with permanent visual feedback to the user. The system then duplicates as a runtime environment. This means that it neither generates code that would have to be integrated into other systems, nor does it interpret a script program.
The ability to use the same software for a multitude of different installations reduces training and instruction requirements and speeds up system development. Specific requirements of individual applications that cannot be solved using built-in functionality can be met through a dynamic-link-library interface. This interface also allows the use of ActiveX components or routines from third-party libraries if required. For OEM customers the use of an OLE automation interface is imperative.
The latest software developments in image processing must also take advantage of Internet technology to support and guide users by offering comprehensive system information and supporting the latest Microsoft 32-bit operating systems in enterprise computing environments. These developments enable scalability from field level to management level with easy data exchange through standard formats. Extensible Markup Language will be of growing importance in this area as a general file format.
The development of industrial image-processing tools will continue in the same cycle, that is, geared toward a machine-vision tool based on the observation that application development is more efficient when carried out by an engineer who cannot do traditional programming. In sum, new vision features will aim toward reducing the number of mouse clicks required and improving flexibility, adaptability, and all the ergonomics of software based on the Windows platform.
Prospects for Industrial Video Cameras
President/Chief Executive Officer
Pulnix America Inc.
Even though the machine-vision industry is accustomed to the ‘silicon cycle’ in the semiconductor and electronics markets, 2001 differs from previous cycles. The world economy seems to be in a giant business downturn, with no strong regional area observed at this time. The US economic slowdown has created a panic in Asia, where US dependency is high, and the drop in business there from last year is more than 50%. Even European countries cannot see independence from the world’s present economy.
The Sept. 11 incident is obviously pushing machine-vision business further downward. The economic recovery seems to have been extended another six months, and at least the first half of 2002 appears dismal.
In the industrial video-camera market, the high-volume machine-vision business is expected to remain soft for some time. We have seen price erosion, but even at low prices, products are not moving.
In the meantime, all companies are concentrating their research-and-development efforts on new product development or on improving existing products. When the economy bounces back, they hope to be ready to crank up production. After a dormant period of the silicon cycle, a new wave of products and technologies usually emerges. So, what is happening now?
1. Getting out of silicon-cycle business – Several companies have decided not to compete in the highly competitive commodity market. Severe industrial restructuring is forcing many companies to look closely at profits and survival modes. Closing operations, relocation of offices, layoffs, losing key personnel, or engaging in mergers and acquisitions will continue.
The same industrial vision-system technology can be used in other markets, such as defense electronics (which is timely in today’s environment), intelligent transportation or traffic-control systems, medical and scientific, and high-end security such as biometrics.
2. Downsizing packages – For companies committed to staying in the high-volume machine-vision market, cost reductions and smaller packages are the name of the game. When new machines are introduced to the market, they will be equipped with miniature cameras.
3. Progressive scan – There is a strong trend from TV-format cameras (interlace scan) to progressive-scan cameras. An analog-output, low-cost VGA-format camera would be a new bread-and-butter product for many machine-vision companies. However, the same battle would occur in this market as happened in the TV-format camera market.
4. Digital output cameras – Thanks to the new Camera Link interface standard, digital cameras and digital frame grabbers are gaining popularity. With the continuing improvements in machine vision and computer power over the years, high-speed, high-resolution applications are now commodity items. The cost of a digital frame grabber is becoming close to that of an analog unit, and the ease of use of application software and camera selection would expand the market. The majority of high-end machine-vision systems offer digital interfacing today.
Another move of digital cameras is to the IEEE-1394 (FireWire) interface standard. This standard is also becoming popular in some areas and is expected to coexist with Camera Link.
5. Resolution, speed, and camera types improve – Machine-vision cameras are moving from 1- to 2- and 4-Mpixel resolution. Operational speed is getting faster. Today’s 30-frame/s requirement applies also to megapixel cameras. Moreover, current 30-frame/s applications are moving to 60 or 120 frames/s.
Even high-end, high-speed applications are approaching 400 to 2000 frames/s, with limited scanning lines. In these applications, we will see special CMOS cameras, where CMOS has the great advantage of addressable readout.
6. Smart cameras – Smart cameras already make up an independent market segment. A variety of smart cameras are being introduced, and their applications may be seen in nontraditional machine-vision and factory-automation fields.
For many small companies, and even for some large companies, in the machine-vision industry cash flow is not healthy. The prolonged recession may cause difficult times for a number of companies. We all hope that good things happen while we are still in business.
Thoughts on the Sub-Thousand-Dollar Vision System
Chief Technology Officer
Modular Vision Systems Division
Twenty years ago, in the earliest days of the industry, machine-vision systems generally sold in the $20,000 to $50,000 range. Today, $3000 to $5000 is typical for a low-end general-purpose system targeted at general manufacturing applications. The advent of CMOS sensors and low-cost, high-performance digital-signal processors (DSPs) makes sub-thousand-dollar systems conceivable, and, indeed, limited-function systems approaching that range are beginning to appear. The sub-thousand-dollar vision system may be inevitable, but what will such systems look like?
It is probably unrealistic to expect that the sub-thousand-dollar system will look just like today’s $5000 device. Materials are only one factor in the true cost of installing machine vision. Reducing the cost of materials without corresponding reductions in the vendor’s cost for research and development, marketing, and selling, and in the end user’s cost for training personnel, system installation, and setup, would not make the sub-thousand-dollar system economically feasible. One must look for applications, technology, and methods of use that substantially reduce the per-unit cost of the above activities.
Just about all general-purpose vision systems targeted at manufacturing applications have a style of use in common. This style was pioneered by Itran in 1983, wherein a ‘good part’ is taught by a user who selects appropriate image-analysis tools from a set provided by the vendor. The user places those tools on an image of the good part using a pointing device, such as a mouse, and configures the selected tools to achieve the desired results.
In 1993, I coined the term ‘reverse-CAD’ to refer to this style, since the goal is to go from a manufactured object to a description, the reverse of the goal of a CAD system. Will sub-thousand-dollar systems use the reverse-CAD style, or will a substantially different, and perhaps much simpler, style of use be needed? Can the reverse-CAD style be scaled down or will better alternatives be found?
One possible answer comes from the vision systems approaching the thousand-dollar range that are beginning to appear on the market. These vision sensors have been inspired more by a scaling up of presence/absence photodetectors than a scaling down of more conventional vision systems. They generally rely on a single image-analysis tool, such as a pixel counter or a template matcher. This tool is trained on a good part and configured to perform advanced presence/absence detection and simple quality inspection. It can run on inexpensive hardware, and, just as important, the limited functionality makes these sensors easy to sell and install.
While the style of use of such sensors is appealing, the key question is whether there are sufficient applications that can be solved by pixel counters, template matchers, and the like. Advanced presence/absence seems plausible, but are more sophisticated quality-control applications feasible? Lengthy experience in machine vision has taught us that there is wide variation in the appearance of good parts, and such tools have not been very successful in distinguishing good parts from bad.
Of course, customers may have much lower expectations in the sub-thousand-dollar price range, but this remains to be seen. Can we keep the single-tool style of use, but develop more sophisticated technology that can address a much wider range of quality control applications?
Another approach is to abandon the idea of training from a good part and use a design-rules method instead. Such a device, which I call an expert sensor, would be targeted at a specific application, such as expiration date inspection. The vendor would program it with considerable knowledge about what a good part should look like. Using this knowledge the sensor would be more effective at distinguishing good from bad. Eliminating good-part training, as well as targeting a specific application, would make the expert sensor even easier to sell and install. The expert sensor gives up versatility, however. Then, the key question is whether targeted applications can be found that are high enough in volume to justify the research and development.
3-D Vision Science – A New Generation
Chief Scientist and Engineer
Until recently, many three-dimensional (3-D) materials-handling systems fell short because of high cost, difficult setup, lack of integration with robots, and a temperamental nature unsuited to hostile manufacturing environments. The ‘ideal’ system is a 3-D vision-guided robotic system for parts handling, inspections, and operations with open architecture, using a single conventional CCD camera, updateable 3-D parts location, and orientation science. This system would circumvent the higher costs of components and the installation and maintenance of more traditional stereoscopic or laser triangulation systems. In addition, they would offer speed benefits, robustness, and require only a simple calibration procedure with high accuracy.
Several research institutes have successfully built laboratory-based, single-camera 3-D systems. They include King’s College, London, England (www.kck.ac.uk/eleceng/vision/inmdex.html); The University of British Columbia, Vancouver, BC, Canada (www.cs.ubc.ca/nest/lci/home); and Carnegie Mellon University, Pittsburgh, PA (www.cs.cmu.edu).
With recent step changes in the capability of processing power, open software, and components, the capabilities of these systems are approaching the level and robustness necessary for harsh industrial factory environments. A number of robotic-vision vendors have announced commercial single-camera 3-D systems, but have not released full-performance details.
The vast number of 3-D sensors are based on three principles: triangulation, time-of-flight measurement including broadband interferometry, and classical interferometry (sometimes known as stereoscopic vision). Drawbacks of the current generation of these 3-D approaches include specialized, proprietary, and expensive sensors and integration; complicated and lengthy sensor calibration routines, and computationally intense algorithms. Specific drawbacks of triangulation include random localization errors, shape alterations of the spot image, and inhomogeneous spatial resolution and shading.
Significantly better and newer components, faster and more accurate algorithms, higher computing power including post-MMX processors, and the fusion of depth-measuring principles are speeding up 3-D developments. Currently, the main 3-D applications include true coplanarity measurements for integrated circuits in semiconductor manufacturing; solder-paste volumetric analysis in electronics; gap and flushness of sheet metal in automotives, as well as visual servoing of robots for welding, assembly, and sealing operations; and shape and sorting in the food industry.
From automotive to aerospace, through electronics manufacturing and pharmaceuticals, and in every industrial sector in-between, there is an urgent need for more-robust, dependable, and flexible 3-D materials-handling systems. With the greater power of the latest open computational environments, greater industrial maturity, and pervasive connectivity, the stage is set for major industrial automation advances addressing these needs.
The Graying of Machine Vision
Vision Systems International
Machine vision is an accepted technology in virtually all manufacturing industries. While the semiconductor, electronics, and automotive industries are heralded as the major adopters of machine-vision technology (and they are), the reality is that virtually every consumer product (and/or packaging) produced in North America has most likely been inspected by a machine-vision system. For example, the container and lid of the soda can you purchased from the vending machine were inspected to assure geometric, coating, and component integrity. Your McDonald’s burger was inspected by an x-ray-based machine-vision system to assure no bone or cartilage was present, and the French fries were inspected to make sure there were no visible blemishes and, maybe, even that they satisfied a minimum size requirement. The oranges and apples you purchased at the grocery store were probably inspected and graded for size, color, and shape by a machine-vision system.
With ongoing advances in the underlying technology, machine vision is becoming faster, cheaper, better, more reliable, more consistent, and more essential to ensure product quality standards. Virtually every industry has application-specific machine-vision systems performing quality checks on products during production and packaging processes. In general, these systems sort products for reject conditions before they reach the ultimate customer. They also collect data that reflect trends, which, when properly interpreted, can result in process modifications that ultimately avoid scrap, add value to scrap, reduce rework costs, lower rework inventory, and avoid warranty costs.
At the same time, as more and more of these application-specific machine-vision systems become available, basic vision toolkits are becoming cheaper. Embedded-vision computers and smart cameras are now available that sell for approximately $1000. The gains made in camera technology and lighting result in stable, repeatable images of captured scenes and often make it possible to use these tools. Vendors are suggesting that these tools are ‘vision sensors’ to reflect ease of use.
Because of their low cost, machine-vision systems are being widely used to monitor value-adding steps along assembly, production, and packaging lines. While simple crossbeam or reflected-beam sensors might verify a single condition associated with a process step, a vision system can verify the results of the process step comprehensively by detecting part present and oriented, date code present and legible, and print pattern present and registered.
These lower-cost vision tools are opening up entirely new markets. The net result has been that for the past few years unit growth in the sales of machine-vision systems have increased at a 25%+ rate. Even with the economic slowdown in 2001, overall unit growth is not expected to stagnate. Because of the slowdown in capital spending in major machine-vision user industries (electronics and semiconductor), unit growth in 2001 will probably be less than 25%. Nevertheless, as lower cost vision technology continues to find new applications, a unit growth of 10% is not out of the question.
This article appears by kind permission of PennWell Publishing. It originally appeared in the December 2001 issue of Vision Systems Design.