Power from the people

The UK government is supporting an ambitious plan to create a national grid of computer power equivalent to a supercomputer, but at a fraction of the cost.

From the outside it looks just like another grey 18th-century church on a quiet Edinburgh back street. On the inside, however, the shiny new National E-Science Centre is home to some of the most advanced technology under development in the UK.

Opened by Gordon Brown in May, the centre takes overall control of an ambitious public and private sector plan to create a ‘national grid’ of computing power. That power will be used by academia and industry on a variety of research projects, from modelling climate change and global warming to mapping out new bus routes and increasing engine safety for Rolls-Royce.

Grid computing has evolved from the internet as a way of sharing the computing power of many thousands of PCs to create a network with the power of a supercomputer, at a fraction of the price. The aim is to create a national, or international, grid for computing resources. In the same way as hundreds of power stations around the country contribute to a single network from which users draw their energy, many thousands of ordinary desktop PCs would be yoked together over an internet protocol network.

Whereas a single supercomputer can cost several million pounds, a huge network of PCs with nearly equivalent power can be put together for only a few hundred thousand. Prof Richard Kenway, chairman of the National E-Science Centre, explains: ‘Computing power should be a resource, like electricity or water, that scientists and companies can turn on like a tap.’

The extraordinary capacity of a network like this could be brought to bear on nearly any problem that requires a large amount of processing power, or sheer number crunching. Just take the problem, divide it into its millions of constituent parts, and send each of those parts out over the network to be crunched by a randomly assigned PC.

As most PCs do not use all their processing power all the time, the calculations can be carried out while the machine is idle or sometimes even in the background while it’s being used. The only requirement is that the machine should never be turned off at the mains. Each computer digests the sums, performs the required calculations and sends the answers back over the internet protocol (IP) network to be amassed and sorted into their correct order by a group of central servers.

Such a concept has already been proved in several internet projects and within some businesses. For instance, the SETI@home project asks internet users to donate their spare processing time to crunch numbers from radio telescopes to try to detect patterns that might indicate intelligent life.

In an Oxford University project internet users signed up to give their computers power to crunch through billions of likely molecules that might provide drug treatments for cancer. In industry Intel has used such a system internally to work out the calculations to test its new chip designs. When Intel employees leave work their computers start labouring on the testing calculations.

Of course, there are some applications for which a supercomputer is still preferable: defence, for instance, where systems may have to test the likely consequences of nuclear explosions. And grid computing is very much in its infancy, with some serious obstacles still inhibiting its full implementation. What happens if a single computer makes a mistake, or if it fails to send back its finished calculations? How do you ensure that the returned results are collated in the right order to make sense? The software needed to run such a widely distributed processing system is extremely complex, and has to be robust enough to cope with the errors and many small failures that such a vast network will throw up.

Nevertheless, the researchers at the National E-Science Centre are confident they can crack the problems. With £5.5m from the DTI and £3m from industry, they plan to open up the national grid to industry as much as to academia. ‘Knowledge workers of all kinds are busting to do global collaboration, to share their know-how to work on today’s big issues,’ says John Taylor, director-general of research councils. ‘The present infrastructure of the internet does not scale to this extent, so we need a new structure, and the development of the E-Science Centre puts the UK at the forefront of creating that structure.’

Already the centre is assisting in schemes that will benefit UK industry. One of the biggest is DAME (Distributed Aircraft Maintenance Environment) , funded with £3m from the EPSRC. York, Leeds and Sheffield universities make up the academic contingent, joined by Rolls-Royce, Data Systems & Solutions and Cybula from the commercial sector, with consultancy and project management from Wetherby-based Esteem Systems. DAME will run for three years, and attempt to increase the safety of aircraft by building up a vast database of engine problems and their correct treatments, to be used to make fast diagnoses.

At the end of the project the participants hope to have created a generic Distributed Diagnostics Grid Test-bed; an Aero-gas turbine Application Demonstrator for the maintenance of aircraft engines; and techniques for distributed data mining and diagnostics and an evaluation of the effectiveness of the grid infrastructure for this type of application.

Peter Cowley, chief scientist for research and technology at Rolls-Royce, says the project arose from the company’s enormous databases of aircraft information, of engine performance and vibration, including a collection of all the sounds a Rolls-Royce engine can make, what they indicate, and advice on how to deal with each of them.

‘This was being held essentially in a large collection of audio CDs. What we needed was a way of putting all this information together and making it searchable very quickly, because you want to be able to diagnose what’s going on with the plane’s engine while it’s on the ground, awaiting maintenance, or when it’s in the air,’ he explains.

Every engine can produce more than 1Gb of data per flight, which must be stored and searched to provide a response that is quick enough not to delay the plane’s maintenance.

York University provided its search engine technology, which has been under development for 15 years, says Prof Jim Austin, and the system is being proved on the White Rose Grid, a part of the proposed national computing grid being developed between York, Leeds and Sheffield. ‘This is a major project that will prove many aspects of grid computing, and the lessons that we’ll be able to apply elsewhere,’ he adds. The search engine uses complex mathematical techniques to spot anomalies in large volumes of unstructured data.

When the system is completed, Rolls-Royce is likely to move the technology on to a grid-like structure of its own, possibly using its own computer networks. But the project would have been impossible without the university involvement. Cowley says: ‘We’re very pleased with the DTI funding for the research, as it’s important to fund big projects like this to see how the grid would be run for real, rather than being a collection of toy university pieces of technology.’

This is one of the most important developments under way on the grid, and one of the most advanced commercially. Other industrial developments in their early stages or under discussion include one with a bus company aimed at trying to improve its routeing and timetabling systems; an engine design optimisation project called Geodise (Grid Enabled Optimisation and Design), based at Southampton; a scheme to analyse the motion of ships at sea; and one to apply grid techniques to the monitoring and management of key data associated with major civil engineering sites.

One of the main problems to be solved before the grid can be fully opened to commercial use is how to charge businesses for using it. Just as with electricity, the amount people use must be metered in some way; but when the resource is computing power spread over thousands of PCs, how can it be measured? When the grid has been completed, all businesses will be eligible to apply to use it. In fact, businesses wishing to try out the system could join in the pilot schemes around the country.

The quest to create a nationwide grid infrastructure for industry and research is enormously ambitious, and will take several years to bring to fruition. But the most heartening sign so far is the involvement of the very top levels of government in support of the project.

In opening the centre this summer the chancellor firmly put the case for using projects like this to improve the UK’s competitiveness, promising more money for science and a determination to seek partnerships with industry. In his words: ‘We are fully committed to raising the competitiveness of the UK through research.’ It’s up toindustry now to ensure that the government is not allowed to forget that commitment.

Further information

Researchers at the White Rose Grid (informatics@comp.leeds.ac.uk) invite companies to get involved with their networking project, and some public funds have been pledged to help business involvement. If you would like to contribute, or have ideas on ways to use the grid, contact Jim Austin (austin@cybula.com) or Philip Morris (philip.morris@cs.york.ac.uk).

For general information contact: www.nesc.ac.uk or www.geodise.org