Virtual reality used to diffuse tense stand-off

New virtual reality programs developed at the University of Southern California are being employed to teach US military personnel sound judgement and clear thinking in an emergency.

New virtual reality programs developed by the University of Southern California’s Information Sciences Institute (ISI) are being employed to teach the US military sound judgement and clear thinking in an emergency.

The virtual reality programs developed by the University of Southern California’s Information Sciences Institute (ISI) and two other co-operating USC institutes are said to meld advances in artificial intelligence with advanced work in rendering virtual environments in animation and sound.

A ‘Mission Rehearsal Exercise’ developed for the US Army by ISI, the USC Institute for Creative Technology (ICT), and the USC Integrated Media Systems Centre (IMSC) takes soldier-trainees on a virtual reality mission in a troubled town in Bosnia. There, they must deal with a situation threatening to spin out of control.

It uses a large curved screen that looms around trainees. Combining with the screen images is highly directional and lifelike ‘immersive sound,’ creating an illusion of being present at the scene.

The scene is populated with animated figures that exist only as computer programs, but are nevertheless autonomous agents who can interact with human trainees in real time.

In one simulation scenario, an officer enters the village to deal with one problem — a weapons inspection team being threatened by an angry crowd — and finds another one as well: an American jeep has accidentally struck and injured a local child.

Should the lieutenant split his forces to deal with both situations? If so, how? Meanwhile, a TV camera crew arrives, further complicating the situation.

The scene of the village uses 3-D computer modelling to create basic shapes visible from any angle enhanced by texture mapping.

The group used commercial software from Boston Dynamics to add animations of people, and a sound system — with multiple sound tracks (up to 64 tracks for some effects) played through 12 speaker channels.

Two kinds of autonomous software agents inhabit this environment. Most are said to be basic robotic programs that carry on a limited range of pre-scripted, routine behaviour.

Three other types of autonomous software agents — the detachment medic and sergeant, and the anxious mother — are more complex. These are software actors who have substantial abilities to react to what the trainee does. Their faces change expression and can respond to speech.

Scripted characters are relatively easy to create, said project leader Jeff Rickel. AI characters are more difficult to program, but can interact with people and with their environment in more flexible ways. This makes them well suited for key roles such as the mother, sergeant, and medic, which all have to interact with the human officer.

‘The work we have done in one way shows how far away the holodeck is — but in another shows how useful it may be,’ Rickel said. ‘The project represents a grand challenge for both AI and virtual reality, but the potential payoff is a powerful new medium for experiential learning.’