UAV obeys spoken commands

Aeronautics researchers at MIT have developed a manned-to-unmanned aircraft guidance system that allows a pilot in one plane to guide another unmanned airplane by speaking commands in English.

Aeronautics researchers at the Massachusetts Institute of Technology(MIT) have developed a manned-to-unmanned aircraft guidance system that allows a pilot in one plane to guide another unmanned airplane by speaking commands in English.

In a flight test, the UAV (unmanned aerial vehicle) responded to sudden changes in plan and avoided unexpected threats en route to its destination, in real time.

“The system allows the pilot to interface with the UAV at a high level, not just ‘turn right, turn left’ but ‘fly to this region and perform this task,'” said Mario Valenti, a flight controls engineer for Boeing who is pursuing a Ph.D. in electrical engineering and computer science at MIT. “The pilot essentially treats the UAV as a wingman,” said Valenti, comparing the UAV to a companion pilot in a fighter-plane squadron.

Tom Schouwenaars, a Ph.D. candidate in aeronautics and astronautics, and Valenti are principal researchers on the guidance system, which is part of the capstone demonstration of the Software Enabled Control (SEC) program. Professors Eric Feron and Jonathan How of the Department of Aeronautics and Astronautics (aero/astro) are among the principal investigators on the SEC program.

The new guidance system is designed for volatile combat situations. For instance, a pilot might be commanded to gather images of an enemy site located in unknown territory. Rather than putting himself in danger, the pilot could assign a nearby UAV to the task. The UAV moves toward the enemy site, avoiding known threats (no-fly zones) and the unexpected (radar emanating from a missile site), all the while communicating its actions to the pilot in the other aircraft, which follows behind at a higher altitude and a safe distance. The technology also could have applications in the co-ordination of multiple air or space vehicles, such as in air traffic control or the reconfiguration of distributed satellite systems.

The guidance system performed flawlessly in flight tests involving a Boeing F-15 fighter jet and a Lockheed T-33 trainer fighter jet at Edwards Air Force Base in June. A pilot in the manned F-15 issued mission-level commands in everyday English to the T-33, and the T-33 executed them, maintaining a trajectory safe from threats, and at one point adjusting to a last-minute change in the predetermined mission plan. The T-33 was a substitute for the actual UAV in the test. It was manned by a pilot and crewperson who were on board to manage the aircraft in case of failure, but the vehicle was controlled entirely by MIT’s software, which ran on laptops placed inside each plane.

“Through the recent experiments, the SEC program has demonstrated advanced behaviours that may now be integrated into the next generation of unmanned vehicles,” said John Bay, the US Defense Advanced Research Projects Agency’s (DARPA) SEC program manager.

Three elements combine to make MIT’s manned-to-unmanned air vehicle guidance system more flexible and more “intelligent” than previous systems.

First, the team worked with Teragram Corporation, a software company specialising in language technology, to create a natural language interface through which the two vehicles communicate and co-ordinate their actions. The interface translates the pilot’s human language into the UAV’s machine language, and vice versa. “It allows us to task machines at a higher level, improving safety and efficiency,” said Feron.

Second, Valenti designed a task scheduler that keeps track of the frequently changing mission data from the manned vehicle and interprets it into tasks the UAV can perform. The task scheduler is integrated with the third element, Schouwenaars’ safe-trajectory-planning algorithm. The algorithm is based on mixed-integer linear programming (MILP), an optimisation framework originally developed for operations research.

Feron and Schouwenaars started applying MILP to the problem of aircraft routing in 2000. Unlike earlier technologies, MILP allows trajectory planning that can guarantee against collisions. Moreover, the trajectories can be computed while the vehicle is flying, requiring no pre-planning.

Using the off-the-shelf optimisation software CPLEX, Schouwenaars fine-tuned the MILP-based guidance technique to enable the UAV to choose the fastest safe path to its destination, and then change course in a split second when faced with a new command or a sudden obstacle. The June flight tests marked the first time that a manned air vehicle used a MILP-based guidance system to control a UAV.

Already, said Feron, “the aerospace industry is using our system in its most advanced UAV programs.” He and his team are currently working toward implementing their guidance technology in systems with multiple air vehicles.