Friday, 01 August 2014
masthead+quote+image
Advanced search

Live music production system is controlled by hand gestures

Sound engineers in collaboration with recording artist Imogen Heap have developed a live music production system that is controlled entirely by hand gestures.

The team said the system will allow performers to wirelessly compose, arrange and perform music live on stage using intuitive hand movements.

The interface of the system is a pair of sensory gloves designed by Fifth Dimension Technologies (5DT) with additional sensors from x-IO providing pitch, roll and yaw input.

Dr Tom Mitchell, an acoustic researcher from the University of the West of England, told The Engineer: ‘The majority of the development has been in integrating these sensors then mapping them.

‘So we analyse the data to identify gestures and once those gestures have been identified it’s a case of mapping those onto audio processes.

‘We’re well aware that other people have used gloves for musical purposes and even used comparable sensors, but we’re using a mapping system that’s designed in collaboration with Imogen so that she can move away from her equipment and just perform without disengaging from the audience.’

As well as this basic recall, Mitchell said the system allows the performer to adapt and improvise. For example, by making a grasping motion the performer can ’catch’ vocals or any other instrument, and filter these sounds by clasping their hands together and then ’release’ the sound again by opening their hands.

However, Mitchell said one of the key challenges was to integrate everything seamlessly without any lag.

‘We worked on trying on to get the latency as low as possible, which is why all the software has been written from scratch, so when Imogen waves her arm the sound happens… it’s perceivable, instantaneous.’

Heap performed with the gesture system at the recent TED (Technology Entertainment and Design) concert in Edinburgh.

’I walked onto the rehearsal stage, gloved up, without any other equipment and began to sample, loop and apply effects to my voice, acoustic and virtual instruments. It feels so fluid to have everything on me without the need to keep flitting back to my gear on stage,’ she said in a statement.


Readers' comments (3)

  • Essential accessory will be a pack of yellow sticky notes for the gesture hints!

    Unsuitable or offensive? Report this comment

  • Wow, things have moved on since the days of John Michelle-Jarre and his keyboard of lasers.

    Unsuitable or offensive? Report this comment

  • Why need a glove when it is possible to track a hand with a webcam?
    See http://code.google.com/p/ehci/wiki/HandTracking

    Unsuitable or offensive? Report this comment

Have your say

Mandatory
Mandatory
Mandatory
Mandatory

Related images

My saved stories (Empty)

You have no saved stories

Save this article

Digital Edition

The Engineer July Digi Issue

Poll

London Mayor Boris Johnson is lobbying for a £10 additional charge for diesel cars to drive into Central London by 2020, and for road tax on diesel cars and all pre-2006 cars to be increased, to counter air pollution. What option most closely matches your opinion on this?

Previous Poll

Europe's largest tidal array in the Pentand Firth off Orkney will eventually generate up to 86MW of power. What will it take for tidal energy to make an appreciable contribution to the UK's energy needs?

Read and comment on the results here