Digital Crew applies AI for persistent observation

Thales has developed Digital Crew, an artificial intelligence solution to the problem of information overload and tedium in security and surveillance operations.

Digital Crew
AI makes sense of digital overload

Digital Crew is designed for continuous observation of digital imaging streams. The software application uses pattern recognition to identify people and objects but also has the ability to learn.

It can simultaneously monitor numerous feeds from any digital source – CCTV, a digital imaging camera, a smartphone or a digital periscope.

“Observing a scene, it could recognise a vehicle approaching in the distance before a human, and would recognise it as a tank rather than a car,” said Amyas Godfrey, Thales external communications business partner (land). “It could then recognise the tank as a Russian T-72.”

Lift-off for first electric propulsion system from Thales’ Belfast facility

Interview: Thales UK chief Victor Chavez

The software uses convolutional neural networks, which mimic the way the human brain learns, to build up a library or database of images it can recognise. It is provided with an initial database of objects (or persons) of interest as a starting point and examines images pixel by pixel to search for patterns.

For example, Digital Crew could be shown a flashcard of a T-72. If it then saw a moving T-72, it would recognise the tank when it came into a similar orientation to the original image. “It then records every frame/pixel combination as the tank drives past, so it can in future recognise it from the side, back, or in a cloud of dust,” said Godfrey.

It can be applied to any long-term observation, said Godfrey, such as a 12-hour silent watch or a surveillance operation where one person might be monitoring several screens. Digital Crew can monitor multiple feeds and issue alerts, and never gets tired. “Its real benefit is its persistence,” Godrey added.

The idea was devised in Thales’s Glasgow site in 2016 by an engineering doctorate intern who was looking into applying convolutional neural networks to thermal imaging. It received development funding from the Defence and Security Accelerator (DASA). “This allowed us to develop it into something we could offer,” said Godfrey. It was developed in-house in the UK, with academic support from local universities.

Digital Crew is not offered as an off-the-shelf product but as a software application capable of being tailored to a given requirement. Thales works with prospective customers to devise an initial bespoke database for the application at hand. The device and its database are self-contained in a shoebox-sized package – it does not need large bandwidth to interrogate a database in the cloud.

Digital Crew has been demonstrated to the Royal Navy and Army, and R&D contracts are currently under way in the UK, Canada and Australia.