Anti Ai Ai — Wearable Artificial Intelligence

The post-truth era is just getting started. Near the end of 2017 we’ll be consuming content synthesised to mimic real people. Leaving us in a sea of disinformation powered by AI and machine learning. The media, giant tech corporations and citizens already struggle to discern fact from fiction. And as this technology is democratised it will be even more prevalent.

Preempting this we prototyped a device worn on the ear and connected to a neural net trained on real and synthetic voices called Anti AI AI. The device notifies the wearer when a synthetic voice is detected and cools the skin using a thermoelectric plate to alert the wearer the voice they are hearing was synthesised: by a cold, lifeless machine.

A Powerful IoT Device

To make this work we developed a custom circuit board using the tiny Simblee BLE Arduino chip, and custom iOS app that streams audio to a Tensorflow application running a neural net trained on synthetic voices.

Our PCB being manufactured

Thermal Feedback

We wanted the device to give the wearer a unique sensation that matched what they were experiencing when a synthetic voice is detected. Common feedback mechanisms use light, sound or vibration to alert users. By using a 4x4mm thermoelectric Peltier plate we were able to create a noticeable chill on the skin near the back of the neck without drawing too much current.

Anti AI AI cooling element – Peltier Module 3.96mm x 3.96mm x 2.40mm

Anti AI AI – Wearable Artificial Intelligence

Architecture

To tie the hardware together we developed several independent components. Our neural network formed the backbone of the system, remotely classifying audio data being streamed in via a phone (more about the network below). All training was done in an offline step prior to running our classifier. One improvement would be to continually improve the network based on new input it receives ‘in the wild’, but that was a little beyond our five day timeframe. The iPhone app provided the glue between our IOT wearable and the neural network. This ended up being a rather simple app, doing as little processing as possible as ultimately we can see a future state of this project where the device stands alone. The architecture of the whole system is shown below.

Keras and Tensorflow

Building neural networks has never been more accessible. Keras is a python library that sits on top of Tensorflow and allowed us to build a neural network at a very high level. Rather than just getting something working, we are able to design our network and bring it to life rapidly. Keras essentially does all the work of generating a Tensorflow network, that we can then use to train and subsequently evaluate data. When it comes to actually training the network, Tensorflow still requires decent dedicated hardware to train in any reasonable time frame, and despite having done it several times now, the setup process to get a network training on the GPU remains a challenge.

Cloud infrastructure platforms such as Cloud ML seem promising when it comes to deploying a pre-trained model, but during development having a dedicated machine with a powerful GPU is almost essential. In developing our approach we experimented with several network layouts, including a recurrent network which was in theory, similar to Deepmind’s Wavenet. The neural network we had most success with though was a fairly simple network comprised of multiple 1D convolutional layers. All the code can be found in our github repository

Next Steps

This project is a work in progress. In 5 days we created a functional prototype of our wearable AI. An end-to-end powerful Tensorflow neural net connected to a custom wearable BLE device. When the circuit boards arrive from the manufacturer we’ll bring it all together as a wearable device and continue to improve our neural net with more synthetic content as it is released.

Project Context

As AI starts to robustly understand and synthesise all manner of things, from human speech to the creative style of famous painters, we’re entering an era where anything that can be digitised can be mimicked by AI.

Experiments with style transfer — Gene Kogan

We see the ability to mimic anyone’s voice to say anything as a closing of the loop — if it looks real and sounds real, maybe it is real.

Face2Face in action

Why is this relevant?

The irony is, the more technology improves the more adversarial it seems to become. Asking us fundamentally difficult questions. A great example being how to design software that chooses the best person/people to kill in a self driving car accident.

http://moralmachine.mit.edu/

“The future holds countless more difficult questions posed by technological advances and we won’t be able to answer them all with more technology.”

Blog Logo
AKQA R&D

AKQA is an ideas and innovation company. We exist to create the future with our clients.