Build it and they will come: revolutionary arm sensor seeks application

<b>The technology is there, the question is how you use it? Tim Fryer takes a look at some interesting new sensor technology that could point the way forward.</b>

Thalmic Labs states its mission is to merge people with technology. Its core product, the Myo armband, provides the necessary link and, at the time of its launch last year Stephen Lake, CEO and Co-founder of Thalmic Labs said: "This final design is rugged, while also being lightweight, making it easy for the Myo armband to become a part of our everyday lives."

And the product is pretty unobtrusive. At 93g the armband weighs less than a typical watch and is essentially an elasticated band that is worn round the forearm. With a maximum width of 1.1cm it fits comfortably under most clothing, and as it has no cameras in its sensing armoury then it is no problem if it is covered.

Since development began in 2012 the look of the product has evolved from early 3D printed prototypes to the production design that was released at the end of 2014. The technology has advanced at a rapid rate as well. With a team of engineers brought together from across the world, Thalmic Labs developed a new type of muscle activity sensor from the ground up, made countless advances in gesture recognition algorithms, and developed a one-size-fits-all industrial design that will accommodate ages twelve and above.

"The band itself is worn on the forearm, the thickest part of your arm because that's where you get the most muscle mass," explained Chris Goodine, who handles developer relations for the company. "This is where we get the strongest signal and the best performance."

The device includes a nine-axis motion sensor to detect all the movements and rotations of the arm. "This is strictly the accelerometer gyroscope and magnetometer together in one package called an IMU - an Inertial Measurement Unit," said Goodine. "The way it works is it understands the orientation of the device, so if the device is oriented upwards it knows that it's going upwards, when oriented down it knows that it's going downwards" The same is true of left and right and rotation (roll).

"This is all the motion sensor at work," continued Goodine. "Probably the most interesting thing is that you can combine that with gestural data. Gestural data comes from the muscle sensors at work."

The sensors that do this pick up the electrical activity in the forearm muscles. There are eight of these EMG (electromyography) sensors in the band. When a hand is flexed the forearm muscles are contracted, producing most electrical activity. By comparing the signals from each of the sensors it can be calculated what hand movements are taking place. Goodine said: "You start to recognise these sensor patterns. You can extrapolate those patterns, make them work with everyone, and that's basically how our gesture classification works."

There are five gestures that can be captured currently: a clenched fist, finger spread, wave in, wave out and 'double tap' pressing middle finger and thumb together twice. However, combine this with the motion information and the possibilities are vast. The device samples at 50 frames per second for the gesture recognition, and in terms of positional accuracy it is a fraction of a degree on all axes.

The device can be put on in any orientation and can be calibrated by simply performing a couple of hand manoeuvres. The device will buzz when synchronised and equally will buzz if for any reason the device becomes dislodged and needs resynchronisation.

The Myo armband has a rechargeable lithium-ion battery and is designed to last a day of continued use. When not connected to a device and in standby mode, the battery lasts more than a week. The Myo armband works with Windows, Mac OS X, iOS, and Android devices that support Bluetooth 4.0 LE. Developers can build applications with C++, Objective-C, Java, Lua scripting as well as a number of community supported language bindings.

Through the Bluetooth connection the Myo can be used to interact with any other digital device, but while some applications – video gaming being the obvious one – seem made for this technology, Thalmic Labs is looking to the developer community to realise its potential.

Making MYO mine
Thalmic Labs presented the technology at SolidWorks 2015, not because the product was developed in SolidWorks, but because the event attracts upwards of 5000 design engineers, and that is the sort of developer community that Thalmic Labs' needs.

It could even be used directly in the CAD environment. By recognising the pattern in the sensor channels that were the result of the user making a fist, for example, that motion can then be translated into an action, like holding down a button. So the user could clench a fist to effectively grab control of an object on a screen which can then be moved around and rotated by movement of the hand.

This type of application is very much for demonstration rather than daily design, as Goodine clarified: "This isn't really a keyboard or mouse replacement. If you were a design engineer working with SolidWorks it's unlikely that you're going to be in front of your computer manipulating your models on a regular basis. What this is really meant for is in front of a larger audience – when I have a model and I want to walk you through it.

"This is an emerging technology. It's a completely open technology and we built it up knowing that it's really our developers that are going to take this to the next level. So in addition to building up this hardware, we put a lot of effort, basically half of our engineering efforts, into building up a software developer kit that enables developers to work with whatever data that we have."

Moreover, Thalmic claims to have bought the technology into an affordable arena. Other EMG systems with dedicated controller and electrodes not only would cost 'thousands of dollars', they also require the user to shave the skin, apply a conductive gel and adhere the sensor onto the area somehow. By comparison the Myo armband costs around $200 and, claimed Goodine: "It can do essentially the same thing. It may not have the same frequency, but it has the same capabilities."

There are some applications that appear ready made for the Myo. When watching a movie online, the viewer can play, pause, fast forward, rewind, turn the volume up and down by using the keyboard or mouse. The Myo armband could very easily be used for this level of control – it is an application that already exists.

"Another really interesting consumer application is music control," said Goodine. "If you're a jogger or a biker, snowboarder or skier, and you want to control your music without having to stop or take out your phone from your pocket and taking off your gloves in the cold, you can just flip your wrist to change tracks.

"So these are consumer focused applications that do exist today and we do encourage developers to build up these use cases. There is a consumer focus for certain use cases like presentation control - if you go up on stage and you're constantly presenting. The great thing with this is that the average consumer can do it without having any sort of technical background. So we do encourage that, but we also encourage this sort of maker community where people are taking it to the next level through interesting innovations."

www.thalmic.com