When I first reached out to tap the floating holographic button in front of my face, I didn’t expect to be able to feel it as it was pressed. But that’s exactly what the sEMG band I wore made possible. Equipped with a small haptic engine, this floating virtual button suddenly felt real as I tapped it. The experience was mind-blowing, and I couldn’t wait to see what else it could do!
Android Central Labs
Android Central Labs is a weekly column devoted to deep dives, experiments, and a focused look into the tech you use. It covers phones, tablets, and everything in between.
Practically every Sci-fi movie shows someone using a futuristic computer with a holographic display controlled by a wave of their hands. However, the practical use of these kinds of interfaces is extremely limited without one of our major senses: touch. While an sEMG band makes it possible to “feel” virtual buttons and objects via a built-in haptic engine, Meta has a much larger vision than that.
This is about ensuring that XR is as mind-blowing as the original iPhone when a proper pair of AR smart glasses is finally commercially available. We got a glimpse of that with Meta Orion at September’s Meta Connect conference, and it’s clear this band is the way Meta plans users to interact with all future AR glasses from the company.
The band is worn like a smartwatch but isn’t used to tell time or track your heart rate. Instead, it helps make hand tracking more accurate and, most importantly, subtle, by sensing muscle movements and translating them into actions.
But the sEMG band isn’t just an alternative input method. It’s more private than voice input — especially in public places — and it allows you to keep your attention forward instead of on a physical device like a phone or keyboard. That’s important in a world where people are constantly together but never present. It’s the perfect way to control the next-generation pair of Meta smart glasses, and that’s exactly what Meta’s newly published white paper details.
Orion is the North Star
Using a pair of smart glasses without this band is akin to using a smartphone without a touch screen. It can work, but the experience isn’t very good.
Meta’s first major AR glasses prototype, Meta Orion, uses this kind of sEMG band and Meta fully believes it’s a requirement for true AR smart glasses products scheduled to debut this year. Using a pair of smart glasses without this band is akin to using a smartphone without a touch screen. It can work, but the experience isn’t very good.
I’ve seen this kind of band referred to as a “neural interface” before but I think that’s really overselling it. Yes, the band can detect the electrical signals in your arm muscles to better understand how you want to interact with virtual objects, but it can’t read your mind. If anything, neural interface makes the band sound creepy when it’s really not.
In my demo, I was able to almost effortlessly scroll though song tracks by rubbing my thumb forward or backward on my index finger. It felt almost like I was using some kind of fidget roller to control an iPod, but no physical object was in my hand.
Scrolling though an Instagram feed was similarly simple, although that required a little more of my attention than Meta talks about in this white paper.
The sEMG band is vital for next generation virtual interfaces because it allows full input without a touchscreen or a set of cameras that can see your hands. That second part is most important because, if companies like Meta are going to launch a successful pair of smart glasses with a display, they’ll need to cut out as much “fluff” technology as possible to make the glasses small and comfortable.
As Meta points out in the white paper, being able to control a pair of smart glasses even when your hands are full is a sort of holy Grail of device input. It could easily replace the smartphone experience because it’s even more effortless, and we all know people love convenience and ease.
Being able to control a pair of smart glasses even when your hands are full is a sort of holy Grail of device input.
Because it doesn’t require the use of a camera to detect movement, it’s not subject to the same sort of tech bias that cameras can suffer from. There have been many examples of camera-controlled interfaces over the years that didn’t perform as well for people with darker skin tones, but the sEMG won’t have this problem.
The sEMG band I wore was extremely comfortable as it was made with stretchy, breathable fabric and fit very comfortably around my wrist. Unlike traditional EMG medical devices, it’s completely non-invasive and feels no different from wearing a smartwatch. If anything, it’s more comfortable than a smartwatch because the weight is evenly distributed all around the band.
Because you’re not holding anything in your hand, using an sEMG band with something like smart glasses is more ergonomic than a smartphone. There’s a reason “text neck” is a recognized medical condition and this could very well solve that problem entirely.
How an sEMG band works
This is not a “brain-reading device” and cannot read your thoughts. It also cannot beam Zuck’s thoughts (or anyone else’s) into your head.
A full-on pair of smart glasses like Meta Orion naturally have cameras built in that can track the environment around them, similar to a VR headset. These cameras can also be used for hand tracking, but hand tracking is usually a bit more of an active input type since the cameras actually have to be able to see your hands.
An sEMG band helps augment this by providing a secondary input based on wrist muscle movement. The band can detect the electrical impulses in your muscles and understands hundreds of different gestures because of it. It can even detect slight movements, allowing you to be extremely precise with input. The animated image above shows a real hand alongside what
Meta stresses that this is not a “brain-reading device” and that it cannot read your thoughts. As I said before, I thought it was a bit much when Mark Zuckerberg referred to it as a “neural interface” back at Connect. It’s not. It also cannot beam Zuck’s thoughts into your head which, apparently, some people were worried about or they wouldn’t have mentioned it in the white paper.
The sensors come in contact with the skin and read electrical signals in your wrist muscles in the same non-invasive way a smartwatch reads your pulse.
Meta also stresses that the sEMG is non-invasive which is important since clinical EMG technology is invasive. The inside of the band looks a lot like a smart ring, including sensors that slightly protrude. The version of the sEMG band I used was the third generation of its kind and looks substantially sleeker than the first two generations, both of which looked more like a smartwatch than anything else.
The sensors come in contact with the skin and read electrical signals in your wrist muscles much in the same way a smartwatch reads your pulse. They don’t provide any sort of electrical stimulation or feedback. Rather, they are just used to read muscle movements and infer more accurate hand tracking data.
These types of sEMG bands are more attractive for mainstream adoption for a bevy of reasons. Outside of looking nice and being fully customizable, they don’t look any different from a cloth or leather wristband that someone might wear to complete the look of an outfit.
It also feels incredibly natural to interact with virtual interfaces when using an sEMG band thanks to the haptic feedback engine inside. This motor can vibrate and provide physical feedback based on what you touch, similar to how a virtual button on a smartphone display can feel more real when it vibrates correctly.
Touching floating buttons that don’t exist can feel very strange on a Meta Quest headset because there’s no feedback, but an sEMG band solves this problem. It’s a huge win for the concept of floating virtual screens and UIs and would be a legitimate replacement for a tablet in many scenarios.
Meta doesn’t have a timetable for when such a band would make it to market, but I’d be willing to hedge my bets that we’ll see one alongside the launch of Meta’s first pair of smartglasses with a display sometime this year.
Discussion about this post