Facebook reveals plan to let users control augmented reality with their thoughts
Facebook is researching how to take minute nerve movements in users' arms and turn them into gesture controls for gadgets. The idea announced Thursday would help the social networking giant launch augmented reality (AR) glasses, which would rely on new ways to control computers and interact with the virtual world.
Picture this: Your home assistant asks if you'd like to play your favorite podcast, and a flick of your fingers in the air lets you select play. Or, you're wearing AR glasses that display images over the real world, and you scroll through your text messages in midair while your smartphone stays in your pocket.
Those are the scenarios Facebook has in mind if it were ever to deploy a muscle-sensing wristlet.
"We're developing natural, intuitive ways to interact with always-available AR glasses because we believe this will transform the way we connect with people near and far," Facebook said in a blog post. But to launch such a product, the social media behemoth would also need access to a new type of data: Your thinking.
Facebook says the wristband product would rely on a "neural interface" that adapts to you and your environment.
In demonstration videos, the prototype looks like a thick, black iPod strapped around the wrist. In theory, sensors on the device would be able to pick up what hand movements you intend to make through a technique for recording nerve signals known as electromyography, or EMG.
"If you send a control to your muscle saying 'I want to move my finger,' it starts in your brain. It goes down your spine through motor-neurons, and this is an electrical signal. So you should be able to grab that electrical signal on the muscle and say, 'Oh, OK. The user wants to move the finger,' " says Nathalie Gayraud, a research scientist at Facebook Reality Labs, in a video.
Rolling out nerve sensing gadgets would enable Facebook to reinvent the computer "click." So instead of using a mouse or tapping on your phone, you'd think about moving a finger to trigger a reaction in AR.
It's too soon to tell whether the device will ever come to market. Facebook says that the research is in the early phases, and that any type of consumer product would be years away. It is not the first company to imagine a world where futuristic glasses enhance the real-world with digitized imagery.
Google Glass, a $1,500 pair of spectacles, flopped with consumers in 2014. The search conglomerate gave them a facelift and remarketed them for businesses. Snap also tried with its Spectacles for Snapchat, a $150 pair of snazzy glasses equipped with cameras, audio and microphones. Those also did not make much of a mark.
Facebook is going after your wrist because that's where people are used to donning wearables. People have used Fitbits for over a decade and Apple Watches for half as long. Your wrist is also close to your hands, which people typically use to control devices.
"It's located right next to the primary instruments you use to interact with the world - your hands. This proximity would allow us to bring the rich control capabilities of your hands into AR, enabling intuitive, powerful and satisfying interaction," Facebook says.
The Mark Zuckerberg-led company insists that data privacy is a top concern, as it should be with a device that takes personalization to a whole new level. Facebook notes that information would be stored locally on the device rather than sent into a cloud. The only information meant to be collected is the intent for you to move your hand, Facebook says.
The development comes two years after the social media company bought CTRL-Labs, a startup developing a wristband capable of turning brain impulses into computer input. The acquisition was reportedly worth between $500 million and $1 billion. The firm was absorbed by Facebook's Reality Labs, which develops Oculus VR headsets.
Controlling devices through thoughts has been shown to work. BrainCo developed a headband with sensors that lets people move items with their mind. The company later developed robotic limbs that detect and react to muscle signals.
Facebook's research news is part of a trio of announcements related to the company's push into immersive technologies.
Last week, it said it's working on a "stylish" pair of glasses to replace a computer or smartphone. Its next such announcement will come this year and has something to do with an all-day wearable, "soft robotics" and haptic gloves, Facebook says.