Virtual reality and augmented reality need brain-computer interfaces to reach their full potential

What does the future mean in terms of personal technology? In my vision, our tools – which are currently bulky, radiation-emitting, time-consuming devices – will become intuitive and unobtrusive. There is no better example of this transformation than your communication terminal which also doubles as a media consumption device – the smartphone. Having already replaced portable media players, the phone is also slowly replacing books (or e-book readers, if you will) and tablets, with the exception of compact cameras.

This is not the end point of progress however. The future is hands-free, when users no longer need to learn typing rituals to use their devices, where holding a screen to type commands would be obsolete. I believe there will come a time when what we actually carry with us will be a device without an integrated screen; instead, the phone will simply become a computing and communications brick, which is tied either to Microsoft HoloLens or Google Glass-style glasses, or more futuristically, to contact lenses or implants that directly project light (real or virtual) on your retina.

Despite the growing interest in augmented reality (AR) and virtual reality (VR), I think companies are still only looking at the short term. Their current implementations only succeed in limited areas, such as media consumption (as with the Gear VR) and heads-up display navigation (as seen on Google Glass). Some of the biggest names in tech are those involved in AR/VR – Google, Sony, Microsoft, Facebook (with Oculus VR), Samsung, Valve and HTC.

So far though, none seem to be trying to solve what is commonly referred to as the “virtual reality input problem”, or in other words to create innovative and intuitive input methods for support. . Instead, users will either have to use gestures on the built-in touchscreen to control the device, or resort to entering commands via the keyboard, controllers, sensor gloves, or eye tracking. Learning these gestures or input commands could prevent AR/VR products from becoming as widespread as the smartphone, which had only the bare minimum of inputs and required no learning time.

We can all successfully push pixels and press buttons on our smartphones, even the dinosaurs among us. These are usually directly translatable actions limited to a few buttons and gestures. A left swipe brings up the right panel, a tap on an icon opens the app, a long tap brings up expanded options, and more. On the other hand, learning to properly manipulate virtual worlds would require the user to learn a much larger set of gestures. There are some cases – like manipulating a 3D model to find design flaws – where hand gestures are appropriate enough, but in other cases, like selecting option number two from a menu dropdown, we’re just wasting time doing something very basic and Easy.

We would also look pretty silly, walking down the road sweeping the air. You could argue that eye tracking negates those flaws – others won’t really notice your manipulation of AR/VR with eye gestures, and you won’t sacrifice the use of one hand either. But let us be both realistic and idealistic here. How hard is it to expect users to control a complex series of inputs with their eyes? We can’t even help but blink, but we’re expected to be able to steer a machine to run our auctions, while not squinting and still managing to see what’s actually ahead. us – which brings us to the primary eye tracking flaw, sacrificing focus and vision for machine manipulation.

Voice commands could be used, but leaving aside the limiting factor of the maximum speed at which you can speak intelligibly, you would of course be audible to others around you. The use of inner voice, or subliminal speech, would be preferable on this front, and is actually an interesting starting point for entering text into a brain-computer interface.

Although some users will accept this learning of a new set of rituals to perform a few actions, a wide range of tasks would leave most people exasperated trying to memorize the actions. The willingness to learn a wide range of gestures to perform a multitude of tasks is something we cannot take for granted. It’s also what separates a casual gamer from a “real” gamer, and we know not everyone will sit down and play properly. If there was an audience more receptive to learning complex entry rituals, or satisfied with controllers as an entry mechanism, it would be the gaming audience. For them, it’s just another combo, a way to get a desired output through a long series of inputs.

That’s why of the four or five players in the game, Sony is perhaps in the best position to make its VR headset a satisfying product; with large audiences already planning to use their controllers to play games while immersed in 3D environments produced by the Morpheus headset. You could argue that Valve would enjoy the same advantage, due to the large number of hardcore gamers on its platform; however, it will have to convince PC gamers to buy a lot of expensive new hardware.

I think complex, non-intuitive gestures and input mechanisms will be a barrier for AR and VR systems to reach their full potential, and the only solution is a brain-computer interface. Imagine looking around a virtual world without moving your head in real life, or running around without seeming to sleepwalk. Of course, once developed, such technology will not be limited to these display mechanisms alone, and would herald a paradigm shift in the way we interact with our machines.

Unfortunately, the first generation of AR and VR wearables seem clunky to use, with the “immersive” aspect their only claim to being truly revolutionary.

While the future seems to me to ideally be a brain-computer interface (BCI), no tech giant seems to have invested too much in technology in the recent past – at least as directly as reported acquisitions and granted patents. Even a product as secretly developed as the Microsoft HoloLens has been leaked several times over the past two years before being announced earlier this year. The fact that nothing has yet been announced about the big companies working on mind control only indicates that these companies seem to be focusing on other areas like artificial intelligence, behavioral prediction, cognitive computing and quantum and natural language interfaces.

Of course, a multidisciplinary approach would be needed to advance brain-computer interfaces, where noise and random thought filtering may be one of the most advanced forms of signal processing to date, skipping the wheat for l tares, and AI, behavior prediction, as well as quantum computing would certainly help speed things up on that front.

It seems that the big funds spent on brain-computer interface research come from government-funded university research or small startups, and big tech companies seem to be waiting for breakthroughs rather than investing their own money in the technology. Notably, BCI research has relied on both invasive and non-invasive methods of reading brain signals, and this has arguably been one of the reasons for the slow progress. Invasive interfaces have in the past been tested on volunteers with refractory epilepsy or other severely disabled patients who consent to graft BCI research onto the intracranial electrodes already implanted in their brains for potential surgical treatment. Modern non-invasive techniques aim to provide deep brain resolution similar to older invasive techniques, with the added benefit of having the subject in motion.

Both types of research have yielded fruits that have helped disabled and paralyzed patients, as well as those with neurodegenerative diseases, to relearn how to move paralyzed limbs or control prostheses.

Some consumer and developer headsets have already entered the market, but with very limited capabilities; headsets like Mattel’s MindFlex Duel, Uncle Milton’s forthcoming Force Trainer II, Neurosky MindWave and Emotiv EPOC. The latter is perhaps the most popular at the moment and provides a wide range of readings gleaned from the brain as well as positioning data that could be used by researchers and developers as input signals in future applications.

The state of mind control apps in the real world is just as limited as the above headsets, such as the MindRDR app for Google Glass. It allows users who wear both the Neurosky MindWave glasses and headset to take pictures with a thought. There have been other examples of mind control apps in the recent past, with game demos such as Throw Trucks With Your Mind and MindMaze requiring users to wear EEG sensors, relax their minds, then trying to control in-game objects. Work on this front is accelerating, with an annual NeuroGaming conference and expo being one of the places where scientists and developers working in the field can share ideas. The hope, of course, is to see the fruits of this labor spread beyond just players, to the rest of us as well.

While major tech companies are publicly working on basic augmented reality and virtual reality implementations, sci-fi fans are hoping for something more. After all, a functional, noninvasive, and intuitive BCI would herald a paradigm shift in personal technology. Admittedly, research on this front is difficult, but that shouldn’t stop the tech giants from investing a lot of money in it. It’s the next big thing, right up there with driverless cars to solve the traffic problem, and tissue regeneration via stem cells.

Of course, it’s always possible that some of these tech giants are already secretly working on mind control. I hope so, because Google Glass and Microsoft HoloLens are just short-term goals.