In movies people slip on VR headsets and all of a sudden are transported to another place. So it’s understandable that some may be disappointed to find they can still feel and sense their home surroundings even in VR, with current headsets more or less acting as video game consoles you control by moving your hands and face.

After decades of hype, how far are we away from full immersion, really?

Mark Zuckerberg tries a prototype Meta headset, which uses holographic lenses to shrink complex optics into a small package. Meta said creating a consumer version would require lasers that don’t exist yet.

Given how quickly VR has progressed, from almost nothing to the immersive and wire-free experiences we have today, it can be tempting to think that Matrix-like experiences are just around the corner. Purely in a visual sense, getting from where we are now to VR that looks like real life will require some enormous engineering breakthroughs.

At a recent media roundtable, Meta chief executive Mark Zuckerberg shared some early prototypes designed to tackle the issues. One set aimed to introduce variable focal distances; current headsets are fixed at around a metre-and-half, so looking at virtual objects up close can cause discomfort as your eyes fail to focus.

These prototypes have moved beyond mechanical camera-style moving lenses to small electronic ones, but there’s still a long way to go. Eyes can move and focus very quickly, and the brain will pick up on subtle lag and inconsistency. That means a good varifocal system will need to track a user’s eyes constantly and adjust depth to match, with split second movements and incredible accuracy.

Another prototype headset, designed to create incredibly bright HDR VR images.

And then there’s the issue of brightness. Imagine looking up at the sun inside a VR simulation; all current headsets are quite dim due to power and heat concerns. The scene might be washed out or covered in flares to indicate brightness, but you certainly wouldn’t need to close or shield your eyes. It would be as dull as the rest of the scene.

Meta’s research shows that around 10,000 nits of brightness would be needed for realistic VR. High-end TVs and phones capable of high dynamic range (HDR) currently hit around 1000 nits or less, while VR headsets hit around 100.

Meta has built a prototype, which it believes to be the only HDR VR kit that exists, that can hit 20,000 nits. But with some serious caveats. It’s so big and heavy that it requires bicycle-style handles to hold, and the massive lamp that provides the light is so warm the unit needs to be covered in fans.

And of course some prototypes merely seek to increase the resolution of the screens; all current VR systems fall far short of the level of detail needed to achieve 20/20 vision on a standard eye test. The current prototype only covers part of our range of vision and obviously needs to be connected to a powerful computer to generate complex enough images.

A collection of Meta’s current prototype headsets.

“If we can make enough progress on retinal resolution, if we can build proper systems for focal depth and if we can reduce optical distortion and dramatically increase vividness, then, we have a real shot overall at creating displays that can do justice to all the beauty and complexity of physical environments,” Zuckerberg said.

“And at the same time we’re working on how to package all these different technologies into smaller, lighter, and ultimately affordable headsets.”

Yet while visual realism seems to be where the industry is focusing much of its efforts, it’s only one small part of making systems that can truly trick you into thinking you’ve been transported somewhere else.

“The real challenge lies in the fact that we humans integrate our senses together. Yes, your visual system is being stimulated, but your other senses don’t accord with what you’re seeing,” said Professor Spike Barlow of UNSW Canberra.

An artist’s rendering of a potential Apple headset.Credit:Ian Zelbo

In fact having perfectly realistic visuals and no tactile sensory data to match could increase so called VR sickness, or the dislocation and nausea you can feel when there’s a mismatch between senses, which is the exact kind of thing innovations like Meta’s varifocal lenses are trying to circumvent.

“We can get higher visual acuity, but it’s insufficient. It will not convince you that you are there. Getting to a full immersive virtual or simulated environment is going to require a much more invasive connection to the human body,” Barlow said.

Truly tricking a mind into believing it’s somewhere else would require stimulation of all five senses, plus internal faculties such as our visceral senses and inner ear. This could be done with full body suits, invasive machines and rigs that moved a human body around, or it could be done with brain implants. But all this is a long time away, if it ever eventuates at all. For now, experiences will rely on sights and sounds.

“The best VR applications at the moment pay attention to and create experiences where the other senses are not as important,” Barlow said. “So you might be floating in a balloon above a field of flowers, but you’re not going to be able to get out and touch them or smell them.”

So what kinds of new headsets are we expecting to see in the future? Meta has been open about its work on a device it calls Project Cambria, which may be close to market. It’s reportedly a slimmer and lighter, but also more powerful, version of its popular Quest 2 which can pass through a full colour high resolution view of the real world. This would mean it could handle all the VR tasks the current headset can, as well as augmented reality.

And Apple has been rumoured for a long time to be working on a “mixed reality” headset, which like Cambria would allow both VR and AR. Recent copyrights and Apple software have made reference to RealityOS, which indicates the headset and its operating system may not be far away. When asked last week by Chinese media what Apple thought about the future of AR and VR, chief executive Tim Cook said “I could not be more excited about the opportunities in the space. Stay tuned and you will see what we have to offer.”

But while next year could finally be a watershed year for consumer VR and AR, led by Apple and Meta, we could be waiting a lot later for the true promise of perfect virtual worlds to be realised.

Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.

Most Viewed in Technology

From our partners

Source: Read Full Article