Realities Explained

-“Jaime, can you tell us what is Augmented Reality”-. The professor asks.

I very much know the answer but I struggle with the wording. I take my time. I change the definition twice in my head and I end up mumbling something about layering digital objects and information over the real world through a headset in harder that I wished terms. 

No surprise my professor showed a grin of disbelief questioning my, honestly, not-good-enough answer. The problem? I used the term headset to refer to the visual interface of choice in AR… which is totally correct but definitely confusing for an audience used to attach the term VR to the word HEADSET.

AR is best experienced in headsets like Microsoft Hololens or Magic Leap One. I can tell. But very few people have access to them.
Microsoft Hololens 2


I've been lucky to work in VR in the past years with the likes of STEAM, ACCIONA and RAA NY bringing cutting edge experiences for the audiences as head off VR/AR through a contractor in local and international projects.

Developing with VR headsets since Oculus Rift DKI our team was even blessed with a HTC vive Pre sample uint sent by Chet Faliszek himself. I’m eons aways from John Carmack’s expertise level in VR but still not bad for the standard.

HTC Vive Pre

You can say I know a thing or two about the subject… and funnily enough  that never granted me an easy answer for these question.
  • What is VR? What is good for?
  • Should we be using  AR for this and that solution?
  • That new term Mixed Reality. What does it refer to?
  • ...what if we start calling these things XR?
In this post  I’ll try to go through some of the most commonly asked topics related to the subject.


XR variants


Let me loosely describe some concepts:
  1. Virtual reality (VR) is a simulated experience that fully happens inside a computer generated virtual environment. It’s immersive and interactive.
    The goal is induce every sense into the virtual world. Motion, visuals, touch, haptics,...
  2. Augmented reality (or AR) are immersive experiences that seamlessly blend virtual objects with the real world. They present the physical world with superimposed three-dimensional virtual objects, creating the illusion that these objects actually exist.
    The goal is to overlay at different levels the real world with digital information.
  3. Mixed Reality are techniques mixing both VR and AR.
  4. XR is the new term for everything related to visual displays and interactive digital content on it.
I’ll now go through some of the similarities and differences that define VR and AR. 

VR

VR really made the leap with Oculus DK1 Headset around 2013. It is considered to be the first end-user accessible unit mature in terms of usability.

Since then HMDs (Head Mounted Displays) with varying degrees of quality, price and innovation have arised  raising the bar in terms of:

Visual fidelity
Resolution
Motion sickness reduction
Efficiency in FOV (field of view) rendering techniques
Tracking
Head and body tracking
Input tracking
Tethering / Autonomy
Standalone/autonomous
High Bandwidth wireless transmission

I would say that VR visual interface not only is mature nowadays but has also survived the hype cycle and is now finding its place into the market. Although we might argue that locomotion, haptics and input methods are in their infancy there is no denying that VR current state as a whole is suitable for commercial and industrial developments.

AR

We have not reached prime time in AR yet. The aforementioned “Oculus Rift DK1 moment”.

We certainly can use our phones to point at AR tags (serving both as positioning anchors and identifiers) and display fancy toys over them but it is only a proof of concept.
There is still a device to come that really puts AR on everybody hands in a useful and meaningful manner.

Microsoft Hololens and Magic Leap One are the only mainstream focused AR headsets in the market. But at the moment they can only be considered development kits at the moment.

Both VR and AR optimal use scenario should only be considered using standalone headsets.

While you can say that displaying a 3D object over a printed tag through a smartphone’s screen is technically AR, it is definitely a suboptimal windowed experience. It meets ARs definition by today's standards, but it does not make it any good for the intended use and it’s true potential.

Now watch this.

Would you say that solutions like those by the awesome Johnny Lee back in 2007 stand as VR today?

Johnny Lee

In this experiment sensors attached to the user’s head show a virtual word inside a screen so as to replicate the actual perspective looking through a window.

You do have a VR world recreated through that window that behaves as expected but I would arguably say it lacks immersiveness, full-scale context, interaction, and many other things that be now feel are essential to VR.

These same arguments can easily be applied to a comparison between a full fledged autonomous AR headset and an AR app in your phone.

In AR, virtual contraptions appear over real world 3d topography mapped by the device itself. The amount of overlaying happens at various levels.

Imagen the lowest level of overlaying as a tag floating over your friend’s head with an emoji showing his mood. And the highest level as a 10 foot dinosaur totally replacing your friend on-site and replicating his movements as if it was a giant costume. Or maybe a building totally covered up with images or painting or interactive contraptions.

That last capability, to layer our world and create build over it, feels mind blowing to me.
It’s the end of the phone. The desk computer. The TV. The tablet. The watch… every device with non intensive input tasks will be hopelessly exterminated by an unobtrusive, always present device that sits over your glasses.

The same way we started to consider dumb phones those who lacked connectivity and apps 10 years ago, every device not aware of its surroundings will be considered dumb as well in the future.

This wearable computer, this eyewear, not only will know your surroundings but will build over them and will enable contextual interactions  in ways we just can’t imagine now.

Wrapping up

Not only AR makes sense in terms of usability and evolution of current technology but lacks many of the limitations of VR which is dragged by hardware complexity and the extreme difficulty of translating things such as motion, haptics, touch, etc... to virtual worlds that are extremely costly to build.

Pure AR and VR are two very different beasts in “usage scenario” terms.

Also if you consider that an AR headset that overlays 100% of user vision is almost technically a VR headset we could say that a very capable AR headset could also act as a VR headset eventually and not the other way around since VR headsets do not map the environment.

In my humble opinion AR will take the lead as soon as some tech giant as Apple builds a commercially viable AR headset (which is expected around 2022-2024). It will take 5-10 years for it to mature. Meaning computational autonomy (untethered usage), hardware improvements and UI/UX refinement.

Not only that, but it will allow rivaling platforms as Android to build on such innovations offering cheaper alternatives for the mainstream market and standardizing it’s usage.

Thanks for you time.
I hope you enjoyed it.

Comentarios

Halina ha dicho que…
Jaime, muy buena la explicación. Desde mi punto de vista, la diferencia que existe entre realidad aumentada (con o sin dispositivos extras) y virtual, es la "inmersión" en un entorno real (AR) o virtual (VR).
Jaime GM. ha dicho que…
Gracias :).