The Transition from Flat Surface to Full Immersion

Remember this spectrum?

Our perceived reality sits on the opposite end of virtual reality. It’s the one we’ve come to know through continuous interactions and everyday experiences.

Anyone that has been sentient for the past decade knows that a large number of these interactions in our current perceived reality are with 2-D screens. Flat interfaces. iPhones, laptops, and monitors. They dominate the information that we consume and the stimuli that we react to.

But this could change in the near future. Given this odd technologic transition phase that we’re in, it’s worth considering that our lives may be partially dominated by different mediums of consumption in the near future — mediums like augmented reality, virtual reality, and mixed reality. The most popular types of media — namely movies, games, artwork, and news — already exist as virtual reality experiences, and these initiatives are only just beginning.

I say partially dominated because it is unlikely that all media will eventually be fed through some trippy immersive medium. But we’ll be given the option to do so anyway, and in some cases, it will be the better option. If information consumption will inevitably undergo this drastic shift, shouldn’t we be talking more about immersive reality, 2-D interfaces, and how to design experiences for each platform?

Rethinking Immersive Reality

I began discussing this question with a close friend the other day (he’s a great sound board for all of the ridiculous ideas that spring into my head and ricochet around). In an attempt to begin organizing thoughts and making distinctions, our conversation revolved around the general overarching question:

“What experiences are made better by immersive reality versus a 2-D interface?”

We wracked our brains trying to figure out the advantages of immersive technologies, coming up with a few: objects and environments in immersive reality prompt exploration that can lead to better learning, characters in immersive reality are more lifelike because you can interact with them, there’s more physical space to put information, etc.

However, given the dominance that 2-D digital interfaces have in our world today, it was hard to imagine this transition to immersive experiences as the primary technology that we interact with in the future. Everything that we currently access through 2-D surfaces is not necessarily a better experience in 3-D or in an immersive environment, and even if something is, it warrants some rethinking and redesign before implementation into an immersive environment. Immersive experiences will continue to develop, but they will only fill their own unique niche of applications.

This prompted us to ask a different question:

“What experiences can’t 2-D interfaces fulfill?”

From there, we could discern the gaps in the functionality of 2-D surfaces, and evaluate whether more immersive experiences could fulfill them.

As that conversation progressed, three problems with 2-D interfaces stuck out:

• You are susceptible to distraction when using 2-D interfaces. Distractions pop up constantly in your environment outside the screen.
• If you’re shown an environment or story on a 2-D interface, you experience it through pictures, videos, and text. You are more of a passive observer, rather than an active participant.
• 3-D visualizations are difficult to visualize in 2-D. It also requires effort to rearrange 3-D designs on a flat surface.
• Some things are better learned through interactions and person-to-person/person-to-environment experiences, like learning a new language, dancing, or rock climbing. 2-D interfaces don’t provide that full experience.

With these considerations in mind, what applications can we dream up for immersive technologies that solve an actual problem?

Potential Applications in Immersive Reality

Let’s consider the last bullet. Imagine that you are rigorously learning Mandarin Chinese, a dialect that is incredibly complex with its ornate characters and 4 tonal options. With any language, it’s very difficult to become fluent without practicing the dialect and fully immersing yourself in the culture. I’ve personally seen my ability to speak, read, and write Mandarin grow exponentially while I was living in China, simply because I was forced to communicate in Mandarin and subconsciously listen to the ambient chatter around me. Today’s language learning tools are fantastic, but imagine the benefits of stepping into Beijing in virtual reality and practicing your Chinese with locals or searching for a store in the bustling marketplace by reading characters. The entire language learning experience provided by a platform like Rosetta Stone could be deconstructed and re-implemented through a journey in immersive reality.

Beijing, China, image via.

Here’s another scenario: imagine that you are currently studying a model of the basal ganglia for your neuroscience exam in a couple of days. It’s confusing to visualize the entire circuit via a 2-D image, and may be better learned if one were to see the actual 3-D structures, walk around it, and point to specific parts. Before implementation, we’d have to handle the fact that individuals will interact differently with a visualization of the basal ganglia in virtual reality or augmented reality, versus a visualization on a flat textbook page of computer monitor. The key thing to note is that these new interactions can open the door to more effective methods for teaching. After redesigning the experience of learning the basal ganglia, we can observe the educational benefits of viewing this complicated structure in VR or AR that were previously unrealized when attempting to understand it from 2-D diagrams.

The incredibly complex structure that is our basal ganglia, image via.

What’s next?

I think these 4 observations are good starting points, especially when you are beginning to define a problem worth solving through immersive mediums. We’re exploring new turf, and old guidelines don’t always apply to new technologies. If we inappropriately force 2-D applications to fit the mold of an immersive experience, we’ll create problems that didn’t exist in the first place. Extrapolating new technology across every possible use case is pointless and dangerous, and we’ve got to compare and contrast every increment of progress if we’d like to make a positive impact. It’s important to question every decision made during the entire process of designing these immersive environments.

Before embarking on a VR/AR/MR addition to your product, ask yourself:

1. Does the feature already exist in another medium?
2. If yes, does the existing feature satisfy its function(s) and user(s)?
3. If no, is VR/AR/MR the appropriate medium to pursue this feature?

Virtual reality, augmented reality, and mixed reality technologies have a long road ahead before they’re fully ergonomic and feasible for use during long stretches of time. But it’s never too early to start thinking more about how to use them, and why we’re giving a d*mn about them in the first place.