Escaping the 'Flatland' using Augmented / Virtual Reality.

Roughly a century ago,Edwin Abbot's satirical novel -'Flatland: A Romance of many dimensions' introduced us to an imaginary world that had only two dimensions! The inhabitants of that world were 2D geometric figures (Square,Circle,Pentagon etc.,) moving on a flat surface. They all moved around like a striker in a carrom board and 'depth' was a complete mystery to them.The novel proceeds to narrate a dramatic confrontation between a Square from the Flatland and a Sphere from the Spaceland. Spaceland is another world that resembles the one we live in.The crux of the story boils down to a thought experiment that goes like this:

"How will a two-dimensional Square understand a three-dimensional world? Can a flat square understand 'depth' ?"

Our situation is no different from that of the Square who is stuck in that flat world. 'Flatland',in our scenario, is the construct we live in where space and digital reality is not yet leveraged to its fullest.This article is about transcending such a Flatland and foreseeing the untapped potential hidden in the Spaceland!

0. Defining our realities.

It's pretty much easier to get confused between AR,VR and MR than it is for us understand them. So let's define them before we dive into it:

• Virtual Reality(VR): Completely immerses us into a different environment. We won't be able to see the real-world once we wear VR headsets like Google Cardboard or Occulus Rift.
• Augmented Reality(AR): adds a layer of virtual content on top of our real world. It doesn't completely replace our surroundings, rather it supplements. The virtual content lack spatial awareness and direct interactability. For instance, you cannot place a virtual Flight Schedule card to your left side by dragging it directly in the air.
• Mixed Reality(MR): allows seamless interaction between the real world and the virtual layer that it augments on top of it. For instance, you can edit a virtual bike using a holographic user interface projected in the air. Hololens,Meta are great examples for MR.
• Digital Reality - Although it is not officially defined anywhere, let's use it as an umbrella term for Augmented Reality, Virtual Reality, Mixed Reality and other Immersive Experiences (IX).

Drawing inspiration from the Flatland story, in our scenario, 'Spaceland' would constitute the experiences & possibilities offered by the aforementioned Digital Reality technologies. Now that we have understood the distinctions, let's start the journey!

1. Honey,I shrunk the devices!

50 years ago,when Gordon E. Moore was interviewed about the future of computing, he predicted that the processor speeds, or overall processing power for computers will double every two years.This eventually became the 'Moore's Law'.

In order to understand the effect of Moore's Law, let's take cars for example. If fuel efficiency improved at the same rate as Moore's law, we would be able to drive a car on a single tank of gas,for an entire lifetime! Devices tend to get better over time in terms of size and power. The fact that we have arrived on devices like Hololens, Vuzix Blade and MagicLeap solves the hardware part of the equation for AR/MR. On the contrary, if you don't see people wearing Hololens headset while walking down the street, remember this:

Not everybody wants to look like Daft Punk!

It was evident in the case of Google Glass which turned out to be aesthetically unappealing to the masses leading to outright rejection.However, the transition to better devices would be gradual and hence there could be two major phases of gadget/wearable revolution.

1. First Phase: We would see the mobile devices getting more capable of handling and serving AR/VR content. In this phase, even though first generation standalone AR/VR gadgets exist, they are limited by power, affordability etc. The development of dedicated A.I co-processors and the rise of Graphics Processing Unit (GPUs) could provide a significant boost in the transition phase.
2. Second Phase: Gradually the clunky hardware gets trimmed down to look more like natural glasses and economic at the same time. This phase would be targeted towards the consumer market.Light weight AR glasses will be replacing the mobile phones in the second phase. MR/VR devices will be self-contained with custom processor architectures (like HPU) with long lasting batteries.

At the time of writing this article, Intel announced its new Smart Glasses called Vaunt which closely resembled a normal glass! This is a significant improvement because it addresses the first and foremost aspect of wearable technology; It has to be wearable! Products like these will drive the adoption rate as well as ubiquity.

2. Get ready to use the new canvas: Space.

For the first time, Space becomes our canvas instead of a flat-screen. I was literally thrilled the first time I used Occulus Rift and Hololens headset. At the same time, I felt bizarre on how they carried the old interfaces such as browser and window based UI into Hololens!

Our brain has evolved to think spatially and we'll eventually understand that the flat-screen interface is an oversimplification of our real world!

This creates a need for better design languages and tools that provide a proper spatial interface rather than just a plain window interface. Addressing that challenge, Microsoft has introduced Fluent Design System(FDS) that factors light,depth, material, motion,scale etc., for a better spatial interface. Meta's design and development philosophy,based on neuroscience, strives to minimize the learning curve and cognitive load for the user. Higher cognitive load leads to discomfort; Remember MS-DOS ? ;)

On the tooling side, tech giants like Google and Apple have made their ARCore and ARKit frameworks publicly available to enable rapid AR app development. They are the leading proponents of AR technology and since the announcement of ARKit, 2000 AR enabled apps have been built already! Apart from that, a handful of toolkits and frameworks exist in order to enable such developments. To name a few, ARToolkit, Wikitude, AR.js, Vuforia etc., are extensively used in such developments. On the VR side of things - Unity, Unreal Engine, Occulus SDK, Amazon's Sumerian, etc., are enabling VR content development to a great extent.Apart from Facebook's ARStudio and Occulus Rift, they recently introduced a feature that lets you upload and share 3D content. Users will be able to post 3D artwork and models like images and statuses. With respect to content creation, 'Tiltbrush' lets you do great artwork & sketching leveraging VR.

Imagine using MS-Paint in the air! Drawing in the space around you with your hands. You can do that with Tiltbrush!

You can use such content which is created by artists and designers already and reuse them using free asset stores like Google Poly & Unity Asset Store. Also,do you know that Voxels are on their way to replace the age-old classical Pixels? Intel has introduced Voxel based graphics on Intel architectures that captures highly detailed version of real world objects by sampling them in 3D!

This means that if you capture a football match using this technology, you can see the match from each player's point of view!

MagicLeap has already partnered with Alibaba to create a brand-new AR shopping app.Clearly the means for utilizing space as a canvas is out there and is getting better each day.

3. Whatever we look at or point to, becomes a valid source of information.

Google Lens app offered a sneak peak of what it could be for us to use Augmented Reality much more realistically. In a way, we were introduced to the power of AR when we were playing around with Pokemon Go.

Camera would become the new browser of tomorrow which will be used for rapid information discovery and learning as well.

Fundamentally they are the building blocks for the second phase of AR devices. Aparna Chennapragada, Vice President at Google, mentioned that they will be incorporating Lens into the Google Assistant app. Essentially they are building an intelligent Phase 1 AR device.

Research has shown that using augmented reality in education improves cognition.Knowledge workers will be able to gather information regarding a specific subject instantly using MR/AR. The rate at which data is enriched to provide better information is getting better and the means of 'serving' the data will also be improved by AR/MR/VR devices.

Shared experiences tagged to places and objects could revolutionize communication and knowledge sharing. Microsoft is already working on Volumetric Capture and Presence technology where scenes can be streamed and recorded in full 3D. We would be interacting with Holograms of people,even feel them with better haptic sensors! With Shared experience technology,multiple people can work on a same virtual object as if they were in the same room. Print media combined with AR could bring significant impact to education,advertisement and context-aware content delivery. 'Metaverse', an AR/VR starter guide published by Charlie Fink, is AR-enabled. This means you can scan the markers in the book and you'll see virtual characters popup.

Gradually, we would build an index for real-world objects and people - like how Google has done for web pages.

We will be able to recognize people in business events and exchange contacts with just a blink. When it comes to VR,there are already solutions built to leverage the space around us. Data Exploration and Analytics gets interesting in VR. There are extensive use-cases for learning and training as well. Without a doubt, DR technology is set is accelerate knowledge acquisition for humans than ever before.

4. Evolving Use Cases:

People never knew that computers would become a big hit. In fact, IBM chairman Thomas Watson even predicted this:

"I think there is a world market for maybe five computers."

Although he didn't mean the sleek computers of today, one can pretty much understand that he meant the vaccum tube based big computers. It wasn't until Steve Jobs stepped in and changed the game totally. It was his famous quote "People don't know what they want until you show it to them"! In an interview regarding Vaunt smart glasses, Itai Vonshak, VP/GM of Intel's New Device Group (NDG), mentioned the following:

"When I saw the first smartphone, I didn’t go and say, ‘wow, ride-sharing, that’s going to happen,’” he argues. “But the fact is, ride-sharing would have never happened without smartphones. We’re excited about this because it enables new use cases for developers to come up with. [...] What will happen when we bring a new type of display with new capabilities and new sensors onto your head? I think new use cases are going to happen."

The AR/VR ecosystem and the potential markets which it can disrupt can be seen in the figure below.

*A separate article to cover the use-cases(standard as well as enterprise) will be published soon!

5. 'Spaceland' holds interesting possibilities

As I mentioned in the article before, Spaceland is not just about AR/VR and MR. There are other key technologies which are stupendous on their own and will have a positive effect on the advancement of Digital Reality technology.

• Artificial Intelligence: A.I is crucial to the progress of Augmented & Mixed Reality technology. Dedicated A.I chips that are being built for Mobile devices can make the overlays much smarter and more meaningful. Right from taking our inputs to mapping our environment, A.I plays an important role. Hololens 2 and iPhone X have an on-board A.I chip for such obvious reasons. With the rise of Machine Learning Frameworks and libraries for A.I, we can expect powerful context aware intelligence right before our eyes.

• Internet of Things: The unrealized potential of IOT has always made me think that it is such a 'Sleeping Giant'! Digital Reality will awaken it big time.

For the first time in history, 'Perfect Information' could become a reality for each individual.

The 'Smart Everything' mania is not just about slapping a sensor to everything that we see and calling it 'Smart This or Smart That', it's about the level to which we can dive into and obtain information. 'Perfect Information' is really tough to achieve without IOT. It's knowing anything, anytime,anywhere. In order to make decisions, we need to know things in prior;the better we know it,lesser the risks. IOT gathers humongous data from all connected devices which can be turned into useful information. VR/MR can open interesting ways to manipulate and visualize them.

• Infrastructure: This is very critical to enable Digital Reality. In fact it's one of the entry-level barriers that we have. Data loads will be high compared to normal media when it comes to VR. We will be requiring more bandwidth, better compression technologies to reduce the latency and increase connectivity. The IT infrastructure is seeing rapid progress and it would be a silent yet powerful enabler in the evolution of DR tech.

There are multiple other technologies which can seamlessly tie-up with AR/VR. I wouldn't expect an overnight exponential growth when it comes to DR. However, considering the technological renaissance we are going through, we have better chances of venturing into Spaceland.

Remember, Spaceland is not the end. It would be the start of something big..

What's this QR code?

Visit https://he.ms/ar (Chrome browser preferred) or scan this QR code in a mobile device to access a Web-App that can show hidden content in this article!

Scan the AR markers in the banner & article images using the web-app to play the related AR content. Please do grant Camera permissions to the webapp ! :)

If you are a bit more curious..

Apart from the custom markers present in the article and the banner, which can be scanned using https://he.ms/ar, this is a bonus marker which can be scanned using https://he.ms/ar2 and it has a VR viewer too! Place the mobile device at a distance of 45 cms and scan this marker.Android Chrome browser is preferred for this WebApp. You can access the code for the apps over here