VR, AR and MR Path to Adoption (2023)

Introduction

Virtual Reality (VR)  has found moderate success in consumer applications, such as gaming. Major gaming manufacturers including Sony, Nintendo, and Microsoft, all have dabbled in VR to a limited success. For example, Nintendo hastily released its Virtual Boy back in 1995. Due to a combination of poor design and  headaches from use, it was unceremoniously pulled off the market in 1996. Note : I was one of the few who benefited from this and was able to snag one from Target's clearance shelf.  Augmented Reality (AR) early usage was in Heads Up Display (HUD) in aircraft to overlay important information in front of the pilot, in car information system, and now consumer and industrial use. Mixed Reality (MR) is now helping to drive the future of VR and AR with  product releases such as Meta Oculus Quest family of MR googles, Microsoft Hololens Mixed Reality googles adoption by the U.S. Army Integrated Visual Augmentation System (IVAS), and awaiting Apple to finally launch its Vision Pro to validate the MR market.

   

Quick Definitions

Virtual Reality (VR) : You buy special goggles (usually called VR goggles - which does not have a camera to pass outside world in), put it on, and everything you see in the goggles is not real (virtual). Augmented Reality (AR): You use an every day device (say mobile phone), aim its camera at a thing (say a building), and additional information about the building is overlaid on to screen of the mobile phone. Mixed Reality (MR):  You buy special goggles (usually called MR goggles - which have cameras to pass outside world in), put it on, and and you see both real world and virtual world visuals.  This is where all the action is, with : Meta Quest (version 3 launched October 2023), Microsoft HoloLens (version 2),  Apple Vision Pro (not yet released, but website is up).

 

Enterprise Adoption

Fast forward to 2018. I am at Mobile World Congress (MWC) in Barcelona to explore how emerging technologies, including VR and AR, is making inroads into industrial applications. Here at the show, AR seem to have gained a stronger traction over VR. VR is a bit "intrusive" in its use model - usually requires donning special glasses or goggles - and if the Nintendo Virtual Boy is of any predictor, adoption might be resisted. AR, on the other hand, is less intrusive - usually only requires holding up a tablet (such as an iPad) or mobile phone (such as iPhone) towards a "re-conditioned" environment.

Gaming Adoption

In spite of early failures, gaming manufacture continue to power through early design challenges to try to  bring VR back to dizzy gamers. SonyVR, Microsoft Hololens 2, and new challengers such as Meta Quest VR Headsets are not giving up the fight to make VR right. In fact, Meta's vision for VR is not squarely targeted at gamers. Rather, it is targeted at those who want to be transported to an alternative reality called The Metaverse.  Meta has already sunk over $20 billion in 2021-2022 and will continue to invest in it - in hardware, software, and content. As an Oculus Quest owner, I have already enjoyed transporting myself to space, looking out at a serene and calm environment - away from the polluted and contentious planet that I want to escape from. 

Daily Life Adoption

AR has now been quietly gaining traction as a utility to improve daily life. In a recent trip to The Hague, I exited the metro and had no idea where to go.  On my moderately new iPhone (Android has this feature too), I have already installed the Google Map app. Google Map gave me the option of using my iPhone to augment information on top of what my camera sees. So from the metro station, I pointed my phone at a nearby building, and Google Maps was able to identify the building, and it proceeded to guide me on where to walk to. Digging into the technology a bit - Apple has made combining maps and camera data easier via its ARKit software libraries.

E-Commerce Adoption

With the mobile phone as the primary way for consumer to discover, learn, and buy products, AR is now enabling the consumer to explore, visualize, and imagine the product more immersive. This picture was taken at MWC. But if you are a Amazon Prime subscriber, you probably have seen AR features to overlay products (such as a lamp) onto your living room.

Keys To Adoption : Content

As a semi-avid gamer for decades, I have owned almost every major gaming console from Atari, Sega, Nintendo, Sony,  Microsoft, and Meta. And from a casual observation, I can say that games sells consoles. That is why every gaming console has a must have launch title  - such as Microsoft's Halo and Nintendo's Legend of Zelda.  Looking at VR, AR, and MR, it will be games and content and a killer app that drives adoption.  In Oct 2023, wedged between the launch for Meta's Quest 3 and the 2023 holiday season, NPR writes Meta Quest 3 review: powerful augmented reality lacks the games to back it up.  And Microsoft's long and contentious $70B acquisition of Activision highlights this phenomenon - Bill Gates was right 20+ years ago when he wrote the "Content is King" essay.

 

How to Scale Contention for VR, AR, MR

Make creation of  virtual worlds easier by further leveraging tools that have already been used to do this such as Unity3D or Unreal Engine. Enable easier 3-D modeling of real world assets, perhaps open source versions of builder tools such as Blender,  Maya (Autodesk), or 3ds Max. Lower  Write code to implement the functionality of your VR experience. This includes user input handling, interaction mechanics, and any custom features.  Use the chosen game engine's scripting language (e.g., C# in Unity, C++ in Unreal

 

Conclusion

MR, AR, and VR is continuing to refine the future of human to machine engagement. It has had a rough start - early failures of Nintendo's Virtual Boy,  soldiers complaining of headaches with Microsoft Hololens,  the snail pace transition to the Metaverse.  But if the gaming industry (and its tools) has anything to teach us - it is that Content is King.

Path from VM to Containers

Business require software to run their operations. These software were written using a software pattern called Model-View-Controller (MVC), bundled into a single software package called monolith (because everything that the software needed to do was bundled into that one software package). That software run on-premise hardware as Virtual Machines. Because those virtual machines ran on on-premise hardware, the business needed to pay for an IT team, plus give them a budget to procure/install/configure/maintain/patch/upgrade/backup. If the software required more compute or storage (say during the holidays when orders are streaming in), IT was summoned to scale up and follow the aforementioned long process - which was as bottleneck. Cloud solved the IT bottleneck issue, where the need to own your own on-premise hardware and the IT team was eliminated. If that need that software to be able to scale up during the holidays, and be accessible from around the planet, while remaining resilient. Also if one of the MVC components of the software failed (which is 8/9 of the combinations), the software dies. Virtual Machines could not keep up with the requirements to scale and be resilient. Problems with monolithic software that ran on virtual machines gave rise to micro-services that ran on Kubernetes containers. Micro-services broke the single monolithic software package into micro pieces, so that when one piece fails, the others run to keep the software running just enough. And microservices did not need the overhead of virtual machines that were designed to run huge monoliths, so containers were invented. Kubernetes is just management software that track containers.

Seeing New Insights From Data - Thanks to Data Lakes

Geo-Data + Social = New Business Intelligence & Insights

Whereas much of the enterprise business analysis has been focused on RDBMS and Big Data sales transactions to test hypothesis and create reports, the adoption of geo-data on business transactions is an area of huge opportunity. Several companies and industries have already adopted geo-data and are reaping financial benefits. For example, UPS is using geo-data to optimize truck delivery routes, aiming for as many right turns at traffic intersections as possible. This will result in an anticipated $50M saving per year. 




Enterprise Insights Exploration of Geo-Data + Social

If you looked at your favorite social media apps, you will find that they want to track your location. These apps take your location—combined with what you are doing, how you feel, who you are with, and why you are there—provide invaluable and difficult to obtain insights about you. For example, if on January 21, 2017, between 2PM-8PM, you were at location 37.79° N, 122.39° W, and you tweeted that you were feeling happy and civic, you were probably part of the Women’s March in San Francisco. Hence, a certain marketing profile can be built up on you for target marketing.

Enterprise Insights   Exploration Hampered by a Lack of Data Diversity

A business analyst, seeing the value of geo-data, wants to perform an ad-hoc query. She has data from Women’s March with an estimated 4 million marchers nationwide. She can query who was at the start location of the Washington D.C. March (38.88° N, 77.01° W), at the starting time (1:15 PM EST), and Tweeted or Liked positively. This is the profile of a enthusiastic, conscientious person. The analyst can also query who was at the end location of the March (38.89° N, -77.03° W), but at the starting time of the March— perhaps a supporter or reporter.  Acting on the speed of thought, the analyst wants access to billions of rows of data, to draw a perimeter of the map to localize around the start of the March, focus on the start time, and filter by contextual data. And after that, try again with another set of criteria so that she can constantly refine her hypothesis to reach a conclusion.  But currently, each click will cause minutes or even hours of calculations before results are seen. This is due to the nature of CPUs – limited number of cores, memory speed, and the types of instructions it excels at.

Augmenting IT Data with  OT Data

Querying a database requires processing cores and fast memory. CPU based servers is limited up to only 22 processing cores and fairly fast memory. CPUs need to be clustered together to be able to serve the queries of billions of rows of data. Another type of processor, called GPU, has thousands of cores and very fast memory. The cores in the GPU process data in parallel and pass data extremely fast to memory. GPUs are so powerful that a single GPU server can sometimes replace multiple clusters of CPU servers. GPU can save money, reduce labor, lower energy consumption, and reduce space over CPU.

Whilst GPU is a great match for looking through billions of records in milliseconds, a database optimized for GPU is needed. That’s where G-DB comes in. G-DB offers two synergistic products – G-DB Server and G-DB Visual. G-DB Query is a GPU optimized database. It is an in-memory, columnar data highly optimized to harness the power of thousands of cores in the GPU.  Every SQL query that you submitted is broken down and re-targeted to run in parallel on thousands of GPU cores. That’s how we are able to return queries on billions of rows in milliseconds.  But the magic doesn’t stop there. Synergistically, GPU is also ultrafast at drawing the output of the query results. This is where G-DB Visual comes in. It renders the results of your queries immediately – so that you can use your eyes to help you brains to discover insights immediately.

Conclusion

Transaction, geo-data, and social media combined will enable insights into people not possible before.  Processing billions of rows of this type of data will be slow and/or expensive on a CPU based system, making this valuable data inaccessible. But GPU based systems, like G-DB, can handle this type and size of data with ease. With G-DB, not only can you gain insights at the speed of thought, you have ultrafast high fidelity visuals to match.

AWS Cloud Storage (S3, EBS, EFS) Explained in One Picture - Hopefully

AWS offers multiple cloud services for storage : 1. Simple Storage Service (S3) 2. Elastic Block Storage (EBS) 3). Elastic File Service (EFS) and more. If you have been reading various documents and comparisons and still confused, I wrote this to hopefully  clear things up a tad.

• S3 (Simple Storage Service - object):  if you have big files, want to share it EXTERNAL TO AWS with many others in the outside world, can use HTTP to access, does NOT need to be attached to an EC2 instance
  
• EBS (Elastic Block Storage - block): fast storage for your EC2 compute to use, usually SSD and NVMe based
• EFS (Elastic File Service - file): share data  (pictures, document, ...) using files INTERNALLY WITHIN AWS, such as between EC2 instances