blog-post
Author Image Author Image
Hakan Karlidag
Head of Tech

Emerging Technologies

2019-07-31

The state of Immersive Web in 2019

In this article, I will try to collate relevant information about Immersive Web (aka WebXR) as of June 2019 while discussing its current state and implications for the future. It is intended as more of a resource building exercise than a deep dive into the technical details of the immersive web technologies. However, you’ll find plenty of technical leads here to extend your research and start building experiences.

What are Immersive Web and WebXR?

WebXR is a term which became mainstream in 2018 to refer to both augmented reality (AR) and virtual reality (VR) capabilities as a web API (going forward, I’ll use the term XR to refer to any immersive technology). Though WebXR is quite a new term itself, recently another term called Immersive Web has entered into our lives - I personally prefer the latter since I think it provides a broader and more extensive description for the technologies covering this space.

Immersive Web means virtual world experiences hosted through the browser. This covers entire VR experiences surfaced in the browser or in headsets with a VR enabled web browser like Google’s Daydream, Oculus Rift/Quest, Samsung Gear VR, HTC Vive, and Windows Mixed Reality headsets, as well as AR experiences developed for AR-enabled mobile devices and headsets.

WebVR enabled Firefox Reality browser.

Furthermore, although AR and VR are the first terms that come to mind, it is good to think about areas like spatial/binaural audio, augmented audio, voice/video-over-IP, voice assistants, IoT and, haptic devices as part of the current or future immersive experiences.

There are no hard limits into what Immersive Web really means or what will it mean in the future. However, I’d like to think of it as the immersive technologies that are open to users ‘on demand’, like any traditional website is, and has the potential to offer users different levels of interaction depending on the hardware they are using to interact. So I pretty much draw the line at the point where users have to download an additional application to run the experience, rather than using their favourite web browser (either via a high-end or basic cardboard-style headset or, maybe in the future, even stationary displays allowing virtual environments to be formed around users in a room).

Finally, before going into the details of Immersive Web, let’s try to define some of the most used terms in the field.

Virtual Reality: VR implies a complete immersive experience that shuts out the physical world.

Augmented Reality: AR adds digital elements to a live view, often by using the camera on a smartphone, smart glasses or headset.

Augmented VR or Augmented Virtuality: These terms are generally used when the user is in a virtual reality experience but can see or interact with a real world visual like a hologram or camera feed of an actual person.

Mixed Reality: Although I have occasionally seen this term being used as a cover term for XR applications, it is widely used for cases where you can jump between VR and AR environments i.e. via an AR porthole in a real-world view.

XR: The term XR has been referred as Cross Reality or eXtended Reality but the X has been mostly used as a flexible placeholder for different types of immersive technologies.

Progressive Enhancement: It is the strategy that progressively adds technically more enhanced layers of features on top of the content as the end-user’s software/hardware allows.

Why would I work on Immersive Web

If you look at the best VR or AR experiences out there, pretty much all of them require a hefty download and a specific headset. Now, while this is all well and good for a user who already owns the necessary means to run the experience, Immersive Web has the potential to lower the barrier to entry, and therefore extend the reach of the experience to the users with varying levels of access.

When AR and VR becomes widely available in the popular web browsers, it can eliminate the need to download an additional application to your device. At that point, the only barrier between the experience and the user is the size of the assets to download - or in other words, how long your user has to wait before the experience starts. Therefore, it is down to the developer’s design choices to build an experience that will work best for the target audience.

Take, for example, an interactive VR experience with high end graphics designed to be experienced with VR headsets and controllers. This will be very limited in terms of reach especially if it is targeted for a specific headset brand. Moreover, whether it is a 360° video or a 3D world experience, it is likely to require a large download, making it even harder for the users on the go. For instance, this 360° interactive video experience requires the Google Spotlight Stories application to be installed and 1.8GB of additional assets to be downloaded afterwards to the user’s mobile device.

Another point where we can see the importance of Immersive Web is the growing number of headsets and the diversity among them in terms of capabilities. Not only do we see different companies entering the scene with their own headsets, but also big companies like HTC and Facebook are broadening their range of devices to target different audiences. This creates a wide variety of devices with different resolutions, FoVs, DoFs, sensors, and controllers to keep in mind when creating an immersive experience. Immersive Web, with a unifying API, can act as a universal building tool that can target all of this space.

Image by dongpung from Pixabay

Optimising and designing for Progressive Enhancement

As of now, while developing tools or applications in this space, the most important thing is to optimise and design your experiences/libraries with a gradually enhanced set of platforms in mind. By employing a more progressive approach, we can provide users with incrementally immersive experiences. Take a VR application for instance. Since we can safely say all modern browsers support WebGL now, and pretty much all smart device users have a browser installed, the following levels of accessibility can be offered:

  • Users with no headsets: a 360° interactive website with mouse (desktop) or reticle and touch (mobile) controls

  • Users with VR headsets: same experience in a VR context with reticle controls

  • Users with VR headsets and VR controllers: same or more enhanced (i.e. physics interactions, locomotion etc.) experience with VR controller support

All these different levels of experiences can be instantly available to the user via a single URL, whether they are accessing through a traditional web browser or a VR-enabled browser in a headset. Although it might look a bit scattered, all it takes is to build your experience to support different levels of controls (mouse, touch, gaze reticle and VR controller) to provide a more frictionless user journey.

Although some VR headsets run attached to a desktop, most users have a smartphone or standalone headset like the Oculus Quest. Therefore, it is always good practice to be mindful about optimisation. If you already have a fully-fledged XR experience you will most likely use a low-fi version (lazy loading, low poly, fewer lights, less draw-calls etc.) of it for your web experience. Think of it as a step or sneak peek on the way to your full experience.

Photo by NeONBRAND on Unsplash

However, I believe the gap between the high-end applications and more humble web applications will reduce very quickly as:

  • more and more native device features become available to browsers,

  • mobile device hardware become stronger in terms of 3D rendering,

  • immersive experiences become more widespread as the hardware becomes more affordable,

  • companies with large user bases create more tools/features targeting this space on top of the current ones, like AR filters/lenses, Facebook Save to WebVR feature, Facebook 3D posts, Mozilla’s Hubs and Spoke, Adobe’s immersive learning platform, Amazon’s Sumerian platform, Unity’s Mobile AR ads, as well as the upcoming Google’s AR in search, Adobe’s Project Aero and AR navigation applications of Google and Mapbox,

  • XR headset adoption increases,

  • more powerful standalone headsets are introduced to the market,

  • the newly adopted file formats like USDZ and glTF gain widespread support and have more matured development pipelines,

  • 5G networks are introduced in widespread in accordance with cloud computing applied to VR and AR use cases.

Facebook Spark AR Studio and Snapchat Lens Studio

As a side note, there is also a middle ground when it comes to AR. You might want to go through the big social networks, which offer the potential to reach an already massive user base. Yes I am talking about AR filters/lenses that Facebook and Snapchat are currently offering. Both platforms are providing tools for creators to make their own AR experiences in a relatively limited but still engaging way. By taking advantage of an application that a large amount of users already have in their mobile devices, you have the potential to share your experience via a single URL just like a traditional website.

Browser support

As far as the mainstream mobile web browsers go, we can say that native Immersive Web support is still being tested on an API level. But, as always, there are polyfill methods. We can either use these polyfill approaches (i.e. stereo camera views, WebXR-independent JS libraries etc.) or provide different views (i.e. 360° view for non-vr-enabled and VR view for vr-enabled browsers) depending on the support level. On the other hand, pretty much all headsets that contain a browser, support Immersive Web experiences out of the box.

I have collated some leads below in terms of browser support since this is likely to get updated quite frequently.

  • WebGL support in browsers to build 3D scenes

  • WebXR support in browsers to build native VR/AR scenes. Currently only Firefox is VR-enabled for the Windows version and Chrome-based browsers offer the feature after enabling a flag. However, keep in mind that XR frameworks or 3D libraries like three.js have stereo view effect to simulate the VR view - effectively enabling VR in all mobile devices.

  • WebRTC support in browsers to create AR experiences using device camera

  • Web Audio API support in browsers to utilise spatial audio rendering

Also there are a number of browser apps available for WebXR experiences.

And of course there are VR browsers supported by a spectrum of different headsets. Oculus Browser, Firefox Reality, Samsung Internet, and Supermedium are among these browsers.

Supermedium VR web browser

Availability in the future

While mainstream web browsers offer most of the immersive features as experimental, pretty much all headsets support one or more XR-enabled web browsers. Especially with the introduction of relatively powerful standalone headsets - eliminating another friction of having to use a PC or smartphone - will increase the level of adoption, the number of Immersive Web experiences and more browser support in turn.

There is no doubt tech giants are investing more and more into the AR space. Since Apple’s ARKit first released in 2017 and Google’s ARCore only came out a year ago, it is fair to say that we will be seeing more AR development as these libraries get more mature and offer more advanced in-built features (like ARKit’s new people occlusion and motion capture capabilities). The immersive web community is already on track to follow these developments with the introduction of the more comprehensive WebXR API. Especially if the companies mentioned above are also chipping in to standardise XR features in web.

Google ARCore and Apple ARKit

Amazon was also one of the latest tech giants joining W3C and taking a leadership role in Immersive Web groups last October. They have already entered the AR scene with AR View back in 2017 and now their Sumerian platform has the potential to ease the creation of immersive content, especially with the power of whole AWS ecosystem behind it.

Although Apple seemed to be focusing on app-specific AR features at first, it wasn’t too long before they brought it to websites with AR Quick Look following the release of iOS 12. While Safari doesn’t seem to be supporting WebXR API in the near future, it is fair to say that Apple is exploring ways of bringing more AR features into the websites.

We have seen Javascript becoming more and more powerful in recent years. One of the main worries when it comes to Immersive Web has always been the incapability of browsers in terms of computationally heavy experiences. However, the perception has been crucially changed with the introduction of new APIs allowing to tap into mobile OS features like sensors, controllers, usb or bluetooth devices. Moreover, all major browsers now support WebAssembly which allows browsers to execute JS code on an assembly-like lower level, effectively enabling more room for expensive computations.

Immersive technologies have great potential in the web space. With huge numbers of web developers and countless open source resources already available, we are set to see more creative examples of usage of traditional web concepts within immersive experiences.

Resources

Developer resources

As you might already know, Immersive Web APIs started with WebVR but were soon superseded by the WebXR API to include AR as well. You can still say AR aspect is chasing behind closely since VR had a head start earlier. It is also important to note that WebXR API is still being refined and there is still time for it to mature.

A-Frame and React 360

Currently, two of the most notable VR frameworks are A-Frame and React 360 (formerly called ReactVR) and they both use three.js libraries (a hugely popular 3D library) under the hood. A-Frame seems to be slightly more popular and slightly easier to start while with React 360 you have access to all useful React libraries.

While these frameworks, along with three.js itself and babylon.js, let you work directly in the code with a lot of control, other tools like Sumerian, Vizor, Hologram or PlayCanvas provide you a visual editor, but with less control.

8thWall, awe and NexTech AR are among the few well established AR frameworks where you can use visual and developer tools to create AR experiences within the platform. Of course you can also still code from scratch by using one of the JS libraries mentioned above.

UX/UI resources

Communities

Notable examples of Immersive Web experiences

Here is a list of some of the notable examples along with a few collections to keep an eye on:

To summarise

In conclusion immersive technologies is one of the fastest growing areas of tech in recent years and Immersive Web has the potential to bridge the gap between different levels of hardware capabilities by offering a more frictionless experience. Web is known to be the platform with most mainstream appeal and it will help Immersive Technologies hugely in terms of gaining mainstream popularity.

The progressive enhancement approach seems to be the key in the transition period we are in, so make sure you are designing your immersive experiences with the enhancement in mind. This reminds me of the time where we couldn’t ignore smartphone screens anymore and started using responsive design as a holy word.

Now that we have started seeing more and more capable standalone headsets with their specialised browsers built-in, it is fair to say that it is not far away before we have to design for VR/AR controllers as well as mouse and touch controls - or gaze controls for immersive environments with no controllers available. There are unique and exciting new UX challenges here to overcome.

There are lots of developer resources that will help you kick-start your first immersive web experiences. Even though XR technologies are not official web standards yet, we have already started seeing awesome examples in web along with great amount of push from companies like Mozilla, Facebook, Samsung and Google as well as the open source community and commercial products.

* Header photo by Stella Jacob on Unsplash