I have written several times about standards for Virtual Reality (VR) and Augmented Reality (AR), most recently in my Display Daily last December that focused on the Virtual Reality Industry Forum (VRIF).
This week I’m going to look at two other organizations that have made progress in developing guidelines and standards for VR and AR: The Khronos Group and the IEEE. According to ABI Research (subscription required), the standards from these two organizations will support a more homogenous VR landscape, when the standards are available. This will allow software developers to target more easily multiple hardware and display platforms for their applications.
Currently in the fragmented VR landscape, a VR application developer must re-target its app for each set of VR hardware. Can you imagine television content creators needing to re-create their content to target it at each different brand of TV? This would not be a very useful business model, to put it mildly.
Fragmentation in the VR Software Market
The five most popular game engines among VR software developers. (Image Source: M-2 Insights, SID 2017)
Example of the current fragmented VR software market. (Credit: The Khronos Group)
VR software, even for professional applications, is typically based on a game engine. Five of the most popular game engines are shown in the top image. The second image shows how this multiplicity of game engines and VR apps relates to the VR head-mounted display (HMD) market. As shown in this image, each of the four developers of VR apps needs to create five different versions of its app, one targeted at Stream VR-based systems, another at OSVR-based systems, etc. This is not always done, or not even commonly done, so a consumer who wants to sample the apps developed by four companies needs to buy up to five HMDs. As new companies enter the HMD market, which seems to happen every week or so, it would be impossible for either the app developers or the consumers to keep up with the new hardware.
Use of an API to eliminate market fragmentation
The VR Ecosystem when an API is used. (Credit: The Khronos Group with edits by M. Brennesholtz)
When an application programing interface (API) is used, The VR ecosystem becomes much simpler for app developers, HMD developers and end users. With an API, the app developers can write just one version of their app, as long as the outputs from the app conform to the requirements of the Application Interface portion of the API. The same is true for the hardware developers. If the embedded software from the hardware developers that drives the VR display and other VR hardware such as the sound system or the VR controller conforms to the requirements of the API Device Layer, end users then will be able to run any VR app that conforms to API application interface layer standards.
Nick Whiting, Chair of the OpenXR Working Group at Khronos, introduced the OpenXR API at the Game Developer’s Conference in March. (Credit: Khronos Group)
The Khronos Group has been developing its VR API called OpenXR since January 2017 and introduced it at the Game Developers Conference in March 2018. The complete one hour session where the first details of the OpenXR API were discussed by Nick Whiting, Technical Director of VR & AR at Epic Games and Chair of the OpenXR Working Group, has been posted on-line. If you don’t want to watch his entire talk, you can download the slides he used.
While this wasn’t the official release of v1.0 of OpenXR and changes may still occur, there is enough information available for both VR app and VR hardware developers to begin on products that will be compatible with the OpenXR API. Tim Sweeney, founder & CEO of Epic Games, said, “We at Epic Games will wholeheartedly contribute to the effort [of developing OpenXR], and we’ll adopt and support the resulting API in Unreal Engine.” Since the Unreal Engine game engine is used both by Epic and others to generate VR game software, I suspect OpenXR-compatible games are not far from the market.
Some app developers don’t want their software to run on any VR hardware – they want to target only a single hardware platform. Remember when Blu-Ray and HD-DVD were in competition with each other? Some movies were available only on Blu-Ray, some were available only on HD-DVD and some on both. Of course, now all movies are available only on Blu-Ray and the HD-DVD technology has vanished. The same thing could happen in VR HMDs. A HMD manufacturer might want to develop an app that runs only on his HMD, not others. He could still make it OpenXR compatible by using the Runtime A approach shown in the image. A year later, when he wants to release the app to run on any OpenXR compatible HMD, he would not need to rewrite the app – just recompile it with Runtime B. Then it would run not only on his HMD but on any OpenXR-compatible HMD.
Some of the other APIs from the Khronos Group a VR or AR app developer is likely to use. (Credit: The Khronos Group)
OpenXR is not the only API from the Khronos Group a VR app developer will need to use. The OpenXR API is specifically designed to interact with VR and AR sensor, haptic and display devices. Of course, there are other things a VR app needs to do and Khronos has other APIs for them as well, as shown in the image. Khronos Group isn’t the only organization developing a VR API. However, with the heavy-hitters working on the OpenXR development process, it seems like this API is likely to be the most widely adopted.
Some of the companies working on the OpenXR API standard. (Credit: Khronos Group)
ARVR Working Group at the IEEE
The IEEE announced the creation of the ARVR Working Group on May 9, 2017. The announcement included a very aggressive standards creation process. The IEEE P2048 AR & VR Standards family, as initially announced, was to have eight parts. Since then, the IEEE has added four additional parts to the standard to be developed. The twelve parts of the Standard for Virtual Reality and Augmented Reality announced so far are:
- 2048.1 – Device Taxonomy and Definitions
- 2048.2 – Immersive Video Taxonomy and Quality Metrics
- 2048.3 – Immersive Video File and Stream Formats
- 2048.4 – Person Identity
- 2048.5 – Environment Safety
- 2048.6 – Immersive User Interface
- 2048.7 – Map for Virtual Objects in the Real World
- 2048.8 – Interoperability between Virtual Objects and the Real World
- 2048.9 – Immersive Audio Taxonomy and Quality Metrics
- 2048.10 – Immersive Audio File and Stream Formats
- 2048.11 – In-Vehicle Augmented Reality
- 2048.12 – Content Ratings and Descriptors
Since the IEEE is a relative newcomer to the VR and AR standards business (Everybody else is, too!) and has been working on the P2048 standards family for only about a year, no currently usable standard has emerged yet from the IEEE ARVR WG. Looking at the list of topics they are working on, when these standards come out, they will significantly reduce the confusion and fragmentation in the VR and AR markets. While the OpenXR API will simplify software development, it will still allow different software systems to work differently. The issues addressed by the IEEE are likely to lead to more uniform human/machine interfaces when the standards come out and are put to use by the VR and AR user communities. –Matthew Brennesholtz
Analyst Comment
I was invited by the Society for Information Display (SID) to present a paper titled “VR Standards and Guidelines” during the upcoming Display Week in Los Angeles. I’ll be presenting paper 3.1 at Session 3: AR/VR I: Display Systems. The paper will be presented at 11:10 AM Tuesday, May 22 in room 515A of the Los Angeles Convention Center, following the Keynote speeches. Hope to see you there!
In addition to presenting this paper, I will be covering as much AR, VR and Immersive Systems as I can for Meko’s Large Display Monitor and Mobile Display Monitor. Unfortunately, due to parallel sessions, I won’t be able to cover everything. Not to worry – in addition to myself and Bob Raikes, Meko will have four others at Display Week and we should be able to cover most aspects of display technology and the display business that come up at the conference, not just AR and VR. (MSB)
