As is usual for a new technology, standards for Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) are, to put it politely, in flux. While at least three formal standards bodies (IEEE, VESA and CTA) are dipping their toes into VR, AR and MR standards, so far there are not any released standards for product developers to work with. Other standards bodies such as SMPTE, ITU, InfoComm and SID have yet to make significant announcements on VR/AR/MR standards.
IEEE
In a May 9th press release, the IEEE announced there would be eight IEEE standards in a new P2048 standards family dedicated to VR and AR. These are being undertaken by the IEEE VR and AR Working Group (VRAR). These 8 standards will cover:
- IEEE P2048.1—Device Taxonomy and Definitions
- IEEE P2048.2—Immersive Video Taxonomy and Quality Metrics
- IEEE P2048.3—Immersive Video File and Stream Formats
- IEEE P2048.4—Person Identity
- IEEE P2048.5—Environment Safety
- IEEE P2048.6—Immersive User Interface
- IEEE P2048.7—Map for Virtual Objects in the Real World
- IEEE P2048.8—Interoperability Between Virtual Objects and the Real World
These IEEE standards deal primarily with software issues and interaction of the user with the real and/or virtual world. They are not hardware standards.
The IEEE VRAR Working Group is scheduled to meet during the 8th Annual Augmented World Expo (AWE), scheduled for May 31 – June 2 at the Santa Clara Convention Center in California’s Silicon Valley. The next working group meeting will be in August in Los Angeles.
If you don’t fully understand why standards are needed, a talk titled “Standardization Needs and Efforts in VR/AR” will be given by two members of the IEEE Standards Association, Rudi Schubert, Director, Innovation, and Yu Yuan, Chair – IEEE Digital Senses Initiative, at 5:00 on May 31st at AWE.
Consumer Technology Association
The Consumer Technology Association (CTA) announced on April 6th that its Technology & Standards Forum in San Francisco, CA, May 9 – 11 would feature a Virtual Reality (VR) Summit. This summit would include the inaugural meeting of the CTA AR/VR Working Group developing AR/VR standards. The meeting was for WG members only but, like the IEEE, the CTA is looking for more working group members. It is not clear exactly what standards the CTA expects to develop – presumably this is the AR/VR Working Group’s first task.
ITU
The ITU is just starting to consider if it needs to get involved in VR standards. If they do, it is likely to be for broadcast VR, not VR gaming or other VR applications with a local host computer.
Graphic used by David Wood in an ITU Workshop on VR and AR (Source: David Wood)
In a presentation to an ITU Workshop on Jan. 19th, David Wood used the above graphic as he discussed the need for standards for VR. He concluded:
- There is a case for preparing Commercial Requirements for Type A VR delivery (panoramic/3DOF) now. [Smartphone-based VR HMDs]
- We need to check whether an adequate number of DVB members would support and use a specification.
- We need to continue to evaluate the situation for panoramic/6DOF and for VR and MR [Type B – Dedicated VR HMDs]
Note that one of the issues he raises is whether anyone will actually use a standard, if the DVB group at the ITU does develop one.
VESA
The Video Electronics Standards Association (VESA) announced on May 8th it has formed a Special Interest Group (SIG) to address standardization for the fast-growing AR and VR markets. One reason why VESA is so interested in VR is the DisplayPort Alt-mode has been used to provide VR HMD connectivity.
“DisplayPort Alt Mode over USB Type-C allows delivery of 4K60 video with audio, USB data and USB Power over a single cable and reversible connector,” said Salman Saeed, director, product management, Qualcomm Technologies, Inc. “We’ve chosen to integrate this technology into the Qualcomm Snapdragon 835 to support emerging, mobile productivity and virtual reality applications, while also reducing our customers bill of materials, design-in time and time to production.”
The VESA AR/VR SIG has been charged with several key missions:
- Establish the hierarchical structure for AR/VR services, including physical connections, data transfer protocols, software drivers and application layers
- Define the basic communication data structure and communication channel between the source and sink devices
- Study the related technologies and algorithms, and standardize them to enable economic and efficient implementation
- Suggest any changes to existing VESA standards that may be needed for better AR/VR support
- Work with other standards bodies to merge suggested changes into related standards
InfoComm and SMPTE
InfoComm and SMPTE do not seem to be taking a very proactive role in VR standards. At least, neither organization has made any formal announcement on standards. This is actually fairly reasonable for InfoComm, since VR is pretty far from its Pro-AV home. Still, they are producing a full day TIDE (Technology. Innovation. Design. Experience.) conference dedicated to VR at InfoComm in Orlando June 13th.
SMPTE’s lack of action is a little less understandable – SMPTE’s current focus is on content creation, transmission and display, key issues for VR and AR.
WebVR
WebVR is an industry group working to produce VR and AR-compatible web browsers, with a special focus on on-line gaming with VR HMDs. The first WebVR browser available is Google’s Chrome for Android systems, not a surprising choice, given the VR focus on mobile gaming. The organization’s FAQ answers the question “What is WebVR?” with:
“WebVR is an API that provides access to input and output capabilities commonly associated with Virtual Reality hardware like Google’s Daydream, the Oculus Rift, the Samsung Gear VR, the HTC Vive, and Windows Mixed Reality headsets. More simply put, it lets you create Virtual Reality web sites that you can view in a VR headset.”
One thing the WebVR group is working toward is what they call “Progressive Enhancement.” If I understand the concept correctly, it means that any WebVR-compatible application will work on any compatible browser on any compatible platform. As you have a more VR-like platform, you will get enhanced functionality and content, compared to just looking at it in a browser on a laptop. Another goal of Progressive Enhancement is to allow users with various levels of computer skills to use WebVR content.
WebVR’s Progressive Enhancement Pyramid (Source: WebVR)
WebVR’s Creative Technologist, Arturo Paracuellos said, “With this Progressive Enhancement strategy we could make content that is easily embedded on Facebook, Twitter or any web page/social network, and could become a great gateway for users to start consuming WebVR content.”
OpenXR
The Khronos Group is an open consortium of over 100 leading hardware and software companies, as shown in the image. They announced a new working group at the Game Developers Conference (GDC) on February 27th.
Khronos Members
The OpenXR working group is going to develop a VR Application Programming Interface (API). An API allows a software developer to write one version of his software conforming to the API standard. This software would then be compatible with any hardware system that also conforms to the API specification. Developing APIs for multiple different technologies is basically what the Khronos Group does.
A VR Ecosystem without an API (left) and with an API (right). (Source: The Khronos Group)
In the image, the VR ecosystem without an API is illustrated. Here, every application developer must re-write its application for every hardware system. With an API like OpenXR, the developer needs to write only one version of its application and interface to the OpenXR Application Interface. Any hardware device compatible with OpenXR would then be able to take the inputs from the application and run them correctly on their hardware. In practical terms, an API is an essential piece of a complex hardware/software system such as VR or AR. With the large number of high-profile members of the Khronos group, it can be expected that the OpenXR API will be the one an independent hardware or software developer will work with.
OpenXR and WebVR are both APIs. Are both needed? While WebVR is ahead of OpenXR in that they already have API software available so users can develop web-based VR and AR applications, in the long run I would bet on OpenXR from the well-established Khronos Group with its heavy-hitter membership.
GVRA
Acer Starbreeze, Google, HTC VIVE, Facebook’s Oculus, Samsung, and Sony Interactive Entertainment announced on December 7, 2016 the creation of a non-profit organization of international headset manufacturers to promote the growth of the global VR industry. The Global Virtual Reality Association (GVRA) will develop and share best practices for industry and foster dialogue between public and private stakeholders around the world.
The goal of the Global Virtual Reality Association is to promote responsible development and adoption of VR globally. The association’s members will develop and share best practices, conduct research, and bring the international VR community together as the technology progresses. The group will also serve as a resource for consumers, policymakers, and industry interested in VR.
Since “best practices” are first cousins of standards, the GVRA will be similar to a standards organization in some respects. Rather than issue standards itself, the GVRA can be expected to encourage other organizations to develop standards derived from the GVRA research and best practices recommendations.
SID
One issue for the display community is that there are no standards for the evaluation of head-mounted displays. The Society for Information Display (SID), through the International Committee on Display Measurement (ICDM) has issued the Information Display Measurement Standard (IDMS 1). Unfortunately, this standard was issued in 2012 and has not had a major update since. The IDMS covers direct view and projection displays but not displays that produce virtual images such as head-mounted displays (HMDs) or head-up displays (HUDs).
To the best of my knowledge, the ICDM is not working on standards for the measurement of these displays. When they do start to work on virtual displays, there will be two major problems. First, virtual displays are very hard to measure correctly. Second, the IDMS is a monolithic document of some 563 pages. A major update that would include virtual displays would probably lead to an update of the entire document, not just the addition of a new chapter on virtual displays. So for years to come, DD readers are likely to keep hearing “I like this one better than that one” from people like me, rather than hard numbers from proven test methods.
Enough, Already!
There you have it: with forecasts in the range of $20B by 2020, everyone seems to be thinking about standards for the VR industry. I haven’t even covered all of them – perhaps the standard file formats and compression schemes from MPEG and JPEG are the most important ones I’ve skipped. While traditional standards organizations are just starting to think about VR/AR/MR, industry consortiums are beginning to actually do something. This is fine, except there are likely to be different “open” standards from different industry groups. (Think DisplayPort vs. HDMI or, heaven forbid, Betamax vs VHS.) At present, the VR and HMD industries seem to be focused more on the Next Big Thing than on standards to make the products of the Next Big Thing compatible with each other. –Matthew Brennesholtz
