I’ve written about the Virtual Reality Industry Forum (VRIF) before and covered their Draft VR Guidelines. The current version of this draft is dated November 27, 2017 and can be downloaded here. The VRIF has been moving fast.
The VRIF started with informal meetings in 2016, formally founded at CES 2017, announced the guidelines project on January 26, 2017, produced a draft dated September 12 (used for my previous Display Daily), wanted feedback by October 31 and hope to have the final version of the guidelines in time for CES 2018. Someone’s burning the midnight oil. The VRIF does admit there is likely to be an update after the January release. Well, of course, anyone working on virtual reality (VR) will need to update himself in January. Not just January 2018, but 2019, 2020, …
The current version of the VR Guidelines is available as a Word document with “track changes” turned on so you can see the changes made between September 12 and November 27. All the edits seem to be attributed to Paul Higgs, who is Chief Strategy Officer of Huawei Technologies’ Video Product Line, a board member of the VRIF and chair of the guidelines working group. He probably deserves a lot of the midnight oil credit.
Paul Higgs from Huawei (Credit: LinkedIn)
This Display Daily isn’t actually about the VRIF Guidelines. Instead, I want to discuss the Workshop On VR Ecosystem & Standards presented by the 3GPP (3rd Generation Partnership Project) and the VRIF. This workshop was hosted by Erickson in Santa Clara on December 4 – 6, 2017. Most of the presentations used at this workshop can be downloaded for free Here.
The 3GPP is concerned with mobile network standards. It covers cellular telecommunications network technologies, including radio access, the core transport network, and service capabilities – including work on codecs, security, quality of service – and thus provides complete system specifications. The specifications also provide hooks for non-radio access to the core network, and for interworking with Wi-Fi networks. Of course, the 3GPP is no longer primarily concerned with 3G, with the current focus more on LTE, 4G and even 5G telecommunications. For a telecommunications organization, they seem to take an outsized interest in Virtual Reality. I guess if VR really takes off, a lot of network traffic in the future will be VR content.
Day 1 of the Workshop on VR Ecosystem and Standards consisted of one session, “VR Standardization Overview”. The session chair was Frédéric Gabin, 3GPP SA4 Chairman and Standardization Manager at Ericsson. The day included seven presentations, all of which are available for download. In addition, the day included eight demonstrations, including:
- Ericsson: VR360 Demo Suite
- Fraunhofer: Multiple demos, including OMAF with Tiled Streaming, Cloud-based rendering for TV sets, interactivity, and spatial audio
- G’Audio: Spatial Audio Rendering
- InterDigital: DASH-based 4K 360-degree Live Streaming Platform
- NGCodec: Live VR360 Encoding in the Cloud
- Qualcomm: MPEG-H Audio for VR
- SK Telecom / Hecas: MMT-based Ultra-low Latency 360-degree Streaming
- TNO: Social VR – Shared Virtual Reality Experiences
These demonstrations are not from some kid in a garage with a PC and an Oculus HMD showing off his startup’s cool gadget. The demos were from major companies demonstrating major portions of the VR standards infrastructure and ecosystem.
Day 2 of the workshop had two sessions, VR Device/Hardware Ecosystem and VR Content Production, each with an accompanying panel discussion. Day 3 of the workshop had one session, VR Service Providers with its accompanying panel discussion, plus a closing panel discussion on Standardization Coordination.
2017 Infographic Map of VR Related SDOs and SIGs (Credit: Jeff Solari of Intel)
The issue of standards and guidelines for VR can be very confusing, due to the large number of standards and guidelines under development. Unfortunately, this explosion in standards is really necessary. Since VR is essentially a new technology from end-to-end, new standards are needed from end-to-end. Since there are so many different aspects to VR technology, there are multiple Standards Development Organizations (SDOs) and Special Interest Groups (SIGs) involved in the development process.
In his talk, Jeff Solari of Intel gave a good graphical summary of all the SDOs and SIGs involved, including the general area of the technology for each organization. Note he lists the VRIF in the Content and Experience category and the 3GPP in the QOS and Transmission category. Even with 26 SDOs and SIGs, “All” is a relative term and this list probably isn’t complete. For example, I know the SID’s International Committee on Display Metrology (ICDM) is working on standards for measuring head mounted displays (HMDs), including VR HMDs, to be included in the updated Information Display Measurement Standard (IDMS).
Apparent Resolution for VR/360° vs Traditional Video Formats (Credit: Michael Chen of Comcast)
Of course, in a single Display Daily, it isn’t possible to discuss every presentation at the Workshop. I want to mention one, however, from Michael Chen of Comcast, who focused on VR resolution and the issues associated with VR bit rates. He said that typically, you must transmit to the consumer a much higher resolution 360° VR image than you need to transmit for traditional television. For example, if you want the consumer to have a VR experience that’s equivalent in visible resolution 480p standard definition, you must provide a 1080p Full HD (1920 x 1080) signal.
Presumably, few consumers want a standard definition VR experience since they are too accustomed to the Full HD experience to accept SD VR, except possibly in the short term. He says to provide this Full HD experience consumers expect, you need to provide the equivalent to a 4K signal. Going beyond this figure, he says that since consumers are becoming accustomed to 4K resolution, to provide a similar VR experience will require a 6K or 8K-type signal.
He called this format “Unobtanium” since 2017 technology simply can’t provide those resolutions and bit rates. Since VR always has rapid motion, motion of the viewer’s head, if nothing else, a high frame rate is needed, perhaps 120Hz. Hence the sometimes astronomical data rates needed for VR. He talks about 8 – 10 Mbps for highly compressed, streamed 360° VR video with Full HD apparent resolution and up to 100Mbps+ for a high-quality, low compression 360° Full HD experience.
Formats for Geometrical Projections and the Interchange of 360° VR (Credit: Michael Chen of Comcast)
Another issue Chen discussed was how to do the geometrical projection needed to pack 360° video into formats that both conserve the full resolution of the 360° video at all angles and are intuitive for the editor who is working on the video. In addition, the format should have a rectangular shape to simplify transmission and storage using conventional video tools. The Cubemap system is one way to do this, but it doesn’t have a rectangular shape.
The compact cubemap has a rectangular shape but isn’t very intuitive, when you look at the flat, 2D image. The equirectangular and the cylindrical projections are more intuitive, rectangular and preserve most of the resolution of the image. Chen seems to prefer the equirectangular over the cylindrical projection because it does a better job of preserving the vertical resolution and field of view than the cylindrical projection.
Note that all the projections involve a “Front” section. But does “Front” really mean anything in a 360° VR image? Paola Sunna of the EBU cited in his talk a YouTube report that 360° Video viewers spent 75% of their time within the front 90 degrees of a video. So, yes, “Front” does have a meaning, even in 360° video. For 360° videos posted to YouTube that have more than 1,000 views, YouTube will provide the content creators “heat maps” showing how often the viewers looked at different areas of the content. This is intended to provide content creators feedback that will allow them to create more engaging content in the future.
As I said, I’ve only scratched the surface of the Workshop. There is a lot going on in the VR Ecosystem and Standards world. Of course, there needs to be a lot going on in this ecosystem for VR to become a viable consumer product in a reasonable time frame. –Matthew Brennesholtz
Analyst Comment
Even without attending myself, I learned a lot from the 3GPP and VRIF Workshop. There were a number of speakers with impressive credentials. I’m used to hearing the founder of some startup touting his VR or AR technology or gadget. The most of the speakers at the workshop were talking at a much higher level than this and were seriously discussing VR infrastructure details. I wish I’d been there. -MSB
