The Hollywood Professional Association (HPA – formerly the Hollywood Post Alliance and now a part of SMPTE) has scheduled its second annual HPA Tech Retreat UK for 11 – 13 July 2017 at the Heythrop Park Resort in Oxfordshire, UK. On 11 July, there will be a half-day programme covering Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) as a TR-x (Tech Retreat Extra).
To introduce the TR-x and to provide background information on VR, SMPTE had a webinar on June 20th led by Nick Mitchell, VP of Immersive Technology at the Technicolor Experience Center. Mitchell was joined by Mary-Luc Champel, Standards Director-Principal Scientist, Technicolor, and Richard Welsh, Co-founder and CEO of Sundog Media Toolkit Ltd. and Co-Chair of the HPA Tech Retreat UK. The webinar did far more than introduce the TR-x, however: it provided significant insight into the state-of-the-art of high-end VR.
Resolution, frame rate, pixels per second and bits per pixel for various Cinema and VR formats (Image: Technicolor)
Mitchell said his background at Technicolor involved the cinema and he was accustomed to very high quality images. One of his major concerns was the very high bit rates required to provide cinema-quality VR images. He said cinema uses JPEG-2000 compression and a bit rate of 250Mbps. For a 4K, 24FPS movie, this works out to about 1.2 bits per pixel on the average. For 10 bits/color, this requires a compression ratio of about 26:1. Higher resolution, higher bit depth or higher frame rates would require cinema to use a higher compression ratio.
He said that VR content is most often compressed using H.264 compression (MPEG-4, AVC) and is normally limited to 150Mbps. VR content also needs a high frame rate to minimize VR sickness. For a 4K x 4K 90FPS VR experience, this works out to 0.099 bits per pixel on the average. For 8 bits per color, this means a compression ratio of 242:1. Mitchell did not have a solution to the very high compression ratio needed for cinema-grade VR, he just pointed out this is a major problem. He added the problem is actually worse than this since many current VR delivery systems do not use 150Mbps but are limited to 50Mbps, increasing the needed compression ratio from 242:1 to 726:1. He called this a “serious bit starvation “ problem.
Champel said people have suggested sending only the portion of the image a viewer is currently looking at, rather than the entire 4K x 4K image, as a partial solution to this problem (We have reported on work by the Fraunhofer in this area – MWC 2017 Round-up – Man. Ed.). His belief is that this is not practical for high-quality VR because of the latency between a rapid head motion and the change in the bit stream to represent the new viewpoint. This visual latency is likely to contribute to VR sickness.
Timeline for MPEG tools and standards for VR content. (Image: Technicolor)
Progress is being made on tools and standards for VR but the progress, especially the standards progress, is slow. Champel showed the timeline on MPEG tools and standards which shows dedicated video coding for VR arriving in 2021 and a codec for light field imaging arriving in 2022. As she is Technicolor’s Standards Director, I tend to accept what Champel says. Standards aren’t going to be here any time soon.
He said the VR Industry Forum (VRIF) was announced at CES and is currently working with standards organizations to develop MPEG-I (MPEG Immersive Media). The VRIF does not develop standards itself – it is an advocacy group. However, members of the VRIF such as Technicolor are working on these standards through existing standards organizations. When complete, MPEG-I is expected to become ISO/IEC 23090 – Coded representation of immersive media. According to Champel, there are five parts to MPEG-I currently planned:
- 1) Architectures
- 2) Omnidirectional Media AF
- 3) New & Immersive Video Coding
- 4) New & Immersive Audio Coding
- 5) Point Cloud Coding
- 6) And more to come
The issues raised in this webinar will be discussed in more detail at the TR-x at the HPA Tech Retreat. Registration for either the TR-x half-day seminar or the full Tech Retreat is available Here. –Matthew Brennesholtz
Analyst Comment
All three speakers agreed that VR was currently in the “Trough of Disillusionment” of the hype cycle. VR faces formidable obstacles including the bit rate issue discussed, the lack of standards for the next four years or so and the need for much higher display resolution, perhaps as high as 10K x 10K per eye. VR has shown value in various professional applications but sometimes I wonder if VR can ever start to climb the “Slope of Enlightenment” as a consumer entertainment medium. MSB
