The Immersive Digital Experiences Alliance (IDEA) used the 2019 Light Field and Holographic Display Summit to release three new documents that describe how advanced media formats, including full light field data, can be packaged for delivery over commercial networks. The group also proved the format works by showing the same content presented on several type of advanced displays.
Called the Immersive Technology Media Format (ITMF), the royalty-free specification is a display-agnostic interchange format for conveying high-quality and complex image scenes to a wide variety of immersive displays. In an impressive piece of collaborative work, IDEA released the first set of ITMF specifications only six months after the alliance made its debut at the 2019 NAB Show.
The ITMF specification is based on ORBX scene graph technology, a format created by OTOY and now supported by dozens of software systems used in 3D animation and game development. Starting with the ORBX scene graph format, ITMF is designed as an interchange and distribution format. Applications range from passive viewing, gaming, and telepresence, including six degrees-of-freedom (6DoF), to next-generation light field displays. As was showcased at the Display Summit event, the ecosystem for light field capture, processing, transport and display is clearly being developed by a number of companies. The ITMF fills the transport gap in the ecosystem.
The initial ITMF suite includes three documents: a scene graph specification, a container specification, and a data encoding specification. The scene graph specification describes the physical attributes of a scene or object in mathematical terms and then provides instructions for the complex computational machinery at the receive end to generate the image.
While not fully implemented in the initial draft, the goals for the ITMF specification are based on five key attributes:The unique ability to achieve a fully immersive image, based on the best possible image representations and with light field technology as the highest common denominator.
- Solutions that are practical for transmission, with the ability to transmit complex data to viewers’ homes without using an inordinate amount of bandwidth.
- Display-agnostic solutions that support any type of display or viewing device, ranging from smartphones and extended-reality headsets to home theaters and expansive cinematic displays at entertainment complexes and theme parks.
- Support for media-aware networks to optimize delivery with available network bandwidth and support of different display devices, enabling deployment of new immersive services
- An open and royalty-free interchange format that any vendor can access and implement, thus spurring a broad range of industry solutions and speeding adoption.
At the Light Field and Holographic Display Summit, IDEA presented proof-of-concept demonstrations based on the new set of ITMF specifications. For the demo, a variety of content was first selected and described as a scene graph. A scene graph is based on a 3D model of the objects and people in the scene with multiple types of textures added to create the degree of immersion desired. The ITMF media is hosted in a cloud-based server. To display the content, a hand shaking occurs whereby the display device declares what type of display it is and what level of fidelity it can support. Only those assets needed by that display are then streamed over the network to the display in the required format.
One example was a VR headset that revealed that the media was indeed 360º media that allowed the viewer to look around inside the content (delivered in a cube format). A second example was a 3D stereoscopic TV which only required a stereo-pair data set to display properly (a viewer position must be specified, however). An ordinary 2D display also showed the same content from the ITMF. Finally, a horizontal-only parallax multi-view image was shown on a Looking Glass Factory display. Here, 42 views of a car were captured and sent to the display to offer a look around capability.
While playback on a true light field display was not demonstrated at the event, it has been demonstrated at the Light Filed Lab facility on their LF display.
The new set of specifications is over 200 pages and I have not had a chance to review them, so there is a lot of richness to the ITMF.
The Immersive Digital Experiences Alliance (IDEA) is a non-profit industry alliance working towards developing a family of royalty-free technical specifications that define interoperable interfaces and exchange formats to support the end-to-end conveyance of immersive volumetric and/or light field media. Created by CableLabs, Charter Communications, Light Field Lab Inc, OTOY, and Visby, IDEA is sure to gain new members as a result of its recent success. (CC)
