The Fraunhofers had its usual booth in Hall 8, with lots of stations showing different technologies. Most of them were looking at audio developments as the Institute is very strong there, with MPEG-H being widely adopted for next generation audio. One of the stations was showing the gaze-directed tiled video rendering initiative that is being looked at by MPEG to optimise the delivery of video content for VR, and which we have previously reported on. We looked in more detail at a couple of video related areas.
First, we looked at the latest developments in Fraunhofer’s development of light field capture and processing. The big news at IBC was a demonstration of light field motion video. The group used an array of 4 x 2 cameras that capture slightly different views of a video. In the past the Fraunhofer has been showing objects rather than video. The multiple captures are then synthesised into a single video stream which can be modified to reflect the direction of view of the user. We verified that moving the viewpoint changed the parallax between the two figures shown in our image.
Fraunhofer is extending its light field technology to video. Image:Meko. Click for higher resolution
The light field video, which is captured at 2160P30 is taken directly to the Unreal 3D engine, and staff confirmed that the video is used in the engine with video rendered as a texture in the 3D environment. At the moment, it cannot be done in real time, but there is work being done to try to do that.
Next, we moved on to look at developments in codecs. Last year, JPEG asked for proposals for a standard light mezzanine codec to be called JPEG XS. We have reported on the Lici codec that had been developed by Fraunhofer and the group decided to get together with Intopix, which has the Tico codec, and the two put in a joint proposal. There were six proposals for the codec and the joint proposal was adopted.
Features of the codec are
- High quality – visually lossless at 2:1 to 6:1 compression
- Ultra low latency – a frame can be encoded and decoded in 32 lines of video
- Very low complexity – can be implemented in ASICs or FPGAs
- The codec is highly suitable for parallel processing and can be run on a CPU or GPU (up to 4K60p on an Intel Core i7 processor)
- Resolutions from VGA to 8K
- Frame rates of 24 to 120fps
- Sub-sampling at 4:2:2 or 4:4:4
- RGB and YCbCr/YUV
- 8 – 16 bits per component
One of the features of the codec is that it can support both natural images and graphics, so is very suitable for the modern developments with the integration of data with images.
Another feature is that the codec can have a maximum bit rate limit to allow for adaptive compression. The codec has also been tested to allow for multiple passes and still performs well after 10 processes.
In terms of timings, the committee draft is due to be complete in October, with a draft being issued in January 2018. Finalisation should be around July 2018 although the Fraunhofer hopes to show a live workflow at NAB in 2018.
As can be seen, at higher bit rates, HEVC starts to do better than JPEG XS, but, of course, with much more latency and complexity. Click for higher resolution
