There was not much content or discussion of high frame rate at the SMPTE fall conference this year, but there was a lot of paper on HDR and related issues. However, a talk by Jim DeFlippis of TMS Consulting and Baylor University did discuss High Frame Rate (HFR). The talk started by noting the challenges that exist with HFR capture including sensitivity loss/more light required; shorter exposure times and increased strobing problems; and how to integrate with lower frame rates.
To address this, TMS and Baylor developed a series of experiments capturing different type of action. Footage showing the results of a baseball game were shown using a 120 fps capture with different shutter angles and camera motions. The team then worked with several partners (IsoVideo, InSync and Re:Vision Effects) to convert the 120 fps sequences to lower frame rates (24, 25, 30, 50, 60 and 100). These conversions included simple frame rate conversion and motion compensated conversion. Sequences were the compared to the reference 60 fps 180 degree shutter angle content.
The conclusions were that motion compensated sequences can effectively mimic native camera blur; 180 degree shutter angles seem to be the best regardless of frame rate; and simple drop/blend conversions don’t always provide a satisfactory result. This led to additional testing.
In Phase 2, the team focused on 120 fps capture for 24 fps finishing. They investigated the effect of shutter angle and camera movement, conversion techniques with a moving car, dancing and green screen.
What did they conclude? Outdoor lighting levels are not a concern in high frame rate or shutter angles, but indoor lighting is critical at high frame rates and shorter shutters, as might be expected. Green screen lighting was also a challenge with issues in being able to balance the color temperature. HFR green screen sequences have less blur but still have problems with chroma bleed (distortion of color mostly at edges).
The issues with the green screen led to the evaluation of two workflows to address the deficiencies, the results of which were then presented, but more work is needed in this area.
Additional frame rate conversion work was done as well and presented. In the future, the team hopes to look at 240 and 300 fps capture, variable frame rate capture and the impact of HDR and HFR.
A second paper on HFR focused on the compression aspects. Here, Damian Ruiz Coll of the Universitat Politecnica de Valencia in Spain suggested that the frame order for 120 fps compression needs to be different than with 60 fps content.
HEVC encoding uses a Group of Pictures (GOP) feature that looks at frames ahead and behind the current frame to analyze motion and find ways to better compress the images. Coll says that default GOP encoding order for 120 fps content in the HEVC reference model may work OK for playback at 120 fps, but for playback at 60 fps, the decoder will suffer an overflow/underflow condition, which can cause picture disruption.
The solution is to change the way that GOP frames are configured. The default and new GOP order are shown below.
Coll says they did some tests to see what the bit rate impact of going to a two-layer scalable 120 fps solution might be. According to their results, the impact is about a 20% bandwidth increase over the 60 fps base layer to maintain the same quality as measured by PSNR. The two-layer approach is also backward compatible with legacy UHD TVs and set top boxes.
