I have long thought that HDR and higher resolution images alone or together makes the effect of low frame rates more obviously, so I was pleased to see a paper in the SMPTE Motion Imaging Journal in August 20211.
I am one of those that quite likes HFR content. I was lucky enough to see a very hiqh quality presentation, on non-DCI Christie projectors at IBC, of “Billy Lynn’s Long Halftime Walk” by Ang Lee in 4K/120Hz. (see my Display Daily Ang Lee Shows the Future of Cinema Display (Updated). I was impressed, although the viewer sitting next to me didn’t like it as it “wasn’t cinematic”. He preferred the look of 24Hz. (For a look at what makes a ‘film look’ there is an interesting BBC R&D white paper, referenced in the SMPTE paper, that is free to download here that looks at the issue.)
The paper, produced by researchers at Dolby Labs, covered an experiment that looked at the comparative smoothness of different kinds of motion. such as panning, action and slow movement at different frame rates (24, 30 and 40fps) and with different luminance levels (48 + 300 cd/m²). Cinematographers have an established set of best practices or a ‘grammar’ of camera and subject movements that optimise the portrayal of motion at 24fps. However, HDR seemed to make a lack of smooth motion more obvious and cause flicker or judder. Some have got around this by limiting the palette of colours used or slowing the motion and panning rates. That’s a shame.
The paper looks in detail at existing research that suggests that there is a clear relationship that higher luminance needs faster frame rates if you want to avoid unacceptable judder. However, much of the work was done on a limited range of scenarios or with test patterns that might not have been representative of cinema content.
14 scenes that were representative of a range of luminance and motion were selected.
From a starting database of 52 short scenes, that were captured and graded for HDR at 120 fps and with a 360deg shutter angle, the group selected 14 as representing a full range of different motion and mean luminance levels. A range of different frame rates were tested in a pilot experiment and 60fps was excluded as it ‘did not yield additional information’ over the 40fps version, leaving 24, 30 and 40 fps. Each of the 14 clips was mastered at 4,000cd/m2 peak luminance and the colour volume reduced to a 48 cd/m2 (chosen as typical for cinema) and a 300 cd/m2 target (matching an available cinema projector). That kept the dynamic range of the LG C8, C9 and CX displays used for testing.
16 subjects were tested (13 in the US and 3 in Europe). There were 42 trials – at the three frame rates for each sample. Participants were shown clips and asked to rate them for smoothness. The results showed that for some scenes, there was little change in smoothness at different frame rates, and that peak luminance alone was not enough to indicate likely issues. Mean luminance was a better indicator, but motion was also a factor and simply applying a 40fps rate everywhere was likely to lead to an effect that was ‘too smooth’.
The researchers then classified the camera and object motion for each of the scenes. That led to the observation that object motion smoothness did not seem to be related to luminance, while camera motion was related. To match the perception of film, bright scenes with high camera motion and any range of object motion may benefit from higher rates. Getting a cinematic look to match film with HDR content really requires dynamic frame rates. If choosing a fixed rate,
“the perceptual smoothness difference between 24/30/40 is much greater when the content is fast and high luminance”.
Further, the mean luminance has three times the effect of camera motion on the appearance of smoothness.
The group plans to do further work in a cinema environment and expanding on the study.
Thanks to SMPTE for making the paper available for this article.
Analyst Comment
Choosing dynamic frame rates between 24 and 40fps for different content seems not too tricky an option from a technology point of view, although this author is not clear whether there are any tricky practical issues in terms of the DCI regulations, which can be restrictive (ask those in LED cinema!). In capturing, it shouldn’t be difficult to always capture at 40fps and then downgrade the frame rate in post-production. For more on this topic, see Chris’s articles RealD Develops Synthetic Shutter Solution for HFR or High Frame Rate Mastering War Stories. Of course, if movie makers start using variable frame rates, TV set makers will have to work out how to map the frame rate successfully onto panels. 120Hz panels will be in a good position to support 24 fps and also 40fps.
I noted that the group didn’t look at different resolutions directly. My feeling, from demos, is that 8K, in particular, needs higher frame rates to avoid judder and provide smooth motion.
1 J. Pytlarz, A. Choudhury and R. Atkins, “Achieving Cinematic Motion With High-Dynamic Range,” in SMPTE Motion Imaging Journal, vol. 130, no. 7, pp. 42-50, Aug. 2021, doi: 10.5594/JMI.2021.3083901.
