That was the topic of discussion I had with Ben Gervais, the technical supervisor on “Billy Lynn’s Long Halftime Walk”, a film directed by Ang Lee and captured in 3D, 120 frames per second and 4K resolution. But the film will only be shown in this format on a few select screens, so for more mainstream distribution, derivative versions are needed. The question is: Can you maintain a similar emotional impact when showing the film in 2D, 2K and lower frame rates?
We started the conversation by first asking where the movie will be screened in 4K/120/3D. In North America, only two theaters have been set up: one in New York and one in Hollywood (Cinerama theater – Want to See Billy Lynn at 120Hz in 3D and UHD?) but these are temporary installations that will only play for the run of the movie. Besides these, the film will only be shown in 2K/24/2D in the rest of North America.
In China, Taiwan, Hong Kong and Macao, the 4K/120/3D version will be shown as well as 2K/24/3D, 2K/60/3D and 2K/120/3D/HDR (Dolby Cinema) versions. Versions with Atmos and 5.1 are also supported. The first European release is not slated until next year and any decision on formats and other regions appears to have not been made yet.
Gervais said that he and Ang Lee first looked at high frame demos from Cameron, the Pickford Institute and Doug Trumbull. “Ang likes to look at faces when evaluating images to see if they convey the right emotion,” explained Gervais. “Sometimes motion blur and judder can be part of the emotion, and sometimes not. However, we did notice that deriving a 24 fps version from a 60 fps 3D master creates strange artifacts with current optical flow and frame interpolation techniques, which need to be painstakingly corrected in post production. We also saw some of the new synthetic shutter tools being developed by Tony Davis at RealD and that this would be a big help in creating derivatives if we captured at 120 fps, so this is the way we decided to go.”
The synthetic shutter idea leverages signal process techniques to create nearly any shutter angle to reduce judder and other temporal artifacts and is especially helpful when going from 120 fps to 60 or 24 fps derivatives. By capturing content at 120 fps and a 360-degree shutter angle, custom, non-square shutter angle shapes and frame rates versions can be derived (see more detailed article: RealD Develops Synthetic Shutter Solution for HFR).
Davis first developed a custom shutter angle shape he called Magnesium, which was initially used on the dailies for the film. “We noticed a bit of a strange effect however,” explained Gervais. When we processed and ran the dailies at 24 fps, when an actor stood up rapidly from a chair, he started to blur before he moved as the Magnesium shutter shape was looking at frames before and after the projected frame. Tony modified the synthetic shutter shape a bit to lop off the front end allowing a little more sharpness and judder on the front end but retaining the softness on the backend. This new shutter was named Carbon and it fixed the problem, becoming our “hero” frame, meaning this was used for most of the shots in the movie.”
RealD’s Magnesium synthetic shutter. Each bar on the left side represents incoming 120 fps frame and the height is the weighting applied. The vertical dashed lines are the 24 fps reference. the modulation transfer function(MTF) on the right plots the frequency response of this custom synthetic shutter
RealD’s Carbon synthetic shutter is more asymmetrical and was used on most of the Billy Lynn frames.
Gervais also said that they experimented with shots that allowed variable motion blur within the frame. This might allow a face to be windowed in post production and run at a higher or lower frame rate than the background for an artistic effect. Such techniques never made it into the final movie, but the 120 fps version does have some 60 fps sequences that were used because Lee felt they created a little more emotional distance in those scenes.
Besides the use of the synthetic shutter in the derivative versions, there were some adjustments of the lighting and contrast of some scenes in the lower frame rate versions. Gervais explained that this was necessary because the lighting was flatter than usual for the 120 fps capture, so needed so adjustments for lower frame rate versions. Color correction was also needed on version with lower or higher peak luminance levels as well.
Editing the film designed to play at 120 fps was a challenge as well as Avid can only edit at up to 60 fps and audio was captured at 30 fps. “You had to be careful where you cut a scene,” said Gervais. Conforming the movie from all the assets was a massive job as well with no tools ready to do this at 120 fps.
Gervais said he has spent the last two years working on the film and is now about to start his next project with Lee – a boxing film that will again be shot at 120 fps. But there will be additional challenges here as they need a very light weight, beamsplitter-type handheld 3D rig for the boxing shots in the ring. They will shoot about twice as many days as “Billy Lynn” and record uncompressed 4K data, which means about 12X more data compared to the “Billy Lynn” shoot – they estimate 48 TB per day! And, there will be a lot more CGI work, so they have a whole R&D project initiated on this aspect as well. Production is likely to start in late summer 2017.
So will these derivative versions of Billy Lynn capture more of the emotional look and feel of the 120 fps version compared to conventional 60 or 24 fps capture? Gervais says yes, we will have to see if these techniques become adopted by other film makers going forward. – Chris Chinnock
For comments on the IBC showing of the movie in 120fps see Ang Lee Shows the Future of Cinema Display (Updated)
