The Academy Color Encoding System (ACES) has long puzzled me. At the November 18th section meeting of the Society of Motion Picture and Television New York chapter, Jim Houston, co-chair of the ACES Project Committee, gave an explanation of ACES that finally made sense to me. He primarily discussed the recently released version 1.0 of ACES, not the pre-release version 0.1 of ACES that has been around for several years.
Jim Houston, ACES Co-Chair
The Academy in ACES refers to, of course, the Academy of Motion Picture Arts & Sciences. This is the organization that awards the Oscars every year for motion pictures. Given this background, it is not surprising that ACES is targeted at motion pictures and not television applications.
ACES 0.1, which I understood to be a color encoding system, didn’t make sense to me because there was already another color encoding system that seemed to be able to do everything that ACES could do. This was the Digital Cinema Initiative (DCI) encoding scheme that, like ACES, could encode all colors that were perceivable to the human eye with sufficient bit depth to eliminate contouring in the image. Why was a new system needed?
One of the key differences between ACES and DCI is that DCI uses 12-bit unsigned integer code values while ACES uses 16 bit floating point numbers. Since the DCI system stores the 12 bits in 16 bit integers with the lower 4 bits set to Zero, this does not mean larger ACES file sizes compared to DCI. Houston explained that the ACES committee considered using 32 bit floating point but decided there was no image quality advantage to justify the doubling of file sizes.
This encoding difference is important because the 12 bit integer system used by the DCI is tied to the brightness range of existing digital cinema projectors which have a maximum luminance at the screen of about 14 foot-lamberts or 48 cd/m². DCI encoding produces excellent images in existing theaters but has one key failing: it cannot encode high dynamic range (HDR) image content.
ACES, with its floating point encoding, can encode not only any color visible to the human eye, but any brightness over an extremely large range. Existing digital cinema cameras can record images with more dynamic range than can be encoded by a 12 bit integer encoding scheme. In order to preserve this extra information, cinema production and post production facilities use a wide variety of formats to maintain all of the information. This additional information is used both in the editing process used to generate current motion pictures but also can be archived to allow it to be used in the future to generate HDR versions of the motion picture.
The phrase “variety of formats” raises a big red flag to anyone in the digital community, especially when it is used in conjunction with “archive.” If you archive in one of many competitive formats, the content may be at worst lost or at least made difficult to access in the future. Can you read the Lotus 1-2-3 files you created 20 years ago? Can you even read the 3½ or 8” disks they are written on?
One of the original goals of ACES 0.1, as I understood it at the time, was to provide a common archive format for all motion pictures. ACES 1.0 goes beyond archiving, however, and provides a common format for all acquisition, production, post-production and archiving of motion pictures.
The basic ASES production system is shown in the image. In this system, the original image is captured by an image capture device such as a camera, a film scanner or a “virtual camera” for special effects. After acquisition, it is immediately converted to the ACES format using a Input Device Transform (IDT).
IDTs are available now for some digital cinema cameras and new ones can be created as needed. Once the content is in the ACES format, it can be edited by standard tools. When editing is complete, the complete motion picture can be saved still in ACES format. To display the content, the content is passed through a Reference Rendering Transform (RRT) and an Output Device Transform (ODT). The RRT converts the content to a standard format, for example the DCI cinema format. The ODT then converts the standard output to display correctly on the target display, e.g. a SMPTE or DCI reference projector. Note the RRT and the ODT would normally be in real-time and would not affect the ACES version of the content stored on the server.
Houston says the RRT and the ODT transforms as well as the 3D LUTs are designed to be implemented on relatively low-cost graphical processing units (GPUs) and this real-time transformation does not place a heavy burden on the equipment supplier. This real-time nature is especially important during the post-production process because it will allow editors to view the content as they edit it without the need to create intermediate files for display. It would also be important while the movie is being shot because it allows the easy production of digital dailies, which allow the director to see scenes shot that day and plan the next day’s work.
ACES decouples the image data from both the input camera data and from the output display format. ACES can provide content to a variety of display types with a variety of color and brightness gamuts and electro-optical transfer function (EOTF) requirements. These various EOTF requirements can be a part of the RRT, as shown in the image.
Houston also mentioned EOTFs for the PQ1000 and PQ2000 formats. These formats are intended for HDR content to be shown on displays with peak luminances of 1000 and 2000 cd/m² respectively.
Another type of transform is available in the ACES system, called the look modification transform (LMT). Directors often want a specific “look” for a movie, perhaps bright and sunny with pinkish skin tones for a romantic comedy or dark and gloomy for a horror movie. A director can create a LMT and supply it to the various post-production houses working on the movie to ensure a uniform look. The post-production house can use the LMT but can’t see inside it to determine exactly how it works to adjust the look of the content.
As an example of the use of ACES, current digital cinema cameras can acquire more data than is ultimately used in the digital cinema projector. This data provides an overhead buffer that allows edits such as exposure adjustments in post, to be invisible to the end user. The camera data is often stored in a RAW format. This isn’t a single format, every camera or camera product line has a different RAW format. The goal of a RAW format is to preserve all the data acquired by the camera image sensor without any internal processing by the camera. This allows necessary processing such as de-Bayering, to be done by the more powerful processors available in post compared to the processors in the camera itself. Without ACES, the RAW format would then be converted to one of the many available post-production digital formats. With ACES, the RAW format would be converted to the ACES format using an IDT, presumably supplied by the camera manufacturer. If a studio or post-production house is not satisfied by the IDT supplied by the camera manufacturer, it could develop its own. This ACES file is then available for further editing.
This information includes all the HDR or wide color gamut (WCG) information generated by the camera. If you have a high end digital cinema camera with 20 stops of exposure latitude, you preserve all 20 stops. If you use a GoPro camera (yes, they have been used in cinema production) it preserves all the information from the GoPro camera. The content generated by these two cameras can then be seamlessly edited together without further file conversion.
Since ACES is not defined by a specific set of display primaries, it can encode the color of any object, real or created by the special effects team. Houston gave several examples of real colors in the natural world that cannot be encoded by three-primary color systems such as Rec. 709 for HDTV. These colors included the blue of butterflies and the yellow of daisies. Most, but not all, of these colors can be encoded using Rec. 2020 primary colors. He also added that film contains colors that are outside the Rec. 2020 gamut and these can also be preserved.
In the Q&A session following his talk, Mr. Houston was asked about the use of ACES in television production. He said that the ACES committee focused on top quality content, i.e. motion pictures, not television. He added, a little more graciously, that the ACES system was largely developed by volunteers from the motion picture industry so it wasn’t surprising they focused on motion picture requirements. He said if a committee of volunteers from the television industry worked to adapt the ACES system to television production, the Academy would be glad to work with them.
For more information, see www.oscars.org/aces –Matthew Brennesholtz
Analyst Comment
For more on Aces, check out Chris’s video from NAB. https://vimeo.com/125162125
