Source-Based Tone Mapping (SBTM) is a new technology developed by the HDMI Forum to solve a specific problem. The problem is an inconsistent viewing experience when the source device (set top box, gaming console, PC, etc.) has some combination of HDR, SDR, dynamic HDR and graphics overlays on the same screen simultaneously. The result can be an image that just doesn’t look right with overly bright or dark areas and distorted colors.
One key example that HDMI mentions is a menu of video thumbnails from a streaming service provider. Some of the thumbnails might be HDR and others might be SDR, and the menu is rendered using graphics. Another example is a game with a mixture of HDR game action, SDR video and graphics overlays.
While HDR displays can do their own tone mapping internally to adjust the incoming luminance and color ranges to the luminance and color capabilities of the display device, they cannot properly perform tone mapping when presented with such a mixed format image. Such optimization must be done by the source, which is why SBTM is needed.
SBTM does not replace existing HDR technologies like HDR10 or HLG, it works in conjunction with these technologies.
According to a written response from the HDMI Forum, SBTM can help in the mixed situation as well as in the HDR-on-HDR case. In those cases, the Source must perform the compositing operation in an HDR color volume that it selects. Metadata can’t help in these situations because the Source does not have, and cannot generate, metadata for composite pictures. Without SBTM, the Source’s selected color volume might not be optimal because it doesn’t represent the display’s characteristics, or if it does, the display might still reserve some of its capability to render signals that go beyond its capability.
Can you confirm the step-by-step process?
1. Source device reads TV EDID data
The EDID contains a color volume specification that the Sink is expected to be able to accurately render.
2. Source evaluates current content to determine class of content (SDR, HDR, dynamic HDR, graphic overlay, etc.). How does it do this?
Streams are coded as HDR or SDR. Static or dynamic HDR metadata (if present) is also encoded within a content stream. Graphic overlays can be encoded in different ways, but the dynamic range is generally known to the application.
3. Based on class of content, it sets a flag in the HDMI stream to signal the TV about the type of content.
Yes, for each frame, the Source signals to the Sink device that the Source is tone mapping the content to the Sink’s advertised color volume (i.e., the one specified in the EDID).
4. Based on the class of content and the capabilities of the TV, tone mapping is performed by the source device.
Correct. In SBTM mode, the TV is expecting to receive HDR signals in a color volume of its own choosing. The Source utilizes the metadata to perform the tone mapping operation BEFORE the compositing operation takes place, and this tone mapping can be from an SDR, HDR, HDR10, or dynamic HDR content source. The color volume the Source uses for compositing is specified by the TV, and the TV knows not to expect signals outside of that color volume. Note that no additional static or dynamic metadata is transmitted to the TV when SBTM is used, since the metadata was already used for the tone mapping that occurred in the Source.
I would think TV/display makers and/or content creators would have concerns about a source device altering the video as they think their TVs will do a better job. Would filmmaker mode override SBTM?
Tone mapping performance could vary from Source to Source; however, a Source product’s performance can be easily compared to another by simply providing the same EDID to two Sources and comparing how they look (there’s a controlled way of doing this defined inside the spec). Content providers, display makers, or industry specifications (such as Dolby Vision or HDR10+) could eventually impose their own additional constraints on how content is tone mapped. UHDA defines “Filmmaker Mode”, so it would be up to them to specify any behavioral constraints that are needed. Even though the Source tone maps to a color volume of the display’s choosing, in some situations the display might perform its own additional tone mapping due to ambient light levels, power constraints, etc.
I understand SBTM can be implemented in HDMI 2.1 interfaces by a firmware update, but doesn’t tone mapping require processing and memory which are not part of the HDMI chip set?
Source devices in the market today generally need to perform tone mapping to a color volume of their choosing when they support mixed SDR/HDR content. Sometimes this color volume can be adjusted via a user control or slider. If the chipset allows this color volume to be sufficiently programmable, a Source device might be able to be firmware upgraded to read the EDID and program that color volume appropriately. Similarly, a TV or display might simply need to add a small number of EDID bytes and adjust its internal tone mapping when the Extended Metadata Packet is received so as not to expect any colors outside the EDID-specified color volume.
How is SBTM useful in gaming?
SBTM also helps enable “plug and play” HDR gaming without the need for calibration screens. Because the gaming device knows the TV’s preferred color volume, the gaming engine can perform appropriate tone mapping to ensure that every gamer is able to see all of the important graphical elements. In addition, none of the TV’s capability is wasted, since none of its dynamic range needs to be reserved for pixel values outside the specified color volume.
SBTM will be available with the release of HDMI 2.1a in Q1 2022. (CC)
