The topic of codecs keeps coming up in the video industry. Not only are new formats like UltraHD and VR demanding ever-higher levels of efficiency, but booming levels of internet video herald a surge in bandwidth consumption.
We thought it would be a good idea to summarise the state of play codec developments, focusing on the technologies used for mass content distribution rather than for capture, production and contribution.
AVC, more commonly known as H.264, is the most common video codec in use today. It is a motion-compensation-based compression standard, developed by MPEG and VCEG. Patents are licensed by MPEG LA, a patent pool unrelated to the International Standards Committee/International Electrotechnical Commission (ISO/IEC)-affiliated MPEG. As well as formats including Adobe Flash and HD Blu-ray discs, H.264 is also used for terrestrial (DVB-T/T2, ATSC), cable (DVB-C) and satellite (DVB-S/S2) broadcasts.
Source: IEEE. Click to enlarge.However, H.264 cannot supply the necessary levels of efficiency for wide-scale UltraHD broadcasts, which take four times the bandwidth of a FullHD stream: 12Gbps compared to 3Gbps, uncompressed. HDR adds between 20% and 40% to that.
The successor to H.264 is H.265/HEVC, which roughly halves the bitrate of a stream – just as H.264 did, coming from MPEG-2. However, competitors continue to spring up, each with their own claims and advantages. So far, none has achieved the reach of HEVC, but a division among patent owners could upset the status quo.
As with H.264, HEVC is part of the MPEG LA patent pool, which handles licensing. However, some patent owners are dissatisfied with the low fees being charged, and last year split off to form HEVC Advance – a new pool. We commented at the time (HEVC Advance Offering Essential HEVC Patents in Competition to MPEG LA) that the move had the potential to cause significant disruption in adoption. The split has led to confusion in the industry, with customers unsure where to license the patents that they need, or even the need to license from both pools.
This has opened the way for new solutions to spring up and try to take a share of the market. See the end of the article for a link to our previous coverage of these codecs.
Perseus
We have been following the Perseus codec, from UK-based V-Nova, since last April. This is a hardware-based approach, but uses the same mathematical transforms as MPEG – meaning that existing chips require only ‘tweaks’ to run the platform. This has significant advantages for both broadcasters and viewers – neither of whom have to upgrade their existing hardware.
Partners are one of the main draws of Perseus. 36 companies have so far integrated the technology into their solutions or supported the concept, including Nvidia, Nokia, Thomson, Hitachi, Intel, Sky and the European Broadcasting Union – an impressive list.
V-Nova says that Perseus offers bitrate savings of between two and three times that of HEVC: it can encode UltraHD streams at 7-8Mbps. NTT Data has introduced the technology in one of its first commercial rollouts: on-demand and live encoding solutions used by Sky Italia, among others. Additionally, Germany’s Institut für Rundfunktechnik recently validated Perseus’ lossless compression of UltraHD at 300Mbps, when running in B2B contribution applications on CPUs or GPUs.
Another advantage is that, thanks to its shared background, Perseus will default to legacy MPEG codecs if a device does not feature compatible chips. This means that there is no requirement to encode a stream multiple times for all device types.
Google’s VP Codecs
VP9 and VP10 are perhaps the other big challenger to HEVC. They are royalty-free and VP9 is already used for all UltraHD content on YouTube, as well as being embedded into the Chrome, Firefox and Opera browsers.
However, this has not sped adoption as much as Google would like; it is difficult to convince customers to move from a global standard like HEVC to a proprietary one like VP9. Still, the codec does have support from both chip makers (Broadcom, MediaTek, Nvidia, ST Micro and Sigma Systems) and device manufacturers (Samsung’s Galaxy S6, Sony’s Playstation and many smart TVs use the system). Apple and Microsoft, however, have stayed away.
While VP9 is comparable in performance to HEVC, Google says that it’s next codec – VP10 – will deliver bitrates of around 10Mbps for UltraHD content. (We have heard the comment that some are nervous about VP9 and VP10, because even though they are royalty-free, they worry that after wide implementation, a patent holder might emerge – BR)
Alliance for Open Media
In September, several companies – including Google – formed a new group called the Alliance for Open Media. The plan to develop codecs, formats and technologies that are interoperable, scalable and web-optimised.
Other companies in the AoM include Microsoft, which has experience with the VC-1 codec; Cisco, with its work on the royalty-free Thor; Mozilla, which developed Daala for Firefox; and Intel.
Although this is a significant grouping of knowledge, nothing has yet come from the AoM. Cisco and Mozilla are both members of the Internet Engineering Task Force, so it is possible that there could be some collaboration on the NetVC project – although there is little information about that, either.
RealMediaHD
A codec that we have not covered in great detail in the past, RMHD, from RealNetworks, was introduced in China last year as a software version. It has been integrated into media players from streaming vendors and OEMs. The firm claims that RMHD provides ‘significant’ cost savings compared to HEVC, due to lower CPU use and battery consumption. The codec is also backwards-compatible with older RealMedia content.
A hardware-optimised version of RMHD is available, and RealMedia says that it can be ‘easily’ implemented onto chipsets supporting the codec’s architecture, avoiding the need for new chip-making cycles (this is also an advantage for HEVC, which is built using the MPEG architecture).
RealNetworks says that its royalty fees are straight-forward, without the confusion surrounding HEVC.
Although little is known about the RMHD codec, it has a strong base to build on in its domestic market.
Tveon
The least-well-known codec (we have only covered it once) is Tveon, from Tveon Systems in Canada. The firm has made big claims (2Mbps for UltraHD and sub-200Kbps for ‘true’ FullHD), but has nothing to support them yet, like V-Nova’s impressive backer list. We assume that it is software-based – the most common implementation of a new codec – but there are many unanswered questions.
Thor
An honourable mention should be made of Thor, a project from Cisco that will eventually become a codec. Development came about specifically because of the murky situation surrounding HEVC. However, a final codec is likely years away.
JPEG 2000 & VC-2
Professional video production tend to use different codecs like JPEG2000 and VC-2, but in the world of content creation and contribution, there are many other codecs.
VC-2 is a standardised (by SMPTE) version of the Dirac codec that was developed by the BBC, and is typically used in contribution systems. At the recent ISE show, Barco Silex was showing a video-over-ip board based on this codec. VC-2 uses wavelet compression.
The Future
HEVC is still being limited by its patent situation, despite HEVC Advance revising its terms last last year (HEVC Advance Lowers Royalty Rates). Technicolor, a founding member, even pulled out of the group in February (Technicolor Abandons HEVC Advance), and is offering patents on its own terms.
Despite this, HEVC’s adoption is hard to argue with. Many TVs, tablets and PCs today, and even some of the newest smartphones, possess chips that can decode HEVC content. The smartphone market should not be overlooked in the codec wars: it is a large one, with fast replacement cycles. Apple’s sixth-gen iPhones, Android 5.0 and mobile chipmakers including Intel, Qualcomm, Samsung and Mediatek all support the technology.
While bandwidth efficiency is the main driver of HEVC on smartphones, UltraHD is the main driver in TVs. However, it will likely be some time before the technology becomes truly widespread. Frost & Sullivan, in a recent research report, estimated that it will take close to eight years before HEVC plays “a dominant role” in video.
That gives a long time for another platform to ‘pull the rug out’ from underneath HEVC, especially with the proliferation of royalty-free approaches. While the industry has been holding on to H.264 for a long time – like LCD technology, it is still being updated to match new advances – eventually new codecs will be required. If you take anything away from this article, it should be this point, first made by telecom industry veteran Fred Dawson: do not automatically assume that because H.264 was the industry standard, the same will apply to its successor. Chipset advances mean that even low-power devices, in the future, will be able to support many, or any, software codec on a generic chipset, rather than a purpose-built one. We could see an ecosystem where there is no standard, but codec use is based on personal choice.
- Perseus: http://tinyurl.com/gksp2l5
- Google’s VP codecs: http://tinyurl.com/gto33oz
- Alliance for Open Media: http://tinyurl.com/zgwdo59
- Tveon: http://tinyurl.com/zo9qcaz
- Thor: http://tinyurl.com/jsmf77j
