Established in 1988, the Moving Picture Experts Group (MPEG) has produced some 180 standards in the area of efficient distribution on broadcast, broadband, mobile and physica medial, and consumption of digital moving pictures and audio, both conventional and immersive, using analysis, compression and transport technologies. It has also produced 5 standards for efficient storage, processing and delivery of genomic data.
With its membership of ~1500 registered and ~600 attending experts, MPEG has produced more standards than any other JTC 1 subcommittee. The standards enable a manufacturing and service business worth more than 1.5 T$ p.a.
We expect that in the future industry will require more MPEG standards for the following areas
- Rectangular video coding. As ~80% of world communication traffic is video-based, new more efficient video coding standards will continue to be of high interest. These will likely incorporate new video compression technologies, driven by extensive research both in algorithm and hardware domain, e.g. in the area of Machine Learning. The market will also demand standards with higher quality user experience such as high-fidelity near-live video coding.
- Audio coding. As for video, future audio coding standards could be in the wake of the MPEG Audio coding family, be Machine Learning based or a be mix of the two. High fidelity near live audio coding standards will also be required to provide new forms of user experience.
- Immersive media will require coding standards that allow a user to move unrestricted in a 3D space, for Point Clouds for human and machine use, for realistic meshes and textures and for light fields. Immersive audio coding standards providing an even more intensely satisfying user experience than the MPEG-I Audio standard currently being developed will also be required.
- 3D scene representation standards will be needed to recreate realistic 3D scenes populated by media objects encoded with other MPEG immersive media coding standards. It is expected that standards for effective human interaction with media will also be required.
- Quality of experience will likely differentiate the market of products and services through the use of sophisticated technologies that reduce the amount of information needed to transmit future media types. MPEG must support the manufacturing and service industries by offering standards with features beyond those of the currently available standards, such as machine analytics for quality measure.
- Video Coding for Machines (VCM) is a new area of endeavour primarily designed to serve remotely located machines and, potentially, human users as well. Audio Coding for Machines will likely be a necessary complement to VCM. Machine vision standards, e.g. for object classification, will continue the tradition of CDVS (image search) and CDVA (video analysis) standards and will mostly be based on similar technologies as VCM. Internet of Media Things standards will also serve as a framework for the above standards.
- Application formats will be required by industry to facilitate the use of the wide range of available and future MPEG basic media coding standards by configuring MPEG technologies to suit application needs.
- Transport of compressed media standards have always been enablers of the use of MPEG media coding standards and we expect the need for them will continue. Ways to cope with emerging constraints on video and audio delivery over internet, such as those witnessed during the Covid-19 emergency, should be considered. New networks, e.g. 5G and other future networks, will likely drive the requests and MPEG should actively work together with the relevant bodies.
- Compression of genomic data, initiated 5 years ago in collaboration with TC 276, has shown that MPEG technologies could be successfully applied to new non-media areas. Standards will continue to be produced to respond to this new client industry.
- MPEG has realised that Compression of other data is a broad new area that can benefit from its approach to standardisation in support to new industries such as Medical, Industry 4.0, Automotive etc. As MPEG does not obviously have domain specific competence, it plans to identify the needs and develop standards for these new areas jointly with the relevant committees.
The above is an ambitious program of work in an expanding area of standardisation that heavily depends on sophisticated and evolving technologies. MPEG is confident it will be able to execute this program because it has learnt how to foster and actually fostered innovation with its standard and because, since its early days, it has innovated the way a standards committee operates, and developed a set of procedures and operations that MPEG calls its “modus operandi”.
Below we will examine how MPEG’s “modus operandi” will be exercised and adapted to the new challenges of the work program.
No one controls MPEG. The key differentiating element is that no entity or industry “controls” MPEG. At all levels, proposals follow a bottom-up path and are accepted, not because of their origin but because of their value, by a mostly flat internally dynamic organisation that fosters the constructive flow and interaction of ideas. The management monitors the activities of the different organisational units, identifies points of convergence, interaction or contention, and creates opportunities to resolve issues. This element of the modus operandi must be preserved.
Connection with industries. The success of MPEG standards is largely based on the establishment of efficient connections with the client industries. MPEG will continue to interact with them providing solutions to their customers’ needs. However, we expect that formal liaisons may soon become insufficient because MPEG standards heavily depend on rapidly evolving technologies, such as capture and presentation, which have a strong impact on its standards but have a scope outside of the MPEG purview. MPEG will need to identify and activate relationships with the industries who are developing the new media technologies that will drive the development of its new standards. At the same time MPEG should enhance its relationships with the content industry that provides the ultimate user experience using MPEG standards, again using a network with varying capabilities to address devices that could have graded tiers of computational complexity.
Connection with academia. Another reason for the success of MPEG standards is MPEG’s strong links with academia and research organisations, an area that accounts for ~¼ of its total membership and contributes to sustaining the vitality of its standards activities. MPEG has always conducted exploration on promising new technologies, before standardisation can begin. However, the number of directions that require explorations are growing fast and MPEG resources may not be sufficient in the future to assess the potential of all candidate technologies. MPEG will need to establish more organic links with academia and create more opportunities to stimulate, get input from and possibly collaborate with academia.
Technology-driven standards meeting market needs. MPEG’s modus operandi has been inspired by applications as supported by technology and we think that in the future this must continue. However, it is imperative that this be complemented by a form of internal competition with new MPEG members who are market-driven. The goal is to strike the right balance between technology-push and market-pull, i.e. to be able to identify targets that are both technology-driven and attractive to the market. MPEG should create a market needs group who dialectically interacts with the requirements group to fortify proposals for new work, keeping timely delivery of its standards as a high priority issue. MPEG should better assert its active presence on all aspects of the standardisation workflow: industry, market, research and academia.
Standards with and for other committees. MPEG’s modus operandi has allowed MPEG to manage the development of technology-heavy standards, some of which have involved hundreds of person-years. Most standards have been developed with internal resources. However, other standards have been developed jointly with entities outside MPEG at different level of integration: e.g. with ITU-T SG 16, with SC 24 for augmented reality, with ISO TC 276 for genomics etc. Several MPEG standard extensions have been developed in response to the need of other committees (e.g. JPEG, 3GPP, SCTE etc.). This fluid interaction must be enhanced by broadening MPEG’s operation, as media technologies interact more and more with other technologies in products and services and use of compression spreads to more domains.
Plans for future common standards. We envisage the need for MPEG to continue to develop common standards with other committees, in particular with ITU-T SG 16 on video coding and with TC 276 Biotechnologies on genomic data compression. Augmented Reality will provide opportunities to develop common standards with JTC 1/SC 24. We also see more work with other committees as an essential component of MPEG future. Examples are JTC 1/SC 41 on a new phase of Internet of Media Things (IoMT), JTC 1/SC 42 on Machine Learning for compression and Big Data, starting from MPEG’s Network-Based Media Processing (NBMP) standard, and TC 215 Health Informatics on data compression. In a not-so-distant future MPEG should seek new standardisation opportunities in the area of Industry 4.0 and automotive. It is an ambitious program that can be implemented using MPEG experience and representation.
Standard development models. MPEG has learned that effective standardisation can be achieved if experts work shoulder-to-shoulder. How can this be achieved with an expanded scope of work and increased number of collaboration parties? One answer lies in the experience MPEG has gathered from its long history of online collaborations and the fully online 130th meeting attended by ~600 experts. In normal conditions, a large part of MPEG meetings will continue to be held face-to-face. However, many standards can and should be developed jointly with other organisations mostly via online meetings. These can be easily integrated with the rest of the MPEG workflow because they deal with specific topics.
Effectiveness of MPEG role. MPEG will need to develop new metrics, beyond the number of standards produced (MPEG is already the largest producer of standards in JTC 1) and apply them in a regular fashion to revise the effectiveness of its role. While MPEG standards continue to have loyal customers, their importance in other domains is less significant. The past is often so sticky that MPEG must make sure that its past successes do not make it an “also-ran” in the future. This does not mean that MPEG must renege its practices, but that it must adapt its business model to the new challenges. MPEG needs to reassert its mission to develop the best standards satisfying agreed requirements, thus continuing to provide opportunities to remunerate good IP for good standards. At the same time, MPEG should develop a strategy to handle the growing competition to its standards. One way is by enhancing the role of reference software, as an equivalent normative version of MPEG standards. Another is to envisage conditions that may facilitate expeditious development of licensing terms by the market.
Role of ICT tools in standardisation. Many aspects of the MPEG modus operandi heavily rely on ICT support. In 1995 MPEG was probably the first committee of its size and structure to adopt electronic document upload and distribution. In the last 15 years MPEG had the privilege to benefit from the support of Institut Mines Telecom and especially Christian Tulvan who has continuously designed new solutions that allow MPEG to be a fluid and flat organisation where every member can contribute by knowing what happens where. MPEG expects to be able to rely on their unique support to implement more features that will enhance the quality and quantity of its standards through enhanced and qualified participation. (LC)
This article was created by Leonardo Chiariglione, MPEG chairman and CEO at Cedeo.net and is republished with kind permission.
