Professor Philip Willis at the University of Bath in the UK and his colleagues published an article in the March 2015 issue of the SMPTE Motion Imaging Journal titled “Taking the Pixel Out of the Picture”. This was based, in part, on an earlier 2012 article in the Journal of Virtual Reality and Broadcasting (JVRB).
Currently, virtually all images you are likely to see in the movies, on television, on the Internet, in newspapers or in magazines were, at some point in their lives, pixelated to a rectangular grid. For motion imaging the size of these grids vary from sub-VGA (640 x 480 or less) to 8K (7680 x 4320) or more. In addition to the vast number of “standard” pixel counts and formats, there are many images and displays with non-standard counts. For example, the image with this article is rendered at a resolution of 500 x 382 pixels and is almost certainly displayed at a different resolution than that on your display. I have no idea at what resolution the image was originally generated. By storing not the individual pixel values but the contour lines of constant luminance and chrominance, images can be stored free of the limitations of an underlying grid. The JVRB version of the article goes into considerable mathematical detail on exactly how pixel values are converted to contours and is not for the faint of heart.
Color components and their contours from the JVRB Article
The short-term value of this technique is to allow rendering of the image from the contour values back to an image suitable for display on a pixelated display (i.e. any modern display) at any desired pixel resolution. One partner of the University of Bath, Smoke & Mirrors (London, UK), is a visual effects production company. The company routinely delivers 50 or more variants of a single short advertisement video. Some of these are based on cultural needs such as language but most of the variants are based on different resolution requirements. This company is interested in the technique because of the ease with which images can be rendered at different resolutions, once the master, contour-based file is generated. For a company like Smoke & Mirrors, the fact that the technique seems to work best with computer-generated imagery (CGI) rather than with natural photographs is not really a problem. After all, CGI imagery is typically based on 3D vector graphics, not a pixel structure. To render the 3D graphics as contours is probably no more difficult than it is to render it as pixels, and you would only need to do it once. Once you had the contour file, it would be simple to render it at any resolution needed by a customer.
This technology is currently in the research stage and is not ready for use by broadcasters. Another of the University of Bath’s partners on this project is Root6 (London UK), a company that supplies technology, software and services to post-production and special effects houses. One of the current focuses is how to fit this contour-based step into the pixel-based workflows currently used by Root6 and everyone else in the industry.
Will this render pixels obsolete? Probably not in your lifetime. First, displays are pixelated and I have not seen any indication, from the University of Bath, its partners or the display industry in general, of a turn away from pixelated displays to non-pixelated displays. Second, while certain operations can be done on images in contour format, most image processing is likely to remain as pixel processing. One thing that can obviously be represented in the contour format is high dynamic range (HDR) content – just add more contours at high and/or low light levels.
One possibility for the technology, as I see it, is that it will lead to more efficient compression schemes than the current DCT and block-based MPEG, AVC (H.264) and HEVC (H.265) schemes. With no blocks, there might be fewer visible compression artifacts. Current efforts at the University of Bath and its partners does not appear to be focusing on compression schemes, however.
Finally, the display industry is slow to change. I was looking through the SMPTE archives and the earliest mention of pixelated images was in an article in the October 1930 issue of the Journal of the Society of Motion Picture Engineers (SMPE). (They didn’t add Television to their name to become SMPTE until the 1950s.) The article was by C. Francis Jenkins, visionary, inventor and founder of SMPE and titled “Television Systems”. He didn’t call them pixels, he called them “Elementary Areas” but his meaning is clear. He’s the man who didn’t like the word “Television” and used the phrase “Radio Vision” till long after everyone else had switched.
It took 85 years, from 1930 to roughly now, for the continuous image (i.e. film) to vanish and be entirely replaced by pixels. So, to answer my question “Are Pixels Obsolete?” I’d say “Don’t hold your breath”. – Matthew Brennesholtz
