Nvidia and SMI are working together on a foveated rendering system, which was shown off at Siggraph last month.
Foveated rendering is the practice of showing a very high-resolution image where a user is looking, with proportionally lower resolution towards the edges of the user’s vision. SMI has been working with foveated rendering since 2014; was the first to add the technique to a head-mounted display; and showed developments at CES this year. It announced that it could track a user’s eyes at 120Hz in February, and has now taken that to 250Hz.
SMI added its tracking system to an HTC Vive, which was used in the Nvidia demonstration. Nvidia has designed a new rendering algorithm, which “enables much greater foveation, or reduction in rendering effort, without any discernible drop in visual quality,” said VP of research David Luebke.
Image with contrast-preserving foveation
Jon Peddie of Jon Peddie Research saw the system in action and wrote about it in his Techwatch newsletter. He highlighted the main problem with foveated rendering: namely as resolution is reduced towards the visual edges, aliasing is introduced. A low-pass filter can remove the effect but creates an artefact of blurring. SMI developed a sharpening filter to overcome this problem while also hiding the jaggies. Peddie said that the tracker refreshes almost as fast as one can blink, so there is no perceptible lag.
Existing foveated rendering techniques tend to lead to either blur of flicker in the peripheral vision, as discussed above. Nvidia’s team, using SMI’s system, found that overly-aggressive foveation leads to flickering, while blurring peripheral images can result in a sense of tunnel vision. However, combining blur and contrast preservation, they found that users could tolerate up to twice as much blur before noticing a difference between the foveated and non-foveated images.
