Paraphrasing a recent publication by a team of researchers at Microsoft Research: the lightweight optics typically found in the current generation of virtual, augmented and mixed reality near-eye displays cannot be scaled to provide resolution that is comparable with human visual acuity.
Achieving such performance with a “conventional” approach would require complex, multi-element optics that would be unacceptably large, cost too much and be too heavy. To address these issues, the team has undertaken development of an approach based on the use of digital holography.
The team will present their most recent results at Siggraph 2017. The presentation is entitled “Holographic Near-Eye Displays for Virtual and Augmented Reality.” It will be published in the July, 2017 edition of ACM Transactions on Graphics, Vol. 36, No. 4, Article 85. An on-line copy of the article can be found here. The work presented in this article is discussed in the video appended at the end of this report.
In the past, digital holograms could often be described as monochromatic images that were noisy and that had low contrast and low resolution. The goal of the team’s research was to demonstrate a near-eye display based on digital holograms that had an image quality close to that presented by current commercial displays and to produce such images using existing hardware components.
Software & Hardware Solution
The approach adopted by the team is based on the idea that computational displays “can use software to address complexity, where it is easier to meet a range of optical requirements.” More specifically, the team explains that their approach is “built on the principles of Fresnel holography and double phase amplitude encoding with additional hardware, phase correction factors and spatial light modulator encodings.”
One of the challenges with this approach is a consequence of the fact that complex calculations are required to produce a digital hologram. It follows that display devices of this type are often slow and typically use off-line calculations. The team addressed this issue by, in part, using eye tracking to produce approximate holograms, holograms that produce the correct focus and a high quality image only in the user’s gaze direction. Combined with GPU accelerated algorithms, the team demonstrate real time hologram generation at rates of 90 to 260 Hz on a desktop GPU.
Optical Correction
The digital holographic approach also offers the ability to use software to correct for abnormalities in the optics of the display. More specifically, holographic displays offer the capability to provide a unified focus, aberration correction and a user calibration process. In sum, correction for any optical defects that occur between the light source and retina. This capability enables simpler optics and the possibility to use alternative optical architectures. In fact, the optical corrective capability of holograms is so powerful that it allows arbitrary optical corrections on a per-pixel basis.
One particularly interesting aspect of the digital holographic approach is that it can provide the ability to change the image focus in a near-eye display. By this means, the display addresses the accommodation-convergence conflict, allowing the distance at which the image is focused to be the same as the distance at which the eye is focused.
Another capability of a digital holographic display is the potential to correct for peculiarities in the vision of the user. Peculiarities include near and far sightedness as well as astigmatism. Such a display should allow the user to comfortably view the image without the need to wear their glasses.
The optical correction capability of the digital hologram approach can also be used to enable a compact display with a glasses-like form factor and that has a wide field of view – a display that would be all but impossible to accomplish using conventional means. To demonstrate this point, the team developed a prototype near-eye display that is based on digital holography and shown in the figure below.
A prototype near-eye display based on digital holography and having a glasses-like form factor. The horizontal field of view of the prototype is 80 degrees.
The prototype included a thin, transparent holographic combiner cut in the shape of an eyeglass lens. The researchers reported that, “even with miniaturized optics, the display could resolve pixel-scale image details over the entire field of view.”
The big difference between this prototype and Microsoft’s HoloLens product is the form factor. Although the prototype looks more like a regular pair of glasses, its more favorable form factor come sat the price of a major reduction in performance. The prototype was capable of producing only flat, monoscopic images and needed to be tethered to an external processor. The HoloLens, on the other hand, is able to track the movements and position in space of the user’s head. The HoloLens is also wireless, runs on battery power and produces a stereoscopic 3D image.
The team characterized their prototype as showing promise while acknowledging that significant challenges remain in building a practical VR display. Future work planned by the team includes integration of additional capabilities into the glasses hardware and expanding the exit pupil. -Arthur Berman
Microsoft Research, Andrew Maimone, [email protected]
