The first speaker in this session was Neil Trevett, VP of the Developer Ecosystem at Nvidia and President of the industry consortium Khronos Group. His talk was titled “Khronos Open Standard APIs – the Foundation for AR and VR” and covered Khronos and not his day job at Nvidia. The mission statement he gave was “Khronos is an international industry consortium of over 100 companies creating royalty-free, open standard APIs to enable software to access hardware acceleration for 3D graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks and Vision Processing.” I’ve written about Khronos before when writing about standards for AR and VR. Of course, Trevett is considerably more knowledgeable about Khronos and their APIs than I am.
The Khronos OpenXR API is specifically targeted at AR/VR applications. Two other APIs likely to be useful to AR/VR system developers are OpenVX which is designed to work with vision sensors (e.g. cameras) and NNEF, an API that simplifies programming of neural network systems. Other APIs from Khronos such as OpenGL, OpenGL|ES, OpenGL|SC, Vulkan and OpenVG simplify real-time 2D/3D cross-platform gaming & UIs, necessary features in a AR/VR system. WebGL would be another useful API for a mobile, web-based AR/VR system. Since virtually all consumer AR devices are going to be mobile and web based, this won’t just be useful, it will be essential.
AR Processing Flow using APIs from the Khronos Group (Image: Khronos Group)
Still, OpenXR is the key API needed for AR/VR applications. Development of it started in December, 2016 and development of V 1.0 of an API at Khronos typically takes 12 – 18 months, according to Trevett, so OpenXR isn’t ready for developers yet.
OpenXR Working Group Members at Khronos (Image: Khronos)
Daryl Sartain, Director and Worldwide Head of Virtual Reality and Displays at AMD, gave a talk titled “Building Blocks – A new Ecosystem.” Like an unfortunate number of speakers at SID, both at the AR/VR business conference, he spent an inordinate amount of time on an introduction to AR and VR. He also discussed AR/VR subsystems, where chips and modules form AMD could aid in the design of AR/VR hardware systems. Like Clay Bavor from Google in his SID Keynote Speech, Sartain emphasized the very high bandwidth needed to supply VR displays with data, as shown in the image.
VR Bandwidth calculated by Daryl Sartain of AMD (Image source: AMD)
The only problem with this figure is it doesn’t go far enough – a single 3850 x 2160 display doesn’t have enough resolution to provide a true “Virtual Reality” experience. Perhaps you will need two displays, each with 2x the pixel count of 3840 x 2160, for a total of 4x the bandwidth. This puts you in the 100Gbps estimated by Bavor. Maybe faster chips from AMD are needed…
Amal Ghosh, Senior VP at eMagin presented a talked on “OLED Micro-display Technology for Augmented Reality and Virtual Reality.” Before SID, I had had a thorough briefing by eMagin on both their technology and products and had written it up for Mobile Display Monitor readers. I won’t repeat here what was in that article, published about a month ago. One thing that needs emphasis, however, is that eMagin makes OLED microdisplays that use both the color-by-white technology and the company also uses patterned red, green and blue OLED materials. The patterned RGB systems are roughly 5x brighter than the color-by-white microdisplays from eMagin or their competitors.
The modest brightness levels achieved by color-by-white OLEDs are good enough for VR systems where ambient light is excluded and relatively efficient optical systems can be used, For see-through AR systems where the image on the microdisplay must compete with ambient light and relatively low efficiency see-through optical systems are needed, the patterned RGB microdisplays with their high output are needed. In addition, a very low dark level is needed in order to not wash-out the see-through image.
According to eMagin, the low black levels and high output makes patterned RGB OLED microdisplays the system of choice for AR systems that will be used in ambient light, better than color-by-white OLEDs because of their brightness and better than LCD or LCoS microdisplays because of their low dark level. The high fill-factor and pixel density are also desirable, especially compared to VR systems based on cell phone displays.
AR/VR forecasts used by eMagin. Note the extreme range between the $180B“Accelerated Uptake” and the $20B “Delayed Uptake” forecasts for 2025 from Goldman Sachs. (Image: eMagin)
eMagin should know about AR HMDs – they have been developing and manufacturing OLED microdisplays for military and industrial HMDs since 1997. Recently, they have begun to branch out into consumer products as well. In addition to giving this talk, eMagin was on the exhibition floor demonstrating their patterned OLEDs and Amal Ghosh plus 11 co-authors from eMagin and the US Army gave a paper (18.1) on the topic. While the paper only claimed 5,000 cd/m² (nits) for the microdisplay in the paper, it is believed the patterned OLED technology is capable of brightnesses up to perhaps 25,000 – 50,000 cd/m².
1” 2K x 2K patterned RGB OLED microdisplay from eMagin targeting AR applications (Photo: M. Brennesholtz)
Won-jun Choe, VP of Samsung Display, gave the final presentation of the session, titled “Display Influences on VR – OLED for VR.” Note that he only talked about VR for the Samsung color-by-white OLED microdisplays since, at 1,000 cd/m², they are not capable of driving a see-through AR HMD in the presence of significant ambient light. He focused on the advantage of OLED – any OLED – over LCDs in VR HMDs. These advantages included higher contrast, higher brightness, lower latency and lighter weight. These factors, according to Choe, lead to a reduction in Visually Induced Motion Sickness (VIMS).
Motion sickness, sometimes called VR sickness, is a significant problem for the uptake of VR HMDs in the consumer market so these improvements are all in the right direction. While OLEDs are likely to always have an advantage over LCDs in terms of contrast ratio and weight, his arguments that they have lower latency and higher brightness were less convincing.
Global Units Sales Forecast for HMDs from Samsung Display (Image: Samsung)
