This Day Belongs to Apple’s Displays

Today marks the inaugural day of Apple’s Worldwide Developer Conference (WWDC) 2023, and in a few short hours, we’ll get a glimpse into the tech titan’s plans for its highly anticipated mixed reality (MR) display and other future product launches. While we may not acquire a complete blueprint of what Apple has up its sleeve for 2024 or 2025, the happenings of today should illuminate the company’s potential influence on the display industry.

Indeed, it’s displays that are steering the course of WWDC 2023 this year, beginning with the much-anticipated launch of Apple’s MR headset, widely speculated to be named Reality Pro. There’s a substantial amount at stake for Apple in this endeavor; if the product launch fails to meet expectations, the company will have to shoulder significant criticism. It’s no secret that Apple has been less than watertight with information, thereby stoking the flames of anticipation.

The general consensus is that Apple is likely to unveil a passthrough MR device equipped with 1.4-inch 4,000 ppi MicroLED displays for each eye, believed to be manufactured by Sony. For those interested in understanding the finer details of this technology, Karl Guttag’s presentation from last week’s AWE 2023 conference is an excellent starting point. Guttag delved into the core differences between optical MR and passthrough MR technologies, as well as their respective applications, providing a comprehensive overview of AR/VR/MR headsets. He also skillfully debunked some of the most fervent, yet unrealistic, expectations regarding Apple’s imminent announcement.

Optical MRPassthrough MR
Seeing the real worldTop priorityVirtual world is top priority
LagNo lagMinimal or no noticeable lag
Dynamic rangeHighHigh image quality with high contrast
Focus distancesInfiniteVirtual content is typically presented at a fixed depth or superimposed on the real-world objects without the ability to dynamically adjust the focus distance.
Accuracy100% accuracy in size, location, depthAccuracy is not irrelevant, but prioritizes the visual quality and immersion while maintaining a clear view of the real world.
HandsEspecially accurate for handsUsually with controllers in the hands
Peripheral visionImportantLimited or no peripheral vision
Hands-free operationOften hands-freeRequires controllers or hand-held devices
Gesture recognitionNot hands-freeCommonly rely on controllers or hand-held devices for input and interaction
Movement in real worldSupportedSensors on the headset or external cameras track the user’s position and movements relative to the immediate surroundings but users are constrained by the physical play space and the tracking capabilities of the system
Usability outdoorsPossibleMust be in a “safe” location
Social issuesWorries about looks, eyes, recordingIgnores social issues with wearing headset
Occlusion accuracyDifficult and inaccurateTypically involves wearing a headset that allows you to see the real world while overlaying virtual content. Since the real world is directly visible, occlusion accuracy is not a major concern as the real objects in the environment naturally occlude the virtual content.
ImmersionDepends on optics and form factor of headsetHigh FOV needed for immersive experience
Summarized from part of Karl Guttag’s presentation at AWE 2023. (Source: Karl Guttag)

xrOS, which stands for extended reality operating aystem, is the rumored software that is expected to power the Reality Pro. xrOS is expected to have a user interface (UI) resembling iOS, rumored to include a Home screen with rearrangeable app icons, allowing users to customize their virtual workspace. Also similar to iOS, xrOS may feature customizable widgets that provide at-a-glance information and quick access to specific functions or apps.

Apple is rumored to be redesigning apps such as Messages and Maps, specifically optimized for the Reality pro. These apps are likely to offer immersive and interactive experiences and lay the groundwork for other developers to start working a new software development kit (SDK) for Reality Pro. It seems like that xrOS will integrate with the broader Apple ecosystem, including iCloud, Apple Music, Apple Fitness+, and more.

It will be interesting to see how xrOS feature various input methods, including hand gestures, voice commands, and possibly a dedicated controller or input device. The notion that the Reality Pro will change how people use all Apple devices in the future is part of mythology that has been built around the device. Which leads us on to the specific hardware requirements for xrOS, that are not known, and a lot will depend on how the Reality Pro hardware is setup. Apple’s headset is a totally new platform and developers will have to squeeze every ounce of performance out of the hardware that they can to deliver a fully immersive experience. How Apple is handling the performance of the platform is going to be an interesting insight into its future; by all accounts, all rumor and speculation, the Reality Pro is expensive, uses up a lot of power, and requires a lot of processing power.

Hardware Upgrades at WWDC 2023

Beyond the Reality Pro, Mark Gurman at Bloomberg has done the most comprehensive speculation on Apple’s possible hardware updates to its product line-up, summarized below. It is unlikely that we will see any earth-shattering new display technology, but there may be some insight into the company’s future plans, particularly its expected switch to bringing display development in-house. If, as expected, there is a 15-inch MacBook Air launch, it’s probably a good bet that Apple will move to new display technology to coincide with future MacBook Air product launches based on its M3 chip. You could say that this year is a holding pattern for Apple because there isn’t much here to excite anyone but the Mac faithful.

DeviceExpected Features
15-inch MacBook Air– M2 chip
– 3024 x 1964 resolution, similar to 14-inch MacBook Pro
13-inch MacBook ProRefresh with new M3 chip
13-inch MacBook AirRefresh with new M3 chip
24-inch iMacRefresh with new M3 chip
Mac StudioNew versions expected with M2 Max and M2 Ultra chips

So, it really boils down to what the Reality Pro does today. This years WWDC, in all other aspects, is a holding pattern for Apple. There are software upgrades to iOS, iPad OS, tvOS, and Watch OS, and of coure, xrOS is going to be of particularly interest. However, software upgrades and OS fixes should be considerate mandatory; the fact that we have to applaud companies fixing their own issues seems a little nuts, but where else can executives get their hour in the spotlight to explain how they got more widgets.