BMW Group Technology Office USA, based in Mountain View, has unveiled a research update in collaboration with Meta’s Reality Labs Research. The new development demonstrates, for the first time, the ability to accurately and stably display virtual reality (VR) and mixed reality (MR) content to passengers in a fast-moving car. This is achievable even when the car is making turns, going over speed bumps, and accelerating.
To overcome the technical challenge of displaying stable virtual content in a moving vehicle, BMW and Meta integrated the tracking system of Meta Quest Pro with the sensor array of a BMW car. VR headsets typically use cameras and motion sensors to understand their position. In a moving vehicle, however, the headset’s sensors can be confused by conflicting signals from the device’s movement within the car and the vehicle’s movement in the world.
By incorporating the vehicle’s sensor data, the tracking system can now calculate both the device’s location relative to the car and the moving car’s location relative to the world. This dual calculation system enables accurate anchoring of virtual objects within the moving vehicle, making it possible to create a stable and immersive VR and MR experience for passengers.
The system is robust enough to render content synchronized with the vehicle’s motion in real time, regardless of road conditions, whether on a flat, straight street or a bumpy road with sharp turns. The result is a proof-of-concept prototype that demonstrates a full range of a person’s movements (known as six degrees of freedom) within the XR experience.
In terms of application, BMW and Meta envision a range of use cases. These could include assisting the driver in locating their car in a crowded parking lot, alerting them to road hazards, and providing vital information about the vehicle’s condition. It also opens up possibilities for VR and MR gaming, entertainment, productivity tools, and even meditation facilities for passengers.
As the research progresses, the team hopes to develop AR glasses that could allow passengers to see things like markers for landmarks, restaurants, and places of interest. This technology could potentially transform the way we interact with our environment while traveling.
Despite these developments, Claus Dorrer, Head of BMW Group Technology Office USA, clarified that it’s still early days. The timeline for when this technology will be available to customers is not yet clear, and much of the technology’s potential is still being explored.
Moreover, the update highlights BMW’s commitment to privacy-by-design, ensuring that customer data is protected and securely processed according to data privacy requirements. The company is also open to partnering with other companies to help establish an industry standard for connecting XR consumer electronic devices to vehicles.
