The creation of the ZED stereo camera required Stereolabs, Inc. (San Francisco, CA) to develop depth and motion sensing technology. The company has now gone on to use these technologies in the production of a new product, the Linq mixed reality headset.
The idea behind mixed reality is to combine the best aspects of virtual reality with the best features of augmented reality and to do this by merging imagery of the outside environment with that of the virtual world on a display within a headset.
First, a few words of background information to put the Linq into proper context.
It is common in the current generation of more sophisticated virtual reality headsets to use so-called outside-in positional tracking to determine the location of the user in 3D space. Such tracking is typically accomplished by means that include the use of outside cameras, IR-based sensors or structured light solutions. Using the outside-in approach, headsets such as those offered by Magic Leap and Microsoft provide users with a clear window to view the real, outside environment and then add virtual objects by means of some type of optical/projector system.
The designers of such headsets need to address several challenges. One hardware related issue is to achieve a sufficiently wide field of view. A more subtle difficulty is dealing with the potential for the transparent imagery to compromise the sense of realism experienced by the user. This approach also requires that the outside environment be mapped before initiating the activity provided by an application (such as a game).
The Linq product adopts a different approach to tracking, one that is called inside-out. To accomplish this, the headset includes an enhanced version of the company’s ZED high definition stereo camera. In addition, a display is placed inside the Linq headset.
The ZED camera provides “6DoF inside-out world scale positional tracking.” It scans the environment within a space that extends out to a distance of about 20 meters and performs scans in real time. The Linq headset is able to “understand” its’ environment including the recognition of people and objects. User movement is captured in the mixed reality experience itself. It is not necessary to initially map out the real world environment.
The result of the Linq approach is claimed to be a seamless integration of the imagery from the outside environment with the virtual objects.
Because the Linq uses regular RGB cameras, the company claims that it has advantages over headsets using other means of determining location. These advantages include the ability to work both indoors and outside, to do so at a longer range and to not be subject to interference from other sensors. It is also represented as cost effective.
A video at the end of this article shows the Linq in operation.
One on-line review of the Linq offered the comment that “You could walk around an AR space, place virtual windows and videos in the room around you, have everything stay put without much drift. However, we did notice more motion-to-photon latency than any of the consumer headsets, but we were told that that is to be expected for using combining 3rd party components to create a proof-of-concept device.”
Not surprisingly, one of the principle goals at Stereolabs is to address this latency issue. The company plans to accomplish this before the Linq is provided as part of the soon to be offered development kit.
Another important issue related to the use of the headset is hand tracking. Apparently Linq tracks the relative position of the user’s limbs with respect to both physical and virtual objects in order to determine what interactions should take place.
At this time, Stereolabs has not released an exact price for the Linq. None-the-less, the company has stated that the selling price of the Linq will be competitive with currently available gaming consoles. It can be noted, however, that with a retail price of $449, the ZED Camera by itself is already more expensive than many gaming consoles. Based on this as a starting point, it may be difficult for the Linq meet the company’s pricing goal.
The Linq Developer headset version will be tethered to virtual reality capable Windows or Linux computer. The final version is planned to have a small housing for the GPU and battery, so it can operate without the need to be tethered to a computer.
Stereolabs will make Linq available to developers in early 2017. The company plans to offer a consumer model later in the year. The headset will be compatible with both Unreal Engine and Unity content. –Arthur Berman
