In a blog post, Google Research has pointed out that shared VR experiences are less compelling than they might be, because the face of the users are blocked by the use of headsets. Google Machine Perception researchers, in collaboration with Daydream Labs and YouTube Spaces, have been working on solutions to address this problem by virtually ‘removing’ the headset and creating an apparently transparent headset.
Google pre-scans the user’s face to create a 3D model of the user’s head and scans the eyes in different positions, then calculates where the user is looking (gaze recognition would help here) and maps this in. A marker is put on the user’s headset and if they are then filmed in front of a green screen (or using a depth-based system, we assume), the user can then be mapped into the virtual environment. That means that other people can get a real sense of the experience being enjoyed.
There’s a video at the end of this article that shows the idea.
