In order to enhance the reality of the VR experience, some commercially available headsets include the capability of tracking the user’s head, eyes or limbs. The current generation of headset is not, however, capable of introducing the user’s emotions inside the VR world. To do so requires an understanding of the user’s facial expressions. This issue is being addressed by Veeso (New York, NY). The company is developing a technology that can capture detailed information about the expression on the user’s face and map this information onto the user’s avatar – and do so in real time.
This feat is accomplished through the use of face tracking cameras and sensors built into the headset. More specifically, the Veeso prototype includes an infrared camera mounted in the headset and located between the lenses. It is used to track the position of the user’s pupils, eyelids’ openness and eyebrow movement. The system also has the capability of determining and tracking the user’s line of sight. A second infrared camera is located on an extension arm that projects below the basic goggle as illustrated in the accompanying figure. The camera is pointed at the user’s mouth and jaw and serves to track their positions and movements. By these means, the sensors enable the Veeso headset to determine where the user is looking as well as the user’s emotional state as determined by analysis of the user’s facial expressions.
The Veeso product is a smartphone-based VR headset. The headset is compatible with both Android and iOS devices.
In the prototype, a cable is connected to the headset. The company states that the final product will be wireless. An integrated battery pack will be used to power the device. Bluetooth will be used to communicate tracking data between the smartphone and the headset.
A video can be found at the end of this article that discusses and illustrates the Veeso headset.
The headset is said to be “Cardboard-ready.” There are several consequences to this tie in.
First, any headset that uses the inertial measurement unit in a smartphone for head tracking is limited to the accuracy of such systems. Put another way, the Veeso system is not able to capitalize on the higher accuracy, lower latency sensors found in headsets such as Gear VR or the upcoming Google Daydream platform.
Second, the Veeso headset can use existing smartphone-based content developed for Google Cardboard. The flip side of this situation is that content developed for Gear VR and Samsung/Oculus cannot be used with the Veeso headset.
One of the challenges still facing the company is to make the system work well with all types of faces. On this point and based on results to date, the company believes that it will be possible to achieve a level of success sufficient for a commercial product.
In order to fund the next stage of headset development, the company launched a Kickstarter campaign on July 20th. At the time this article is written, the campaign has attracted 86 backers that have pledged a total of $10,663. The goal of the campaign is $80,000 which must be reached for the project to fund. The campaign is scheduled to conclude on August 19th. Clearly this campaign has a way to go. Assuming the campaign is successful, the company hopes to initiate headset delivery this December. In addition, the company has plans to make available ten custom-built applications and games within 6 months of the hardware delivery date.
The inclusion of user emotions in the VR environment is an excellent idea and the specific approach and execution under development at Veeso seems to have real potential. It is not, however, unreasonable to suppose that, whatever the fate of Veeso, facial recognition will have a role to play in VR. –Arthur Berman
