Augmenting Virtual Reality

Interaction with the physical world while wearing a virtual reality device is difficult, to say the least – flailing arms will inevitably catch errant pens, cups of water and people while you are happily stacking blocks or flying a plane.

Augmented reality does not suffer from that problem in the same way, as in this system virtual data is overlaid on top of the real world, rather than blocking it out entirely. Now a team at the University of Illinois has developed a solution to the issue, with ‘augmented virtual reality’. That is, they overlay the real world on top of the virtual one, with the goal of allowing users to, say, take a sip from a glass of water without knocking it over their keyboard or removing their VR device.

Pulkit Budhiraja and his colleagues began with an Oculus Rift headset. A pair of cameras were attached, which produce a view of the real world in front of the wearer. This view can be accessed in four different ways.

In the first method, the wearer’s hand and the cup of water are superimposed on top of the virtual world. In the second, real objects, such as a keyboard and desk, appear as outlines in addition to the hand and cup. The third method has a window in the centre of the view, which shows a direct feed from the cameras; in the fourth method, this window is shrunk and appears in the corner of the display.

In tests, most people preferred the second method, which was not immersion-breaking but also provided context for movement.

For more information, visit