OTOY Inc. (Los Angeles, CA), is developing a camera system that can capture a spherical light field of a real world environment. Such imagery can be transformed into a virtual world scene that is viewable by a user wearing 3D VR goggles. Most significantly, the new system will eventually allow the user to move freely around and explore the virtual environment.
Regarding this last point, up to now, the presentation of conventional photographic content captured for VR applications has mostly been limited to panoramic images. It is certainly true that this approach provides some degree of realism in as much as the imagery is often in stereoscopic 3D. It is equally true, however, that this approach places limits on a user’s ability to explore the environment. More specifically, the user is usually restricted to viewing the scene from just a single position, it is not possible to “walk around” within the environment. Thus, by enabling a full range of movement within the environment, the light field capture approach has the potential to advance the state of the art in VR realism.
Perhaps a few words of explanation can more fully describe the advantages offered by the light field approach.
The sense of presence that a user experiences while moving about in an environment is not provided solely by the spatial arrangement of the objects within the environment. It is supplemented by impressions received about the materials composing the objects within the environment. This additional information derives, for example, from reflections off object surfaces or the distortion of a distant scene viewed through a transparent object. However, in these and other examples that could be imagined, details in the appearance of objects within the environment can change in response to even a small movement by the user.
As a practical matter, such details of a real world environment may not be captured by a simple photograph taken from a single camera position or even several positions. Stated more broadly and more technically, the problem relates to the occurrence of subsampling and aliasing when there are image components that are of higher frequency than the capture frequency. The result is that the captured “lower frequency” image may not accurately portray the image, often with the result that the image seems “lifeless”. OTOY’s moving camera approach will, however, offer a much better chance of catching such variable details since its “inherent sampling frequency” is much greater than that of a stationary camera. The proper terminology to describe this situation is that the moving camera approach provides a much denser light field capture. As a result, a light field captured by the OTOY system has the potential to produce a significantly more realistic representation of the environment.
The hardware in OTOY’s demonstration system consists of two Canon DSLRs that are rotated and spun in a circular pattern. The real accomplishment by OTOY is, however, the software used by the system to deal with the huge amount of data collected during capture of the light field of an environment. The compression capabilities of OTOY’s ORBX media format is extraordinary and results in the production of a file that has a manageable size. At the same time, the system also enables the high frame rates needed for VR applications.
The video accompanying this article illustrates the OTOY system in operation. It shows a seated user wearing Oculus goggles. The user moves their head and the video shows the two eye views seen by the user. The video does not show the user moving through the environment. Although the OTOY system may have a way to go before fully delivering on its’ potential, it nonetheless seems clear that the approach has a lot to offer and is showing real promise. – Arthur Berman
