While the Lytro Cinema light field camera got enormous buzz at NAB this year, Fraunhofer IIS showed a much smaller camera, similar post-production tools and a short video last year called Coming Home (photo) that was screened at the Technology Summit on Cinema just prior to NAB. We wanted to see what has happened over the last year so we chatted with Frederik Zilly.
Zilly loved the attention that Lytro got because it will bring a lot more attention to light field capture – and ultimately help them too, he believes. Fraunhofer has a different business model to Lytro. Lytro is in the business of building and selling cameras and production services. Fraunhofer is essentially a publicly-funded research lab whose goal is to license technology to commercial partners. They don’t have the budget to build a 755 Mpixel camera and spend lavishly on public relations, like VC-funded Lytro.
As a result, Fraunhofer’s light field camera is much more modest – a 4×4 array of commercial camera modules. But Zilly has been able to show all of the same benefits that Lytro showed such as the ability to create a 3D model of the live action video and merge with CG models, change the focus and depth of field, render stereoscopic images, create virtual cameras and camera tracks and much more.
Zilly is also looking at the potential to apply this technology to the capture of virtual reality content – as is Lytro.
In the movie production they did a year ago, which was only two and a half minutes long, they learned a great deal. They had to develop all kinds of post-production tools as well, which ended up being a series of 10 plug-ins for Nuke, a very popular and power VFX compositing tool. They used Occula as a stereoscopic plug-in as well. This is exactly what Lytro did too as Nuke has built-in load balancing algorithms that help parse the task for a local workstation or for dozens or hundreds of processors in the cloud.
Over the last year, Zilly says they have been working with six beta customers. These are film makers and post-production houses in Canada, the US and Europe that he characterized as ‘top tier’. While Lyro wants to make light field-based movies, Fraunhofer and its customers are more focused on understanding how light field capture can improve some existing processes. For example, light field capture allows for the creation of much sharper roto masks and more accurate depth masks that can be used to speed the compositing of visual effects and make them cleaner and more accurately placed in depth space.
It can also help eliminate the painting necessary in 2D-to-3D conversion projects when objects are occluded. Having a light field capture allows the creation of a virtual camera position so that obscured objects can now be seen.
This latter capability has turned out to be quite handy in some of the VR capture tasks they have done allowing a better “look around an object” capability.
Besides movie production, Zilly says they are looking at other applications as well including video conferencing, fine arts broadcasts, like opera and VR, of course. This is one of the most exciting area of imaging right now so there will be more to discuss going forward. – CC
(You can see a video interview with Zilly from IBC 2015 at https://vimeo.com/106360991)
