Thought the capture of 180° and 360° video content was a recent phenomenon driven by the capabilities of the Oculus Rift and Samsung GearVR? BigLook360 did its first production way back in 1999 and just recently shot the NBA All-Star game in the format. Content should be ready for viewing on the GearVR at the Samsung Milk VR web site in the March/April time frame.
Founder/CEO Lance Loesberg of BigLook360 explained that the company got started when General Motors asked them to shoot a 360° video as an experiential marketing tool that would be a traveling show going from fairs to trade shows and other events. The idea was to provide a new way to show off the advanced features of the car in a way that would have an impact on viewers. It was supposed to be a one year program but was extended to three years as it was leading to lots of showroom visits and sales of cars.
“We shot this first production using film cameras with some special optics to create a panoramic view”, explained Loesberg. “In the back seat was golfer Ben Crenshaw narrating the trip and features of the car. This was then played back using VR headsets in a tractor trailer on a platform that included some vibration to simulate bumps and even smells like pine cones when traveling through a forest”.
The company has continued to produce similar type content ever since. Which brings us to its latest event, the NBA All-Star game. This was held in New York City in early February. There were actually three events that were captured: a behind the scene 3-point and slam-dunk warm up event; the real 3-point and slam-dunk event and the All-Star game itself.
Loesberg explained that the company used four different 3D, 360° camera rigs to capture these events. One was a Red-class rig, one was a GoPro rig and the other two used cameras with capabilities in between these. Each rig features 6-7 pairs of cameras, which makes for a very imposing set-up.
This rig features dual RED cameras “within a 360° rig” says BigLook360 CEO Lance Loesberg, but he can’t reveal more details on exactly how this works.
“We are agnostic about which cameras we use, but we have learned to understand how to use each camera in which positions and situations in order to capture the best images”, explained Loesberg. “For example, these cameras all have different capabilities in terms of resolution, frame rate, field of view, parallax, light saturation and these have to be looked at carefully for each shot. We are capturing in stereo 3D so we only use prime lenses too”.
Loesberg explained that the company started working with Samsung nearly two and a half years ago and fairly recently shifted its efforts to support the GearVR headset for playback. “The playback device makes a difference in how we capture and render the content too”, said Loesberg. Last August, Samsung invited BigLook360 to show NBA officials what the capture and rending system could do and how it might work for an NBA game. Those discussions progressed and resulted in the capture of the All-Star game.
One of the courtside rigs. This one features 5 stereo camera pairs. The one on the other side featured 6 stereo camera pairs.
“The NBA is very cognizant of player and audience safety as well as the audience impact of new capture techniques”, continued Loesberg. “They gave us some pretty constricting limitation for the capture of the game. This included three camera position (near the referee table, in the corner and one mobile unit), plus once the fixed cameras were set up, we were not allowed to go back to them even if someone bumped into them or pulled out a plug. Fortunately, it all worked out”.
Loesberg said it captured about 12 TB of content from the event and the company is now in the post production process. This will eventually be boiled down to several 3-5 minute clips that will be used as a marketing and promotional tool by Samsung and the NBA. The idea is to offer a “court-side NBA experience” to those who never sit that close or to those who have never attended an NBA game. That’s a pretty smart plan.
The NBA has also talked about the idea of creating a service where a 360° camera is positioned court-side and streams this content live as a premium service. This would allow participants to really feel like they were at the game with the ability to look in whatever direction they wanted at any time.
I asked if the company would also create a longer clip to really “live in the moment of the game”. Loesberg said that that was a Samsung and NBA decision, but there might also be some technical issues with that as well. For example, the Milk VR spec limits the GearVR-compatible file size to around 40MB, so longer content might have to make sacrifices in resolution or other parameters to meet this limit.
Loesberg explained that the company will create a much higher resolution master of the segments it puts together from the event. From these it will create an “intermediate rendering” with reduced fidelity. This has about a 5GB file size limit to meet the Milk VR specification.
Apparently, the incoming GearVR format spec places the entire 360° spherical image in one file format container (.mp4 with embedded audio) and this entire file is streamed or downloaded for playback. The video track format is 4096 x 2048 for 2D and 4096 x4 096 for 3D, at 23.976 to 60 fps with a maximum bit rate of 40 Mbps using H.264 encoding.
Samsung then transcodes to three output levels (low, high and super) using increasing bit rates and moving the H.265 in the super version, apparently to support a range of connection bandwidths. Playback is at 30, 48 or 60 fps.
Other approaches, like the one proposed by NextVR, only transmit an FHD frame, but use the headset sensor data to predict where the user will look next. This “look ahead” data is also sent to provide full resolution images as the user moves in the predicted direction.
To me, this latter approach seems better suited to obtaining a higher fidelity image with a limited bandwidth pipeline. While the BigLook360 content probably looks fantastic when viewed from high fidelity files, I suspect it suffers greatly when formatted to meet the GearVR platform spec. We have asked BigLook360 to come to the Display Summit in June to demonstrate the system, so hopefully a number of folks will be able to see this and other demos around AR and VR. – Chris Chinnock
