A team headed by Dr. Rehan Hafiz in the Department of Electrical Engineering of the National University of Sciences and Technology (Islamabad, Pakistan) has developed a “unified scheme” for seamlessly stitching together multiple videos to create a projected panorama.
The most recent work reported by the team is entitled “A unified panoramic stitching and multi-projector rendering scheme for immersive panoramic displays.” This article was published in Displays, Volume 40, December 2015, Pages 78-87. A copy of the article is available for purchase here.
Interest in the panorama format derives, in part, from the fact that such video can present images to an audience that have both high resolution and a wide angular field of view. Panoramic projection of this type is typically implemented in conjunction with a physically large, curved screen. The desirability of producing such panoramic video derives from the fact that they have the potential to enhance the degree of immersion experienced by the audience.
The team’s recent article explains that the task of creating such an immersive panoramic video includes three main challenges:
- Capturing the video from multiple cameras.
- Stitching together the captured video frame by frame.
- Stitching together the images projected by multiple video projectors.
Traditional System Overview – image courtesy Dr Hafiz
As a practical matter, the task of stitching together the captured video and stitching together the projected images can be very challenging. The reason is that the stitched together images must match quite precisely in order for the video to look good. Precise matching includes geometrical factors such as magnification, shape and image rotation. Also included are photometric factors such as brightness, color, contrast and focus. To make the task even more challenging, stitching and rendering adjustments needed to match the images must be applied to every frame in real time.
In their most recent article, the team proposes a so-called unified scheme that handles these two separate transformations using a single, combined Look up Table. The scheme allows projection of planar and cylindrical panoramas on curved cylindrical displays.
Proposed new system – click for higher resolution version. Image courtesy Dr Hafiz
The details of how this is accomplished are discussed in the technical article. The experimental results, however, are quite impressive. It is reported that, for a two camera, two projector panoramic display system, the new method results in a speedup of up to 73% for HD video and 83% for 4K video. Furthermore, for a 4K projector, the proposed scheme requires 2.278 ms for the transformation of content, thus enabling real time panoramic video projection. – Arthur Berman
National University of Sciences and Technology, Rehan Hafiz, +051-90852107, [email protected]
