At this time, typical starting actions in the production of a digital 3D model might be the selection of an acquisition device and thinking through the details of the scanning process. The most likely next steps would include the on-site capture of the images and post processing the data. A production scenario along these lines is needed to produce a correctly registered and textured model having smooth surfaces and well defined shapes.
As a separate matter but also at this time, devices that can capture high resolution images have become commonplace. Think smartphones. The convergence of these two circumstances suggested to a team of researchers led by Michael Wimmer and headquartered at Vienna University of Technology, Institute of Computer Graphics and Algorithms (Vienna, Austria) a new approach to creating 3D models. The resulting so-called Harvest4D research project is developing algorithms to create a 3D model that is based on the use of existing image data. The important point to note is that such data is derived from photos not necessarily collected for the purpose of creating a 3D image.
A top level challenge in accomplishing this goal is to find means of integrating data derived from devices having different sensors and modalities. Furthermore, in order to create the 3D model, it is necessary to develop programs that tolerate errors because the captured data is never perfect. Sometimes images are out of focus. Sometimes an image might include varying light conditions or artifacts that interfere with image reconstruction. The original images might be of very different scales. Some images, taken from far away, may contain a broad view of the scene. Other images, taken close up, may contain fine details. This last point is significant because images captured at different distances may provide different surface-samples of the objects within the scene.
Despite the potential for such data difficulties, the calculation methods developed in the Harvest4D research project are claimed to make it possible to automatically process this type of image data.
In composing the 3D image, the algorithm implements several complex processes. Several of these are particularly noteworthy. For example, if an image is determined to have been taken from far away and has samples that are rather imprecise, then that data is over written by other, corresponding data taken from another image that was taken from closer up. Doing so can allow correct reconstruction of fine details. Another example is to identify images that show the same surface-samples despite the fact that they seem to show different details.
An example of the capabilities of the Harvest4D approach is shown in the picture below. It was created from 100 individual photos.
The researchers commented that, if the 3D imagery is not needed for a demanding application, then it may not be necessary to use the maximum level of detail.
If the times of image captures are known, then it may be possible to add a temporal aspect to the 3D imagery. This can enable an illustration of how the observed 3D scene evolves in time. Adding a temporal dimension is, however, a complex task. Doing so requires that the computer algorithm determine if the depicted objects have actually changed. The difficulty arises since there are non-temporal reasons why corresponding images may look different. They could, for example, have been captured under different light conditions, from different angles or with different devices.
A video is appended at this end of this article that discusses the Harvest4D project.
The researchers explain that their approach to creating 3D models is “high risk.” At the same time, they describe it as “an enabling factor for future visual applications.” With this the case, the current focus of Harvest4D activities is basic research, the goal being “to find answers to those questions required to lay the foundation for the new paradigm of incidental 4D data capture.”
The researchers hope their technology could become a market reality in less than five years. –Arthur Berman
Institute of Computer Graphics and Algorithms, Michael Wimmer, +43-1-58801-18687, [email protected]
http://www.euronews.com/2016/06/06/creating-3d-models-with-your-mobile-phone/
