When we see any movie today we see a wide array of special effects that are computer generated. These computer generated images are than combined with footage of real actors to create the final video content. To make the computer generated objects behave as we expect them to behave, a lot computer modelling work goes into these scenes. Of course, if you want to create some crazy effects (think of ‘The Matrix’, for example) you can do this too. When we think of augmented reality, this is where we will see computer generated content meet the real world. As far as text goes, it is just text floating on top of the real world. For computer generated objects will just sit on top of the real world without any relation to what seems realistic to us and what doesn’t.
Source: Abe Davis – Image-Space Modal Bases for Plausible Manipulation of Objects in Video
Abe Davis and his group at MIT are showing us a way in which we may be able to overcome this hurdle with very little investment. In their work they describe this as ‘Interactive Dynamic Video’ or IDV for short. Please refer to their website for more background information, literature and examples on this topic.
For a short description, they analyze a short video of a real world object to derive some sort of vibration mode that can be used to predict realistic movement of an object in a computer generation. Similar approaches for computer generated content have resulted in videos using wave and wind animation. However, as soon as you see those animations you know what is fake and what not.
The new method described in their articles is a relatively low cost method that can create a realistic real time solution for any computer generated object. In other words, they have created an algorithm that can extract the behavior of an object from real live footage and predict how the object may respond to any form of physical interaction. This does include interactions that have not been observed before.
All of this sounds very theoretical for now, but it is easy to see how this may translate into presenting objects in augmented reality applications. Abe Davis is continuing his work with post-doctorate research at Stanford University starting on the fall of 2016. – NH
