Jim Spare, CEO of 3-D chipmaker Canesta (Sunnyvale, CA) spoke recently at the "NewTeeVee Live Conference" in San Francisco about his company’s 3D input technology. This technology enables, among other things, gesture-driven menus for television, games and computers. Why use a mouse when you can just point at a menu item? Why work a scroll wheel on your mouse when you can just move your finger in a circle?

Matt Brennesholtz
Insight Media Analyst

The Canesta technology is based on a time of flight (TOF) camera system that generates a 2D image plus a depth map, all at a 60Hz frame rate. Since Canesta supplies not only the camera but also the IR light source, the system is immune to ambient lighting and décor of the room. The light source is strong enough to detect gestures made up to 3M (9.8′) away from the system. Software supplied by Canesta can then decode the information into device commands as appropriate. Some problems that are difficult or impossible image processing problems when only a 2D image is present, become simple when the depth map is available. For example, it is easy to detect a moving finger, since it is separated in the depth map from the rest of the image.

Hitachi showed a gesture-based TV interface at CES last January that was based on Canesta technology. It was "Multi-touch" in the sense that the system could detect two-hand gestures and interpret them correctly. Hitachi isn’t the only company working on gesture-based TV menu systems. Toshiba also showed a demonstration at CES, although that was not based on Canesta hardware.

The Canesta system has a number of other applications besides controlling a TV. For example, they have developed a back-up system for automobiles that detects things like curbs, ramps and parked cars. In another automotive example, they have developed a passenger detector. The 3D nature of the data allows the software to determine not only the presence/absence of a passenger, but the size of the person as well. This allows better control of the airbag system.

Non-television interfaces are likely in the offing as well. One of the investors in a recent $16M round of financing was Quanta Computer Inc., said to be the world’s biggest manufacturer of laptop computers.

Several demonstration videos of the gesture-control of electronics are available here. One of them involves the gesture control of an avatar in a virtual world, which looks to Insight Media to be a good way to dim the line between the real and the virtual. "The iPhone’s multi-touch interface has spurred the development of an astounding 100,000 applications for that device," Spare said with obvious amazement. "Well, 3-D natural interfaces go way ‘beyond multi-touch’."

By the way, what happens when you shake your fist at the TV when your quarterback throws an intercepted pass?