Most of us in the display industry first evaluate a display using the parameters we are used to: the quality of the image, whether it is a flat panel or a projected image, contrast ratios, color space, gamma curves, refresh times, and even the type of display illumination. But there is another dimension to the display equation we don’t often think about: the ability to interact with what we are experiencing on screen. Given some recent advancements, we may have to expand our mindset a bit in the future.

Alan Brawn
Insight Media Consultant

Certainly interactivity is not new technology but beyond the fairly simple touch screen, we have has not really explored or really developed new interactive technologies until recently.

Perhaps the most famous example of advanced touch screen technology, was in the movie Minority Report where the actors stood in front of the huge screen and created "magic" by multiple touches and gestures like the waving of arms. The technology allowed for the moving of images at will and changing sizes and formats. While Hollywood can do this with special effect, in the real world, this fascinating capability has been relegated to the realm of what I call "unobtainium".

But now, Jefferson Han a research scientist at New York University’s Courant Institute of Mathematical Sciences, is making unobtainium obtainable. He is one of the main developers of an "interface-free" touch-driven computer screen based on "multi-touch sensing." He points out that ATMs, smart whiteboards, etc. can only register 1 point of contact at the time. But multi-touch sensors let you register multiple touch points, using all your fingers on both hands, simultaneously. The technology is similar to existing touch-screen interfaces but multiple-touch allows you to do a lot more in the process. Han first publicly presented his multi-touch sensing work at the Technology Entertainment Design Conference in Monterey, California and they have released a video of the demonstration on line via YouTube. Since then, Han has founded a company called Perceptive Pixel to develop his touch screen technology further.

He is the first to admit that the "technology isn’t the real exciting thing, more the unconstrained interactions you can do on top of it once you’re given this precise information.  It is inherently multi-user permitting two or more people to address and control the images on the screen at the same time. The interface simply disappears."

Having experienced the demonstrations I was quite impressed.  Many truly believe that interactivity at this level is going to change the way we interact with computers.

He demonstrates dragging photos around.  With two fingers you can zoom in or out, rotate images, change positions, all in one seamless motion.  In another demonstration he used a rear-projected drafting table equipped with multi-touch sensors.   With kinesthetic memory, the visual memory where you left things, it gives the ability to quickly zoom, get a bigger work area if you run out of space, and to changes things.  He is correct when he claims that it is more of an infinite desktop than standard fixed area.  And you can utilize the interactive technology in both graphic and video formats.

"Someone who’s new to computing culture can use this," says Han.  "This could be important as we introduce computers to a whole new group of people. I cringe at the $100 laptop with its WIMP interface."

He knows that there inevitably will be comparisons with Minority Report and their special effects featuring gesture interfaces, which are not touch-based.  One of the main problems with the gesture interface is that you can’t differentiate between slight hover and actual touch.  In actual use this is disconcerting to the user if actions happen without tactile feedback.  Han argues that touch is more intuitive than gross gesture movements which by design are very imprecise.

Jefferson Han likes to think outside the box in terms of what is possible and what might make our interactive lives more intuitive.  Better communication and increased collaboration are at the core of his thinking.  His technology brings forth the promise of a truly dynamic user interface and there are lots of entertainment applications as well with multi-users playing in parallel.

This is all just the tip of the iceberg.  I am certain that Jeff Han’s work will stimulate much growth and development in the field of interactivity and the beneficiary will be you and me.  Stay tuned as we experience the new realm of the possible.