A visual search engine is a seriously cool tool and application. But what is it, you ask? Let’s start with a few simple examples to explain what a visual search engine is and what can be done with it. Then maybe, you will see why the recent news from Evolution Robotics and KDDI caught our interest.

Art Berman
Insight Media Consultant

Suppose you are shopping for some new music. You see a CD cover or some other signage that catches your attention, but you want to hear a few clips to be sure this is what you want. A visual search engine allows you to use your cell phone camera to snap a picture of the CD cover. The orientation of the cover is not critical. In fact, even if a portion of the cover is obscured, that is OK. The photo is automatically e-mailed to a service provider and instantly the cell phone receives an e-mail back that includes links related to the artist, an option to hear clips from the album and the ability to purchase songs and download them on to the phone.

How about if you are shopping for a bottle of wine? Take a picture of the wine label and an email will be returned with, for example, an expert’s review and recommendation, right on the spot. Another possibility is that a user is browsing through a catalog and sees an item that they would like to purchase. In this case, the order can be initiated by snapping a picture of the item, just as it appears on the page.

So it seems that a visual search engine works like a traditional search engine but without having to type any text or go through complicated menus. Instead, users simply snap a picture of items of interest and there is an immediate return of relevant content, all into a device in the hand of the user.

So what’s the news in this area? Turns out that KDDI Corporation (www.au.kddi.com/english), Japan’s major mobile carrier is including the "ER Search" visual search engine on its new Spring 2008 "au" line of camera phones. In addition, the capability will be available for download by any KDDI customer with a prior "au" camera phone.

Evolution Robotics Inc. (Pasadena, CA, www.evolution.com), a leading robotics technology company, in partnership with Bandai Networks Co. Ltd (www.bandai-net.com/english), Japan’s leading mobile content provider, will provide the capability to KDDI.

This launch marks a dramatic expansion in the market for mobile visual search and will enable millions of consumers in Japan alone to do online searches by taking pictures of everyday objects with their camera phone. With this approach an entirely new way is being offered to customers to connect with content and companies said Satoshi Oshita, CEO of Bandai Networks.

All well and good, but the real reason for the corporate excitement surrounding the advent of this capability is that deployment of the technology in the mass market opens up the ability to provide an entirely new range of mobile marketing services. Such services are already projected to grow to $24 billion worldwide by 2013.

Bandai Networks had already deployed ER Search on over one million phones in Japan in 2007. With this deployment with KDDI, the number of users with access to ER Search will expand by millions more in a very short time, making it even more compelling for companies and advertisers to participate in the service.

The ER Search mobile search engine is powered by Evolution Robotics’ ViPR visual pattern recognition system. ViPR is able to learn new objects and images on the fly (such as the cover art on a music CD), without the need for any special encoding such as barcodes or watermarks. Just as significant, ViPR performs well on low cost components such as the cameras used on most mobile phones today, even when lighting and other visual conditions are poor.

The ViPR technology easily supports user-generated content so that users can take new pictures of objects, images, videos or even locations and tag them with links and content to expand the database. That content will then show up in the results returned to other users who take similar pictures, thus creating a robust world-wide visual database for communities to develop and access.

For the previously mentioned music search application alone, Bandai Networks has over 150,000 music CD covers already indexed in their database. Other mobile marketing and mobile commerce applications include providing content and links for print ads, book covers, DVDs, product packaging, movie posters, retail displays, business signs, etc. Even animation, streaming video or images from live TV can be supported.

A video demonstration of Evolution Robotics’ visual search technology running on Apple’s iPhone can be seen at: http://www.youtube.com/user/EvolutionRobotics.

Reply to the author