While traveling this past week to LA, I ran into another house guest at my AirBnB. Anne, a millennial Muscovite who works in IT security and was well briefed on the new iPhone X. Her response to the latest multi-billion dollar, years in the making, flagship offering from Apple was none too inspiring — “Boring” she quipped.
She was not at all excited at the prospect of this +$1K smartphone now with a new full edge-to-edge AMOLED display, CPU upgrades, longer battery life and new cameras including IR technology and more. Price? Size? Display? Face recognition? Weight? None of these factors could explain why the newest of the new from Apple elicited this response, and from the target audience of very high interest to the marketing team in Cupertino.
What occurred to me is that to demonstrate its latest new face recognition feature using the IR camera and new bionic processor, Apple showed-off a small (albeit possibly boring) application that is empowered by all this new technology. Specifically, “Animoji” technology (short for animated emoji’s). I totally agree that on the surface – yes, very boring. And some like me who are loathe to even use the emoji over a simple smiley face 🙂 would not consider using this technology in daily text conversation. Even the on-stage exchange between Apple CEO, Tim Cook and presenter, Craig Federighi, Sr. Vice President of software engineering – claiming “dibs” on one of those new characters (the fox). It all came across as rather awkward instead of comical, as the script writers probably intended. Somehow the prospect of these two highly paid executives of the world’s top market valued corporation exchanging ideas over their next billion dollar initiative using the talking heads of an animated fox, panda and alien did not seem right.- but such is the life of an Apple exec, or so it is supposed.
But past all that, a look at the technology behind the animated emoji gives one pause to consider just what may be on the horizon. Consider this, a hand held device with the power to capture, then analyze, over 50 different facial muscle movements, then map those expressions in real-time over a complex computer generated surface. Ummm, “wow” doesn’t quite capture it, but that description might generate more response than a talking Panda head.
The silly animoji technology achieves what only a few years back took an entire production crew, multiple high powered servers, a team of engineers and animators, and a year to generate similar effects. The technique was perhaps best demonstrated the Robert Zemeckis film The Polar Express. Here Tom Hanks, hard wired for facial expression and full body motion, played the role of half-dozen animated characters in the brilliantly produced (must see) animated Christmas favorite.
Apple’s first application in depth-sensing is “Face ID” and applied to security and authentication on iPhone X. It makes use of what the company calls a “TrueDepth camera system” that includes an IR dot projector, IR camera and flood illuminator projecting more than 30K invisible IR dots that make up an advanced IR image and dot pattern. This mathematical model of the human face is then sent to a “secure enclave” to match and confirm with pre-registered data. This we are told is secure with a 1M:1 chance of security breach, or about double what is achieved using the current finger print reader.
But the IR sensing technology is ripe for new applications, with potentially countless ideas that go well beyond the low hanging fruit of AR and VR, also highlighted by Apple at the launch event. On press day last week, Apple also announced Snapchat is using the new sensors “…to make its lenses attach themselves to people’s faces a lot more accurately and realistically,” the company press release stated. In short, the new IR camera is enabling anything in sensor range to be seen with well beyond visual acuity, and the phone has processing power to map the world around us in ways never before possible. With all the prior work done using the Microsoft and Nintendo IR camera technology, one has to stop and ask, are we on the cusp of a watershed of new apps unlike the smartphone world has seen?
So beyond the initial me too upgrades, AMOLED screen (finally) longer battery life, lighter, and even smaller lower weight than the iPhone 7 Plus, it’s the latest (bionic) processor updates and new “True Depth” IR camera / projection technology that floods the face with thousands of points of invisible light to enable face recognition, that can (I predict will) indeed usher in a whole new generation of hand held technology, cool features and can’t live without apps that will indeed change everything – yet again. We’ll see — Stephen Sechrist
Analyst Comment
Apple has said that the new phone will power up if the user is looking at the phone. That means that there must be some element of gaze tracking and that is very interesting to me (for example, see an earlier Display Daily – Look, Don‘t Touch – Gaze Saves Time.) I have long considered gaze recognition as one of the key technologies for the future and, of course, Apple bought German gaze specialist SMI in June, after Oculus bought ‘The Eye Tribe’. Tobii is still independent, although its investors include Intel. We have mentioned ‘gaze’ in 72 articles in our subscription newsletter in the last three years or so. (BR)
