Despite having spent my career immersed in haptics, it is still shockingly difficult to offer up a clear and concise summary of the field. This is likely due to the fact that my head is way too full of factoids: oh so many trees blocking the view to a much larger forest.
Perhaps a more appropriate term for this post might be, “Organizing Principles to Better Understand Haptics,” since the goal of this post is to take a step back and provide guideposts that mark the way toward a deeper understanding. So, here’s what I want to do: I want to share three guideposts that I think are pretty important, and then and only then, share a few factoids. Forest first, then the trees.
What’s really important? Here are three things:
- Energy: The first thing to know about haptics is that, like all the senses, it gives our brains (and nervous systems more generally) access to the physical world. And in that physical world a really important consideration is energy. Energy is this beautiful but abstract thing that is common to all physical phenomena. Electromagnetic fields, for example, have energy. In our world, much of the electromagnetic energy comes from the Sun in a set of wavelengths that we call “visible”. It is no coincidence that our eyes exist to measure that abundant source of energy. We can also find energy in acoustic vibrations and chemical bonds (hence ears and noses) as well as mechanical strain (“strain energy” is what you find in a stretched rubber band) and heat. When we talk about haptics, we’re really talking about a sensory system adapted to measure the strain energy and the heat transfer at the interface between the body and its environment.
- Haptic perception is a product of evolution: The things we’re good at doing with haptics must have been shaped by evolution. You’ll see some cool studies out there in which an investigator pushes cookie cutter shapes against the palm and asks subjects to identify them, or places vibrators over swaths of the body to convey yesterday’s stocks or tomorrow’s weather. Those are helpful as they attempt to repurpose the haptic machinery that evolution has shaped for us. But, if we really want to understand haptics, we need to think about the original purposes for which it evolved. Warning and Disclaimer: I am not an evolutionary biologist. I am speculating here, but probably not too wildly. Common sense tells us that one purpose is to protect ourselves. We need to know when a mosquito lands on our arm, when a thorn punctures our skin, or when a pot is too hot. On the flip side, haptics also helps us assess that which is good — a cool breeze, the soft skin of a baby, Fido’s fur. Another purpose is to guide manipulation. I can reach into the quiver on my back, select a single arrow, extract it, position it in the bow and pull, all without taking my eyes off that rabbit in the distance (just call me Katniss…).
And that is a reasonably complete list. You may find others purposes, but the ones above will take us a long way toward understanding what haptic perception is all about. - Skin, muscles, the brain and moving about: The final guidepost I’d like to offer is the fact that haptic perception is served by an information processing system which includes things like sensors in the skin and muscles, the ability to position those sensors in space (by moving our hands, arms and bodies!), and of course, the brain.
Now the brain — you can be pretty sure — is looking for interesting patterns in the sensory data. The feel of clothes on your body? Not so interesting. The feel of a spider crawling up your arm? Interesting! (And yes, quite disturbing). Often, the brain will direct the hands to seek out interesting patterns, like when you reach into a cluttered pocket to find and extract only the car keys. By the way, I am not suggesting that this pattern-seeking rises to the level of consciousness. In fact, quite the opposite: it’s mainly happening under the hood.
OK, let’s summarize the three organizing principles:
- Haptics involves strain energy and thermal energy at the body/environment interface;
- Haptics probably evolved to help us identify noxious and useful stimuli as well as to manipulate stuff;
- The haptic perceptual system is good at eliciting and recognizing distinctive patterns.
Ready for some factoids? Here they come! Haptics involves two sensory systems that work together:
- Cutaneous (skin), and
- Kinesthetic (muscles and joints).
Think about groping for the shampoo bottle when your eyes are full of soap suds. Both types of sensors tell you where your arm is in space (a capability known as “proprioception”). Cutaneous sensing lets you know that you’ve touched the bottle while kinesthetic sensing lets you know the bottle size and shape and whether it is nearly empty. Cutaneous sensing alerts you to how slippery it is (and thus how hard to grip), and the two come together again to let you control your squeeze force. And so on.
Both of those systems involve some pretty fancy sensors. In the interest of space, let’s consider just the cutaneous. There is a lot happening in your skin! You have nerve endings for hot and nerve endings for cold. You have nerve endings for sharp pain and nerve endings for dull throbbing pain (actually, these are not usually considered part of the haptic perceptual system, but I’m taking some license). You even have four specialized mechanoreceptors — sensors that sit at the ends of nerves — for different aspects of strain energy.
Now, here is where I think it really helps to have those organizing principles, because strain is a really complicated thing (suffice it to say that tensors are involved). But your brain doesn’t care about that. It cares only about interesting patterns relevant to the evolutionary purpose of haptics. And there are basically two ways to see patterns:
- You can watch how the strain varies from point to point across space or
- You can watch how it varies across time.
When your finger touches the flat side of the shampoo bottle there is not a lot of variation across space, but when you find the edge of the flip top, there is. When your finger first touches it or slides along it until finding the grippy section, there is variation across time.
Pacinian corpuscle is at the bottom – click for higher resolutionThose four mechanoreceptors can be classified roughly by whether they are most responsive to fast or slow changes in space or time. For example, the Merkel Disks respond strongly to things that change rapidly in space (e.g., an edge), but not in time. The Meissner Corpuscles respond to things that change rapidly in time and space. The Pacinian Corpuscles respond to things that change rapidly in time, but not space (and they are awesome for vibration, be it from a cell phone or a mosquito). Finally, the Ruffini Endings specialize in skin stretch where the variations are not too fast in either time or space.
Your hand, by the way, is chock-full of these guys — roughly 17,000 of them. Why? Well, you need that many if you’re going to see interesting patterns.
It turns out, however, that the really relevant patterns don’t come straight out of these mechanoreceptors: some neural processing is required. That shouldn’t come as a big surprise since we find the same thing in vision — early processing extracts useful patterns like the presence and orientation of edges.
One really cool recent development in our field is a study led by Henrik Jörntell of Lund University in Sweden that demonstrated recognition of features such as making contact, breaking contact, slip, and roll, at very low levels of processing (specifically, an area of the brainstem known as the cuneate nucleus, where sensory nerves first arrive from the spinal cord on their way up to bigger and brainier things).
I Can Get Some Satisfaction
I find this work satisfying because it makes so much sense in terms of those organizing principles: patterns extracted from strain that serve evolutionary purpose. There are a lot of factoids in our field that just don’t make as much sense. One of my favorites is the oft-repeated statement that the Meissner Corpuscles are responsible for the perception of “flutter” on the skin. Huh? What does that even mean? Actually, it is a term that was coined in the 60s by the team of Vernon Mountcastle, one of the fathers of modern neuroscience. It refers to the sensation created by placing a small vibrating probe against the skin. Aha! “Flutter” makes total sense in a narrow scientific way, but should a trusted source like, say, Wikipedia really be reporting that the purpose of a certain mechanoreceptor is to sense flutter? No! Of course not.
Devoid of organizing principles (like evolutionary purpose), factoids can so easily lead astray. It is exciting to see that our field is starting to move beyond, as Jörntell’s work illustrates.
I hope this primer has delivered some helpful information and that perhaps I’ve piqued your curiosity. Haptic perception is a remarkable product of evolution — one that is well worth our collective consideration. We are fortunate to live in a particularly fecund time for the field with both scientific understanding and the engineering of haptic interface technologies advancing at a furious pace. So, set your alerts on “haptic” and brace yourselves for sensory overload just ahead! – Ed Colgate
Ed Colgate is Chief Haptics Office at Tanvas Inc. and Prof. of Mechanical Engineering, Northwestern University. We reported on Tanvas last month (TanvasTouch Adds Haptic Capability to Touchscreen Displays)
This article was first published in Touch Panel – Issue 95 from Veritas et Visus and is reproduced with permission.
