Understanding Our Hands is Key to Getting Haptic Technology Right

By Maria Cury and Mikkel Krenchel

We don’t always trust our eyes and ears. But we trust what we feel with our hands. Why?

Meta (formerly known as Facebook) recently revealed that it is working on a haptic glove that lets you ‘feel’ the metaverse. In order to replicate, enhance, and perhaps even celebrate the physical capabilities of our hands in a digital context, it is critical to study and understand what our hands actually do, what they mean to us, and how they work.

By helping our hands ‘feel’ these alternate realities, technologists are not just providing a more immersive experience, they are opening the door to a whole new world of potential opportunities for what technology might do for us, including building and amplifying our skills, deepening our relationships, and making better sense of the actual world around us.

One way to understand the potential of a haptic glove is to think of hands as people. This might sound obvious, buthands have bodies, relationships, and minds of their own. Whether holding oars while rowing, using a scalpel during surgery, or playing with a baby, our hands act as our key point of contact with the world. They confirm our reality.

Hands have bodies. Consider the sleight of hand of a magician. His fingers move independently, in ways that challenge our usual instincts for how to move, and in ways that are useful for magic tricks that challenge our perception. To build up this ability, these hands must train over and over again. Or consider the hands of a chef, whose fingertips perceive traits of the fig they are cutting — its shape, texture, softness, temperature. They are able to modulate their movements deftly in the moment, without losing control, applying just the right amount of pressure to keep the fish in place without damaging its flesh.

Hands also have relationships — they work in collaboration. They can form strong bonds with other senses, body parts, or tools. The hand can do more when it recruits help beyond itself, just like people rely on relationships to achieve more together. Like the hands of a pianist who is blind, her hands build a relationship with her sense of hearing, the two working in unison. Her hands intuit, based on the sound and the volume of a note, whether they are positioned correctly and applying the right amount of pressure to the keys.

The hand is stronger when it recruits the full body and tools to help. The hands of a ceramicist preparing the clay are in relationship with the legs and shoulders, to apply the right amount of strength and pressure needed onto the clay. Likewise, the hands of a manicurist work in collaboration with a set of tools. Through the relationship her hands have built with the tool, they are able to work within a much smaller surface area, with greater precision, than they normally would.

And hands have minds. They rely on their knowledge and past experiences. Like the hands of a bike mechanic, that can diagnose what’s wrong with the bike based on how it feels when compared to all the bikes these hands have worked on before. Hands store up memories, familiarity, and instincts good or bad, over time. The “minds” of our hands allow us to adapt to change, and to reduce the effort we have to put in, to thinking about our actions. They can compare the situation to memories of situations they have been in before. The hands of a sign language interpreter, for instance, assesses when to sign faster or slower, more crisply or more fluidly, based on the dynamics and reactions of the audience.

If we extend the metaphor and think of hands like we think of humans, then we realize hands have culture, ambitions, experiences, and history; they act on the world and learn from it.

The observations above help us imagine, and anticipate, future technologies that are not only relevant and useful, but that also that respect and enable the aspects of ourselves that make us feel most human.

How can technology help them build skills, gain new experiences, and delight in the feeling of expertise? And when we shift our unit of analysis from people to hands, without shifting the way we think about structures, systems, practices, and meaning, what types of input can we provide to developing this technology?

Our study of hands poses some fundamental questions: When so much of design focuses on usability, comfort, and a technology-first approach, how do we design for putting our hands, and their needs, first? How do we ensure that new technologies for our hands celebrate and amplify our skills rather than replacing them?

As companies roll out new technologies like haptic gloves that mimic the sensation of weight and touch of real objects when handled in virtual space, it’s important we keep the technology in the background to ensure hands, and humans, can learn, collaborate, and shine on their own.

We should design for hands like we design for people — with bodies, relationships, minds, and memories.

Maria Cury and Mikkel Krenchel are partners at ReD Associates. This essay is adapted from a Pecha Kucha short film presented at the 2022 Ethnographic Praxis in Industry Conference (EPIC), in collaboration with Facebook Reality Labs (FRL).