Communication through the skin

The tactile sense can be used as a channel for general communication, especially in contexts where the visual and auditory modalities are occupied with other tasks (e.g., while driving, or performing surgery) or compromised (e.g., people with visual or auditory disabilities).

We have created an apparatus that communicates generic words and phrases by delivering vibrations representing the word phonemes  through two voice coil transducers worn  in the forearm.  Participants in our studies practiced with the apparatus for a few hours and were able to:

1. Recognize a set  of 50 words rendered haptically with a user-controlled interval between phonemes with an average accuracy of 94.4%: Mauricio F. de Vargas et al., Haptic Speech Communication Using Stimuli Evocative of Phoneme Production, IEEE World Haptics 2019. 

 

2.Generalize their haptic phoneme identification skills to the understanding of untrained English words, correctly identifying 65% of words in phrases rendered with a user-controlled interval between words, and up to 59% with a fixed interval. Mauricio F. de Vargas et al.,  Speaking Haptically: From Phonemes to Phrases With a Mobile Haptic Communication System, IEEE Transaction on Haptics 2021. 

 

3. Engage in communicative tasks in which they had to overcome a gap in information or collaboratively reasons about a problem using open, unrestricted language, (e.g., schedule a movie outing) receiving a conversation partner’s speech entirely as vibrotactile actuation.  87.5% of all communication tasks were successfully completed.  David Matino et al., Conversing Using WhatsHap: a Phoneme Based Vibrotactile Messaging Platform, IEEE World Haptics 2021.