Sense of agency in gesture-based interactions: modulated by sensory modality but not feedback meaning

Introduction: Gesture-based interactions provide control over a system without the need for physical contact. Mid-air haptic technology allows a user to not visually engage with the interface while receiving system information and is readily manipulable, which has positive implications for automotive environments. It is important, however, that the user still feels a sense of agency, which here refers to perceiving system changes as caused by their gesture.

Methods: In the current study, 36 participants engaged in an experimental time perception task with an automotive-themed infotainment menu, serving as an implicit quantitative measure of agency. This was supplemented with additional self-reported measures. They selected different icons via gesture poses, with sensory feedback either visually or haptically. In addition, (sensory) feedback was also the same for each icon, arbitrarily different or carried semantic information.

Results: Mid-air haptics increased agency compared to visual, and this did not vary as a function of feedback meaning. Agency was also associated with general measures of trust and usability.

Discussion: Our findings demonstrate positive implications for mid-air haptics in automotive contexts and highlight the general importance of user agency.