Using virtual objects with hand-tracking: the effects of visual congruence and mid-air haptics on sense of agency

Virtual reality expands the possibilities of human action. With hand-tracking technology, we can directly interact with these environments without the need for a mediating controller. Much previous research has looked at the user-avatar relationship. Here we explore the avatar-object relationship by manipulating the visual congruence and haptic feedback of the virtual object of interaction. We examine the effect of these variables on the sense of agency (SoA), which refers to the feeling of control over our actions and their effects. This psychological variable is highly relevant to user experience and is attracting increased interest in the field. Our results showed that implicit SoA was not significantly affected by visual congruence and haptics. However, both of these manipulations significantly affected explicit SoA, which was strengthened by the presence of mid-air haptics and was weakened by the presence of visual incongruence. We propose an explanation of these findings that draws on the cue integration theory of SoA. We also discuss the implications of these findings for HCI research and design.