Institute for IDEAS

Institute for Immersive Designs, Experiences, Applications, and Stories

Interaction between static visual cues and force-feedback on the perception of mass of virtual objects

Wenyan Bi, Johnathan Newport, and Bei Xiao

VR Visual-Haptic Demo

We are exploring the integration of visual information and haptic feedback in an immersive VR environment. Observers are performing a picking and dropping task within Unix 3D with Head Mounted Display.

We created an interactive virtual environment which allows concurrent and spatially aligned visual-haptic presentation of virtual objects (A and B). Participants move the stylus of the force-feedback device (B) to manipulate the virtual object and feel its weight. The object is rendered with four materials (steel, stone, fabric, and wood) and two sizes (C and D). We independently vary the visual appearance (i.e. material and size) and the weight of the object, and measure how visual and haptic information affect the mass perception in the interactive virtual environment.


  • Designed and developed the visual-haptic experiment in an interactive virtual environment with force-feedback in Unity using C#.
  • Measured the material-weight illusion in the virtual environment.


We use force-feedback device and a game engine to measure the effects of material appearance on the perception of mass of virtual objects. We discover that the perceived mass is mainly determined by the ground-truth mass output by the force-feedback device. Different from the classic Material Weight Illusion (MWI), however, heavy-looking objects (e.g. steel) are consistently rated heavier than light-looking ones (e.g. fabric) with the same ground-truth mass. Analysis of the initial accelerated velocity of the movement trajectories of the virtual probe shows greater acceleration for materials with heavier rated mass. This effect is diminished when the participants lift the object for the second time, meaning that the influence of visual appearance disappears in the movement trajectories once it is calibrated by the force-feedback. We also show how the material categories are affected by both the visual appearance and the weight of the object. We conclude that visual appearance has a significant interaction with haptic force-feedback on the perception of mass and also affects the kinematics of how participants manipulate the object.

A visual demonstration of the Mass Perception experiment using VR.



  1. Wenyan Bi, Jonathan Newport, and Bei Xiao. 2018. Interaction between static visual cues and force-feedback on the perception of mass of virtual objects. In Proceedings of the 15th ACM Symposium on Applied Perception (SAP '18). Association for Computing Machinery, New York, NY, USA, Article 12, 1–5.