Institute for IDEAS

Institute for Immersive Designs, Experiences, Applications, and Stories

Back to top

UniVResity:

Real-time classroom participation through VR.

UniVResity

UniVResity is a VR app that allows for face-to-face participation in a class by remote classmates. Using only the instructor’s voice as input, deep neural networks generate lip-sync and face-expression animation, non-verbal communication cues, gestures, and the teacher’s avatar movement. 

The remote student is represented in class as an avatar projected onto the whiteboard. The whiteboard becomes a shared space of interaction between the instructor and the remote students. Collaboration on group activities is also possible, with the remote student using local student’s tablet or laptop as a shared interactive surface. The project explores automatic translation of user representation and interaction when medium and modes of input used by each user are different.

A visual demonstration of UniVResity, a VR demonstration of being in the classroom.

Publications:

  1. Krzysztof Pietroszek and Chao Cheng Lin. 2019. UniVResity: Face-to-Face Class Participation for Remote Students using Virtual Reality. In Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology (VRST '19). Association for Computing Machinery, New York, NY, USA, Article 97, 1–2. https://doi.org/10.1145/3359996.3364730