Real-time classroom participation through VR.
UniVResity is a VR app that allows for face-to-face participation in a class by remote classmates. Using only the instructor’s voice as input, deep neural networks generate lip-sync and face-expression animation, non-verbal communication cues, gestures, and the teacher’s avatar movement.
The remote student is represented in class as an avatar projected onto the whiteboard. The whiteboard becomes a shared space of interaction between the instructor and the remote students. Collaboration on group activities is also possible, with the remote student using local student’s tablet or laptop as a shared interactive surface. The project explores automatic translation of user representation and interaction when medium and modes of input used by each user are different.