The goal of the project VoluProf, Volumetric professor for omnipresent and user-optimized teaching in mixed reality, is to develop a Mixed Reality application that is suitable for everyday use within the context of knowledge transfer. The idea is to create a novel and more efficient way of direct, interactive and user-oriented teaching via photo- and audiorealistic avatars of real lecturers for remote learning.
The project is funded by the Federal Ministry of Education and Research (BMBF).
Overview
Traditional online learning, which provides content via static video and audio scripts, is limited in its effects since learners can hardly interact with teachers and give any feedback. Mixed Reality (MR) combined with animatable volumetric video has the potential to provide an immersive and interactive learning medium that enables a completely new level of learning experience.
VoluProf aims to develop a user-oriented, interactive MR application that enables discourses between teachers, represented as avatars, and users by means of multimodal chatbots. To achieve this, high-quality volumetric videos of the teachers are generated and animated as a basis for interaction. Voice of the avatar is generated realistically from a text provided by the teacher. The avatar is then streamed to the MR headsets of the learners over next-generation mobile networks using low-latency streaming techniques. The development is continuously accompanied by user studies that investigate the requirements and effects of the new teaching medium. Ethical, legal and social implications are monitored and continously integrated into the technical development.
The duration of the project is set for September 2021 – August 2024
Project Details
The project partners want to create a space of future learning where both traditional lecturing and online learning environments are combined in a way, where lecturers can be experienced not only on video but also virtually and physically. In other words, the user participates in a lesson with a virtual professor by means of Augmented Reality glasses for example in their living room, while also being able to interact with them via eye contact and verbal communication.
The aim is to develop the following key features:
- Generating a photorealistic and animatable volumetric representation of the lecturer.
- Location-independent applicability through transmission via 5G networks and direct integration into mixed reality devices on the market, as well as subjective optimization of image quality at minimal bit rate.
- Natural and fast synthesis of the lecturer’s original voice.
- Answering ethical, legal and social questions about the implications of the new way of knowledge transfer.
- Media reception and effects research for the analysis and optimization of user acceptance.
- Educational research on the optimal and user-adaptive use of the new way of knowledge transfer
