On 6 May 2022, the 3IT held its annual Partner & Friends event: the 3IT Summit! The 3IT-Team was thrilled to welcome many old and new faces at this well-established and successful format that has been on hold since the pandemic. This year’s 3IT Summit focused on the most recent projects that have been launched by the 3IT Partners and included live showcases in the Immersive Media Experience Laboratory TiME Lab as well as a collective immersive live experience in 6DoF on VR headsets.
The program of the event included the following talks: The projects TeleSTAR and MultiARC presented the perks of AR-based clinical education. The consortial partners of the project VoluProf discussed about how Mixed Reality could be used to create a more efficient way of direct, interactive and user-oriented teaching via photo- and audiorealistic avatars of real lecturers for remote learning in a panel discussion. And the project Virtual LiVe showed the possibilities concerning the virtualization of live events via audiovisual immersion followed by technical demo tours from the project‘s successfully applied technological tool kit.
TeleSTAR/ MultiARC
TeleSTAR (Telepresence for Surgical Assistance and Training using Augmented Reality) utilizes a digital surgical microscope and augmented/mixed reality to provide new tools for teaching and intraoperative assistance. Standard procedures as well as rare and complicated ones can be visually annotated and augmented to make every step of the surgery transparent to a wide audience at the same time. A bidirectional communication platform allows questions to the surgeon. This concept can be scaled to other surgical domains and existing degree programs.
The project MultiARC engages with developing an interactive and multimodal-based augmented reality system for computer-assisted surgery for ear, nose and throat treatment. Input/output data handling, a hybrid multimodal image analysis, as well as a bi-directional interactive augmented reality (AR) and mixed reality (MR) interface for local and remote surgical assistance are part of the framework. The benefits are manifold and will lead to a highly improved patient outcome.
The speaker Eric L. Wisotzky is a research associate at the Computer Vision & Graphics Group of the Vision & Imaging Technologies Department.
Workshop Report „Live from the Other Room: Martin Kohlstedt“
Live from The Other Room is a VR-concert venue in which musicians perform in volumetric video, live streamed to audiences worldwide. Fans can book tickets to live shows in virtual reality, alone or with friends. Audience members can move freely through the space, communicate with each other and with the artist directly. Custom creative concepts are designed in collaboration with the artists, and musicians are recorded in a comfortable environment with direct audience feedback. The immersive experiences will be streamable on the most widespread 6DoF-VR headsets.
Since the beginning of the COVID pandemic many musicians have been trying to bring their music to audiences remotely. Some have been singing in front of webcams, some have made avatars out of themselves and performed in popular multiplayer video games like Fortnite. These performances spiked some interest, but are artistically not always interesting to serious musicians who thrive on audience presence and authentic communication.
Humans have been performing for each other for thousands of years, and a concert is a cultural expression that can affect a person profoundly – alone and/or in a collective context. Currently in its development stage, LFTOR aims to explore the elementary aspects of musical performance and what constitutes a satisfying experience for both artist and audience. Exactly replicating a real life concert situation in VR is technically not feasible and not useful, whereas a well-considered virtual space offers an opportunity to avoid the current disadvantages of established music industry practices and create a more inclusive standard.
Martin Kohlstedt is a German composer and pianist, known for his mixture of classical and avant-garde music, improvised on stage during passionate performances.
The two speakers were Harmke Heezen of creative studio High Road Stories (Berlin) and Oliver Ihrens of communication agency Radar Media (Bochum).
Live Tours
The official program was followed by live tours within the facilities of 3IT/ Fraunhofer HHI. The participants of the event were able to witness the 6DoF-VR-experience „Live from the Other Room: Martin Kohlstedt“ collectively in small groups. Another highlight was the presentation of the Virtual LiVe hybrid concert that took place at Kesselhaus Berlin in December 2012 and was simultanously broadcasted in 360 degrees to the planetarium Bochum as well as to audiences at home in real-time. Pieces of this concert where shown with 3D sound and in a 180° projection in Fraunhofer HHI’s Immersive Laboratory TiME Lab and allowed the guests to get an immersive glimpse of what the future of hybrid live events might look like.
Furthermore, long-time Partners like Schneider Digital, DNS-Consult and Fraunhofer IIS exhibited their latest technologies in the 3IT showroom.
The live tours were followed by a casual get-together, where our guests were able to share their experiences in the field of immersive imaging technologies and make new plans for possible future collaborations.
It was a great day with well-known as well as new faces, with exciting talks and contributions and exciting showcases.
