German

3DInMed


The constantly changing and advancing 3D market also provides new innovative solutions for the medical and industrial sectors. The further development of the medical engineering industry is supported by the Federal Ministry for Economic Affairs and Energy (BMWi) with our latest project: 3DInMed.

3DInMed @ Control 2017

Fraunhofer HHI and Schölly present a technical case study of a 3D endoscope used for online visual inspection tasks of hardly accessible parts at Control 2017.


Stereo image-based measurement for remote visual inspection

Download 3DInMed@Control 2017 Flyer

3DInMed @ Medica 2016

Fraunhofer HHI presented first results of the 3IT project 3DInMed at Medica 2016. Digital stereo camera systems offer new possibilities to extract and visualize additional information resulting in significantly enhanced conditions for diagnostics and surgical interventions. Shown are the latest research results for measuring, tracking of medical instruments, augmented reality applications (AR) and optimized 3D playback.

A positive feedback was received from many sides, including engineers, surgeons and product managers. This result is a strong indicator that the developed methods and solutions of 3DInMed are of high relevance for digital 3D imaging technologies in medical applications.

Download 3DInMed@Medica 2016 Flyer

Overview

3DInMed describes an ongoing collaborated project of different partners, aiming to establish and develop immersive imaging technology for industrial (In) and medical (Med) applications. Including applications such as destruction-free industrial inspection- and measurement techniques, diagnostic and interventional radiology or the introduction of computer assisted designs (CAD) and further digitals tools for planning and production.

Existing technologies used by the entertainment industry, such as 3D-sensors, high-speed signal and imaging processors, high-resolution autostereoscopic displays, touchless interfaces and improved 3D-printers, are actuating those developments. The targeted technologies are more and more used in other application areas, such as planning, production or inspection processes in construction, trade and industry. Further, 3D imaging methods (e.g. endoscopy and microscopy) will change diagnosis and surgical processes strongly.

Objectives

The project has essentially four main goals, including the development of new, solid processes of depth estimation and object detection as well as the design of high frequency and low latency transmission lines for 3D signals. Furthermore the development of real time processes to create endoscopic 3D panoramas through texture- and depth information from stereo-endoscopic views is essential. As well as the development of spatial mapping processes and measurement methods for 3D microscopy/endoscopy data and related AR-applications are planned.

Results and Partners

The main objective is to develop, evaluate and validate key technologies for 3D-capturing, -processing, -transmission and autostereoscopic visualization and integrate these technologies into and beyond the targeted application areas. The consortium is led by the Fraunhofer Society Munich and the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute. The project partners are ARRI Medical, C.R.S. iiMotion, Fraunhofer HHI, Fraunhofer IIS, Schölly, SeeFront and Solectrix. Ultimately the project has associated partners in particular the Technical University Munich / Hospital “rechts der Isar”, research group for Minimal Invasive, Interdisciplinary, Therapeutic Intervention (MITI); Ludwig-Maximilians-University Munich, Großhadern Campus, Cochlea-Implant-Centre; Neurosurgical clinic and polyclinic, hospital “rechts der Isar”, Technical University Munich; University Hospital Erlangen and Siemens Energy.

Industrial Details

3D applications in industry are primarily used in the field of automotive and robotics by improving driver assistant systems, autonomous navigation and related capturing of surroundings. Touchless inspection and measurement processes of inaccessible tubular parts are another field of application. Supported by 3D visualizations in computer assisted designs, these inspection and diagnostic applications are used as well in 3D X-Ray or CT-Volume of chemical or biological structures.

Medical Details

3D applications in a medical environment enables doctors to record stereoscopic data and its three dimensional visualizations to guarantee an enhanced and improved workflow for minimal invasive operations. Associated technologies are surgery-microscopy, spatial approximation of operating areas, recognition and labelling of risk tissues (e.g. nerve tract) and robot or telemanipulator supported operations. Furthermore it aims at enabling the combination of preoperative and intraoperative diagnostic images, for documentation and education purposes.

Graphical Material

3DInMed

Copyright: Fraunhofer IIS - (a) Left image of a typical 3D laparoscope (b) L-HRM disparity map (c) 3D reconstructed point cloud - 2 instruments have been removed

3DInMed

Fraunhofer HHI: (a) Real-time tracking result of instrument tip inside a temporal bone model. (b) Related result of real-time depth map estimation.

Events And Conferences

CARS 2017, Barcelona
CARS 2017-Computer Assisted Radiology and Surgery, 31th International Congress and Exhibition Barcelona, Spain, June 20-24, 2017.
New Tools for Digital Surgical Microscopy
N. Gard, S. Jurk, A. Schneider, P. Kauff, P. Eisert, J.-C. Rosenthal, Fraunhofer Heinrich-Hertz-Institut, Berlin; ARRI, München

Please note: the conference language was German therefore some of the following titles are also in German language.

DGE-BV MEETS ENDOSKOPIE LIVE in Berlin, 06.04 - 08.04.2017

Herausforderungen, Lösungen und Potenziale der Stereo- Bildverarbeitung für die 3D-Endoskopie
Jean-Claude Rosenthal, Niklas Gard



SID-ME Fall Meeting 2016 in Berlin, 03.11 - 04.11.2016

Depth-Based Quality Control for Medical Applications
Jean-Claude Rosenthal, Niklas Gard, Peter Kauff



15th CURAC Annual Conference in Bern, 29.09. - 01.10.2016

Session: Endoscopy & Microscopy

3D-Reconstruction by Polarization Imaging in Endoscopy
J. Sandvoss, T. Wittenberg, A. Nowak, J. Ernst

Evaluierung von 3D-Rekonstruktionsverfahren in der Stereo-Laparoskopie
D. Erpenbeck, T. Wittenberg, J. C. Rosenthal, P. Kauff, N. Lemke, T. Bergen

Program: http://curac.org/programbern2016#Endoscopy-Microscopy

Special session: Optische 3D-Endoskopie & Mikroskopie

Miniaturisierte 3D-Endoskopie
Johannes Ruhammer, Niels Lemke, Schölly Fiberoptic GmbH, Denzlingen

3D-Techniken in der Klinik
Dirk Wilhelm, Nils Kohn, TU München

Potential der digitalen 3D Operationsmikroskopie
Armin Schneider, ARRI Medical, München

Von der 2D zur 3D Panorama-Endoskopie
Daniel Erpenbeck, Thomas Wittenberg, Fraunhofer IIS, Erlangen

Tiefenbasierte Stereo-Bildverarbeitung für medizinische Anwendungen
Peter Kauff, Niklas Gard, Jean Claude Rosenthal, Fraunhofer HHI, Berlin

Program: http://curac.org/programbern2016#Special-Session-Endoscopy-Microscopy



14th CURAC Annual Conference in Bremen, 17.09. - 19.09.2015

Session: Optische 3D-Modalitäten in der Chirurgie

3D-Verarbeitung und Visualisierung mit Optischen Modalitäten
Jean-Claude Rosenthal, Fraunhofer HHI, Berlin

Die Digitalisierung der Operationsmikroskopie
Christoph Bichlmeir, ARRI, München

Aspekte der 3D Endoskopie
Niels Lemke, Schölly, Denzlingen

Panorama Endoskopie - von 2D zu 3D
Thomas Wittenberg, Fraunhofer IIS

Program: https://www.curac.org/programm2015#Sondersitzung-CURAC-DGBMT

Consortium Leader

 

Project Partners






Associated Partners

Technical University Munich, Hospital “rechts der Isar”, Research Group for Minimally invasive Interdisciplinary Therapeutic Intervention (MITI)



Ludwig-Maximilians-University Munich

Neurosurgical clinic and polyclinic, hospital “rechts der Isar”, Technical University Munich

University Hospital Erlangen

Siemens Energy