In the BMBF-funded VIVATOP research project, researchers at the University Clinic for Visceral Surgery at Pius Hospital Oldenburg are working with project partners from the Holomedicine Association to develop virtual and augmented reality applications and 3D printing processes that can be used, for example, to prepare surgeons for operations or to train junior doctors.
Prof Dr Dirk Weyhe briefly introduces his work in this interview:
What is the "Holomedicine Association" and what purpose does it serve?
Prof Dirk Weyhe: "TheHolomedicine Associationis a global association of doctors from various specialist disciplines, specialised industry partners such as Microsoft, IT companies and interest groups from politics and business. The Holomedicine Association was founded in April 2021 and I was appointed as one of the 40 international founding members. Overall, we can therefore speak of a global network of individual experts, science, technology and politics that identify new fields of application for mixed reality technology in medicine. The mission here is to support a digital transformation in the increasingly digital healthcare industry. The most important partner here is Microsoft Healthcare USA, which developed the Hololens glasses (see image Hololens II), and the Hamburg-based company apoQlar, which developed and continues to develop the associated software and operating platform Virtual Sugery Intelligence (VSI). Another important partner is the RPP Group, which sees itself as an intermediary in a common market for the promotion of bottom-up and top-down development processes in European healthcare policy-making and has representatives in the most important European capitals. Overall, it is a very innovative and highly motivated group with some ideas that still seem a little crazy today. For example, one of the founding members is working on Elon Musk's Mars expedition programme to develop mixed reality-supported dental treatments."
What does holomedicine mean?
Prof Dirk Weyhe: "The new term holomedicine describes the clinical applications of augmented mixed and virtual reality in patient treatment. "Holo" is derived from the term hologram, in which medical information from MRI and CT scans, ultrasound, microscope or endoscopy images is transformed into holograms (see image). This hologram is displayed in the mixed reality glasses (Hololens II, Microsoft), while the real environment remains fully visible. This 3-dimensional hologram can be touched and enlarged. Examples of applications include interdisciplinary tumour conferences, surgery planning or implementation. The 3D visualisation of the segmented images provides additional insights, particularly for surgical planning. Through the VIVATOP research project, we have made great progress in clinical applications under research conditions. Particularly in complex liver resections or pancreatic operations, we see great advantages in intraoperative use in clinical trials (see image OP). It is highly interesting that the AI programme supports the processing of MRI or CT images in addition to radiological and surgical expertise. We now have the first case where tumours have actually been automatically segmented that would have been missed by radiological diagnostics."
What is your role here?
Prof Dirk Weyhe: "I have been given the role of researching the clinical applications of holomedicine in visceral surgery. This involves investigating and evaluating the fields of action of mixed reality - from patient education to surgery as well as in training and further education. We are pleased to have been included in the Association's organisational chart as a Holomedicine Centre for Northern Europe. This development is also strongly supported financially by the Association. We are currently working with six mixed reality glasses in research and teaching. The IT support and software development required for this is provided by apoQlar and Microsoft Healthcare USA and supported by our internal IT department at Pius Hospital."
What will be the future tasks here?
Prof Dirk Weyhe: "The future tasks will be to develop a new form of holo-telemedicine following the clinical use of mixed reality. This means that several international participants/specialists will meet in a virtual room and work together on a hologram during the operation or even intraoperatively. Sounds like "Starship Enterprise" - but it's already partly possible in our new operating theatres. My special role will also be to help develop multicentre international studies in the field of visceral surgery. This will involve close coordination with the Association, which also has a lot of funding at its disposal. The first coordination took place in Singapore, which has the world's first 5G hospital. I have also recently been appointed to the advisory board of the company apoQlar for study planning."
How are these new developments communicated?
Prof Dirk Weyhe: "Due to the increasing importance of augmented, mixed and virtual reality in medicine, together with members of the Holomedicine Association, we have succeeded in setting up a "special topic" of the online journal "Frontiers in surgery". This is one of the first platforms in surgery where international science can exchange ideas."