Imagine your surgeon peering into your body — brain, liver or pancreas — before a major surgery, instead of the usual way, floundering in the dark alleys of an organ with a vague idea from a clutch of scanned images. This way, the team of surgeons would get to see an accurate three-dimensional (3D) image floating above the operation table, before they even touch the organ.

Surgeons and medical practitioners all over the world face a similar problem when they plan any invasive or non-invasive procedure on a patient. They often have to plan in advance and prep for what they can’t see and work based on intelligent guesses based on past experience of treating similar cases.

For a rookie surgeon, those initial years in the operation theatre can be a hit and miss affair. Moreover, nowadays students in most medical colleges, especially the private ones, do not get enough experience of handling cadavers. Neither are they exposed to post mortem procedures. The consequence — they miss out on opportunities to know the human body in all its complexity as intimately as their medical predecessors did.

In the past decades, a bevy of diagnostic tests, such as magnetic resonance imaging (MRI), computerised tomography (CT), ultrasound, positron emission tomography (PET) and so on have made things somewhat easier for surgeons and medical practitioners. Some tech companies did try to merge these images into 3D
images, but such visualisations were far from accurate.

In recent times a more refined holographic imaging technology has enabled medical professionals to look at precise quantitative measurements of the anatomical elements inside the human body. The method is revolutionary. It helps create 3D floating projections through the application of “mixed reality” which turns data culled from conventional scan images — MRI, CT or PET — into real-world images. These are like 3D holograms displayed in actual physical space. The holograms even allow doctors or surgeons to make precise medical decisions by allowing them to slice virtual tissues, organs and other body parts from various angles.

One of the pioneers in 3D holomedicine is the Hamburg-based firm Apoqlar’s technology. It is called VSI Holomedicine. The “medical metaverse” can transform various types of scanned images into a 3D hologram for a 360-degree perspective in vivid lifelike colours.

It extracts information layer by layer from individual scans of a certain area of the body to create an anatomically correct hologram. This can be viewed or positioned virtually anywhere in the room — say, near the surgeon’s table — via a headset (called Hololens 2 device) to enhance pre-operative surgical planning procedure.

The technology has been certified, deployed and consulted in scores of medical institutions across the world.

Sirko Pelzl, CEO and founder of Apoqlar, says, “VSI Holomedicine platform is used across 13 different medical fields to collaborate with other physicians globally as virtual avatars, conduct surgical planning in 3D, engage patients and educate would-be-doctors in medical academia.”

Apoqlar is planning next-generation applications in such a way that it can reach doctors in rural areas of India to make precision surgery easier and affordable for the masses.

At this point of time, the technology is but naturally expensive, with high costs driven by head-mounted display sets. It also involves high computational cost and is very data-intensive.

In the meantime, a consortium of scientists, doctors, technical specialists, policymakers and industry leaders has founded an organisation called Holomedicine Association. It is supposed to help transform the new technology for accurate surgical procedures and improve medical education for future doctors.