Israeli start-up RealView Imaging has now confirmed that their medical imaging device, called the Holoscope-i, has received FDA clearance. This is a big milestone for the company which has been working on its holographic-based device for over seven years. To be clear, this is not an eyeglasses-style device but rather a boom-mounted head-up display (HUD) the surgeon or interventionalist uses for 3D visualization and 3D interaction during imaging-guided minimally invasive medical procedures.
The FDA clearance is “general” which means it can be used for a variety of imaging modalities. However, RealView is concentrating on cardiac imaging to start. Besides the 3D visualization technology, the system allows for interaction with the volumetric data via direct hand-hologram interface, voice commands and a wireless 3D control device.
According to the company, this is the first-in-the-world commercial system that is truly implementing interference-based real-time volumetric holography on a commercial scale. This technology provides physicians with a hyper-realistic and natural user experience, allowing them to view and interact with real-time 3D medical holograms of the patient’s anatomy floating in free space in front of the operator, prior to and during the procedure.
This is not a stereoscopic system so it should not suffer from any vergence-accommodation concerns, even when used at hands-reach and for many hours of work. And, by boom-mounting the device, there is no extra weight on the surgeon’s head, although it does require them to be within the viewing cone of the device.
The system uses Intel RealSense hand tracking technology with RealView software control and a high-end eye tracking electro-optical system implemented in the Holoscope-i system that was developed by RealView. One of their promotional videos showed how a surgeon could view the dynamic hologram and use his hand to rotate, mark and track where an implantable device has to go inside the heart.
The display technology is based on a specially-modified spatial light modulator (SLM) illuminated by red, green and blue coherent light sources. Speckle reduction is accomplished with multiple technologies and is said to not be a concern any more. According to CEO and Co-Founder Shaul Gelman, the system uses eye tracking at a high frequency to first calculate the exact location of the viewer’s pupils. Special electro-optics and software helps to improve the accuracy to 1/3mm. This information is used to help keep the image spatially aligned exactly to where the eyes would naturally expect the volume to appear. As many of the medical users use regular eyeglasses, the Holoscope-i system was designed with a generous eye relief supporting users with or without optical eyeglasses.
Next, the incoming volumetric data (3D/4D ultrasound, 3DCT, etc.) is processed in real-time to create a series of slices in the depth direction. Each of these are then transformed to the frequency domain to create an interference pattern that is written to the spatial light modulator. Coherent light then interferes with these patterns to reconstruct the full color image planes. The depth of each plane is also encoded into the interference pattern to create a series of images dense enough to present a volume 3D display at 60 fps. Additional optics project this image to a hand-reach range that can vary according to the user preferences.
Gelman says the system has very high resolution in the x, y and z axes with a 3D geometric fidelity laterally and in depth supporting the human eye vergence and accommodation mimicking real life 3D objects. And there is even a light that illuminates the surgeon’s hand to better visualize and interact with the virtual image that accurately appears at the very same volume as the interacting hand.
Computation of the interference pattern of medical images is compute intensive, but Gelman says, that following years of development of an extremely efficient calculation algorithm, they can now perform the calculations with existing commercial GPUs.
The Digital Light Shaping technology that is used in the Holoscope-i device is covered by 39 patents in 16 patent families with 28 grated to date. The company has already engaged with leading surgical centers and has successfully completed a pilot study with Philips. This involved live 3D holographic visualization and interaction to support minimally-invasive heart defect procedures.
RealView Imaging aims to see its systems installed first in cath labs at major hospitals and clinical centers, which has become the leading market in medical imaging for implementing 3D technologies. In development is the next version that will register and project the holographic image accurately 3D registered within a patient’s closed body to support minimally-invasive Interventional Oncology procedures. This proprietary holographic capability is based on the same core technology and is the only technology which provides the operators with accurate “x-ray eyes”, allowing them to see the treated organ and the medical tools in true 3D through the patient while the patient is closed making the patient literally transparent. (CC)
Chris Chinnock runs Insight Media, an advanced display and imaging consultancy. He is also executive director of the 8K Association.
