The advent of 3D printing has drawn mixed reactions from the prenatal care sector. Non-invasive imaging is paramount, and a model of the environment inside the uterus can be instrumental in understanding tiny foetal anatomies. However, the cost of printing a precisely detailed 3D model for every difficult pregnancy quickly becomes prohibitive, since accuracy demands high-quality equipment and materials. The niche was therefore open for an affordable alternative.
In March 2016, Facebook released the Oculus Rift virtual-reality (VR) headset to consumers, sparking a buzz of speculation on its potential for the healthcare industry. Eight months later, at the annual meeting of the Radiological Society of North America in November, a team of Brazilian researchers presented their results: a detailed 3D foetal model viewed through an Oculus Rift headset.
Working from Rio de Janeiro’s Clínica de Diagnóstico por Imagem and Pontifical Catholic University, the team stitched MRI scans and ultrasounds of developing foetuses into virtual 3D models, incorporating higher levels of detail in whichever section a doctor needed to examine. The virtual reconstructions can provide close-ups of the external and internal structure of the foetus, uterus, vascularisation, umbilical cord and placenta.
Seeing is believing
3D-rendered models on a conventional screen are nothing new, but the attention-grabber was the viewing method: an Oculus Rift 2 headset, which provided a clear, sharp and immersive experience combined with a heartbeat sound effect. Surgeons could navigate through the model for a finely detailed close-up of developing airways, tumours, diaphragmatic hernias, or the complicated tangle of a multiple pregnancy.
Dr Heron Werner, who headed the research team, says the difference between a standard screen and the VR method is in the details. “The quality of the image is much, much better when you use 3D VR,” he says. “But the process and the origin of the imaging are the same.” The virtual construction of a foetus’s face correlates remarkably closely with the baby’s appearance at birth.
Through a series of case studies, the technique has proved to be a useful alternative to 3D printing. In 2015, Werner and his team used it in a case of twin-to-twin transfusion syndrome, in which a shared placenta transfuses blood unequally between twins, potentially resulting in slow growth, birth defects and heart failure. They confirmed that surgeons could use VR imaging to identify the placental equator where blood vessels from both twins connected, allowing treatment by using a laser to block connected veins and arteries.
In another case, published in January 2016, they created a virtual model and a simulated video of a 36-week-old foetus with a cervical teratoma obstructing its airway. The surgeon could navigate through the respiratory tract, visualise
the tumour and prepare for intervention immediately post-birth – opening the door to the possibility of virtual surgical ‘rehearsals’ and training. Werner believes the technique is “especially interesting for training novice endoscopic surgeons”.
There have naturally been a few hurdles due to the variable quality of the images obtained from each scan. Foetal movement during an MRI, amniotic fluid volume during an ultrasound, advanced gestational age, maternal obesity and inconvenient foetal positioning – all common limitations of prenatal imaging – can affect the quality of the scan and final reproduction.
“Of course, we have some limitations,” Werner says, “but not from the technique of 3D reconstruction. The limitations come from the basic exam, MRI and ultrasound. If you don’t have a good-quality exam, the final process will be difficult.”
The team is in talks with several companies, including GE Healthcare, to bring the technique to market. Currently, the major obstacle is that it can take up to six hours to digitally clean up around 200 images, representing 1mm ‘slices’ of the subject, and convert them into a virtual model. But, says Werner, the researchers are working on an automated fix.
“We are creating easy tools that will extract the foetal imaging very fast,” he explains. “We took an old exam that used to take around four hours to prepare, and, with this new program, we took 15 minutes.”
Right for the market
Although a decision on the eventual market price is still some way off, Werner is confident that affordability will be the biggest draw of virtual foetal imaging. An Oculus Rift headset costs $599 (£499) and provides some of the sharpest visuals of the current crop of consumer VR systems, but Werner says the technique doesn’t depend on having the right gear.
“You can use a smartphone,” he explains. “In the beginning, we had the Oculus Rift in the lab, so that’s why we started with that. But afterwards, we saw that the quality in cell phones is also very reasonable. I think VR will be a very cheap technique, and when it’s cheap it will be for everybody.” Basic VR viewers in the style of Google Cardboard, retailing for around $30 (£24), are designed to strap a smartphone to the user’s head and turn it into a simple VR display.
While the implications for surgeons are clear, Werner, who fondly remembers his now 15-year-old son’s first prenatal ultrasound, is also excited about the possibilities for expectant parents. In pathological cases, VR can help surgeons give parents a better sense of their child’s condition, prognosis and treatment. Werner is currently using the technique to help Brazilian parents understand cases of the Zika virus as well as trisomy 11 and trisomy 18, or Edwards syndrome – chromosomal disorders characterised by slow growth, abnormal facial structure and intellectual disability. However, for routine pregnancies, the chance for parents to see their baby’s face before birth is reassuring and can kick-start the process of bonding with their new arrival.
“We got a group of 20 couples with normal foetuses, just to see how the parents feel about the new kind of imaging, and the answer is surprising because they love it – the experience of navigating inside the uterus,” he says. “I think it’s another side of the exam. When they see a good image, they feel secure about the quality of the prenatal care and the progress of the pregnancy. I felt good when I saw my son in the 3D ultrasound; it’s something that introduces the foetus to the family very quickly.”
More benefits
The team quickly identified the potential to use VR foetal imaging with mothers at risk of postnatal depression, which often includes an inability to bond with their newborn. “We are thinking about that,” says Werner. “When you see the response, when you see how they feel with this kind of imaging, I think it could be a good tool.”
While he doesn’t believe 3D printing is the best solution for every patient, Werner acknowledges that sometimes you need something you can touch. “We receive blind, pregnant patients with normal foetuses. The women do their regular ultrasound for prenatal care and we reconstruct the 3D foetus, and then we use 3D printing to give them the experience of touching,” he says.
As tech giants Facebook, Google and HTC scramble to release better and cheaper devices, it’s the perfect time for the healthcare industry to embrace VR. It can offer parents the chance to get to know their growing child and feel confident as caregivers. Surgeons can closely examine every part of their patient, non-invasively and in detail; and students can learn the skills to perform delicate, life-saving procedures in a realistic training environment. Reduced costs and better recovery rates for postnatal surgery are just the beginning: virtual reality is now solid science.
A history of VR
The first VR head-mounted displays (HMDs) were developed in 1968, featuring wire-frame graphics and a headset so weighty it had to hang from the ceiling. However, the term ‘virtual reality’ wasn’t coined until 1987 with the development of the first consumer VR goggles and gloves – which cost around $9,000. Computing power has been slowly catching up to the science-fiction dream of completely immersive VR ever since, with the first Oculus Rift prototype designed in 2010.
HMDs like the Oculus Rift use two lenses in front of a screen to focus and angle the picture for each individual eye, creating a stereoscopic image that reflects the way your eyes naturally see and mimics depth perception. Accelerometers, gyroscopes and external cameras track your head movements so that the display turns to reflect them; eye tracking is also on its way. To use your hands in the virtual space – for example, in surgical training – you can use smart gloves or wireless controllers such as the Oculus Touch. A fully immersive VR experience incorporating realistic touch, smell and physical motion is still some years away.
The potential for medical training using VR was demonstrated in 2002 by a group of researchers from the Yale School of Medicine. They found that surgical residents who trained in laparoscopic cholecystectomy using VR could dissect a patient’s gallbladder in the operating room 29% faster than a control group. A 2007 Swiss study also found that VR simulations of clinical visits could improve the efficiency of a doctor’s communication style, and ‘virtual reality therapy’ is growing more common for the treatment of phobias and PTSD.
The technology has also been used in military flight simulators and infantry training, for the same reason that makes it useful in healthcare: the ability to train in environments where real-world failure can have serious consequences.
