What a Brain Surgeon's Virtual Reality Tool and Pokemon Go Have in Common
The tech pinpoints brain structures using a tech similar to Pokemon Go.
— -- Augmented reality may be grabbing headlines with the rise of the hit game Pokemon Go, but a New York surgeon is examining how this technology could be used to save lives during some of the most complex brain operations.
Dr. Joshua Bederson, chairman of neurosurgery at Mount Sinai Health System, spent his Tuesday morning this week calmly working his way through the brain of a 47-year-old patient. On the walls of the operating room, scans of the patient's brain were projected onto various screens. Two of the screens showed the view that Bederson could see through his surgical microscope, revealing how he slowly worked his way between two lobes of the brain.
On the screen that projected the image from Bederson's surgical microscope, brightly colored lines that looked like a yellow X and a small green circle were superimposed over the brain tissue that Bederson was delicately working around. It was the circle that Bederson was aiming for, an unruptured aneurysm hidden at the base of the brain.
Bederson was using a new technology called CaptiView, which he believes could be the future of brain surgery.
"We’re bringing together several different technologies in this surgery. The first is 3-D visualization of the aneurysm in a way that simulates the actual anatomy a little more accurately than we have done in the past," he told ABC News before the surgery.
"The analogy would be a GPS that we use in our car. We know more or less where we’re supposed to go much of the time. But if we have a map that’s updated in real time and we can see the position of our car in relation to that map while we’re driving, it can make the drive a little more efficient, maybe even a little safer," he said.
Bederson has spent decades learning how to be more efficient during the sometimes marathon brain surgeries. He keeps his watch and wedding ring tied to the drawstrings of his scrubs as he operates and he carefully positions his arms on a padded bar before he starts working on brain tissue.
As he performed the operation this week, he explained to a surgery resident that getting in the right position meant he could easily work for three to four hours without a break. By using this new technology, called Surgical Theater, which marries conventional brain scans with virtual reality and augmented reality technology, he hopes to make surgery more efficient than he could have imagined.
Data from the patient's brain scans are translated through navigation software called Brainlab to create, as Bederson calls, it a "GPS for the brain." That map is laid over the real-life images to denote important areas of the brain, such as the optic nerve, represented by a yellow X or carotid artery, shown on screen as a red squiggle.
"There are many analogies between what we do and flight," Bederson said. In flying, "you can think of information that comes to you through windshield of plane. ... When the cloud starts to come in and you begin to lose references outside, you rely on the plane’s instruments."
Bederson is also utilizing virtual reality and 3-D printing to plan for complicated surgeries. The technology that reveals 3-D scans are translated into data that allow surgeons to wear VR glasses and get a close-up look inside the brain before surgery. With these scans, they can literally wear glasses and take an inside look at an aneurysm or a tumor, while getting a 360-degree view of the brain. In past surgeries, Bederson has 3-D printed part of the brain's anatomy to plan and practice an operation.
"When a sculptor starts a piece, this sculptor sees the hand inside. But it’s covered by all this rock in front of it," Bederson said. "Much of surgery is about taking away what’s not needed to reveal the normal anatomy. ... It's very analogous to sculpture. I saw early on the potentials of 3-D reconstruction."
While Bederson is hopeful for the future, the technology is still being fined tuned. During the surgery, Bederson gave live feedback about the display, color and accuracy as he carefully got ready to clamp the aneurysm. This technology could someday advance to the point where robots may take over some basic surgeries, he said.
"We are driving and advancing the development of next generation simulation and virtual reality technology, which can improve patient outcomes and solve neurosurgical challenges," he said.