Scientists across the U.S. are adopting computer generated imagery and virtual reality (VR) techniques from the film and gaming industries to advance research in neuroscience.

Two projects presented by the University of Southern California in Los Angeles and by the University of Utah at the 2017 Society for Neuroscience annual meeting in Washington, D.C. gave the public a preview for the implications of VR in brain science.

Combining Visualization Software & VR for Virtual Dissections

The first project demonstrated work by a team of researchers from the USC Mark and Mary Stevens Neuroimaging and Informatics Institute, who developed software that enables scientists to view the human brain in 3D and perform virtual dissections.

The recent findings give scientists and researchers the tools to view neurons and brain structures in immense detail.

“The brain is naturally 3D and we measure it in 3D with magnetic resonance imaging (MRI), but, we’ve always been constrained to look at it through 2D flat screens, which is inherently limited,” said Professor Tyler Ard, a neuroscientist at the Stevens Neuroimaging and Informatics Institute. “We wanted to see if we could use recent VR advancements to be able to view and interact with MRI data in its natural 3D state, and explore the advantages that would offer.”

Ard and his colleagues used computer rendering techniques to transfer large amounts of data into images. The team has created software that makes it easy to quickly import data, and intends to make the code freely available to other researchers.

The team combined the visualization software with new VR techniques to enable neuroscientists to take a virtual look at the human brain.

One of the most immediate implications of this research will be in education, said Ard.

“The way my generation of neuroscientists learned neuroanatomy was largely from professors at the front of a classroom, going through 2D ‘slices’ and figures, and saying over and over again ‘Now imagine this in 3D,’ ” said Ard. “This non-intuitive learning strategy often does not leave students with a concrete understanding of brain structure.”

This research comes at a crucial time in the field of neuroimaging. With the growing amount of information and concepts that students need to understand, it is imperative for researchers to seek ways to make learning easier and quicker for students, said Ard.

“As an example of how the field is growing, MRIs are now able to move past simple brain structure, and also look at functional interactions, as well as anatomical ‘wiring diagrams’ of the brain,” said Ard. “Neuroimaging in Virtual Reality (NIVR) makes it possible to examine all of these types of information through a medium that removes the conceptual barrier of ‘now imagine what this really looks like.’ ”

The researchers intend to release their program, NIVR, next year.

Using VR to trace neurons

The second project demonstrated work by a team of researchers from the University of Utah, who developed a way to use VR to trace neurons.

There are 50,000 neurons with close to 6,000 connections to neighboring neurons in only one cubic millimeter of human brain tissue.

Traditional neuron tracing techniques involve a scientist looking at 2D computer images and constantly rotating them to view what’s behind the neuron’s branches.

The new method allows researchers to create and sort through 3D images of neuron pathways. The scientists can use headsets and handheld controllers to rotate and look behind neurons.

“In most labs, this process is performed manually and is a pretty time consuming bottleneck in the overall pipeline of taking scanned tissue to the final analysis on the structure that the scientists are interested in,” said Will Usher, a graduate student research assistant in Valerio Pascucci’s lab at the University of Utah, who demonstrated the program. “A tool which can shorten this time and help them understand their data faster helps a lot.”

To test and compare the program, the team of four neuroscientists mapped neuron images using a conventional desktop computer and the new VR technique. The new method was proven to be 1.7 times faster than the traditional technique, and just as effective.

The team plans to test its VR tool in more labs and make adjustments based on the feedback, said Usher.

In the coming months, the researchers intend to make their VR tool accessible for additional members in their field.

“For this we’re looking at deploying it on Steam, which makes for a convenient VR program distribution platform, and we’re planning to put the code up on Github as well,” said Usher.

The team plans to release the platform for commercial VR devices next month or by January 2018 at the latest.