Is What I See, What I Imagine? Study Finds Neural Overlap Between Vision and Imagination

The study was conducted by an MUSC research team led by Thomas P. Naselaris, Ph.D., associate professor in the Department of Neuroscience. The findings by the Naselaris team help answer an age-old question about the relationship between mental imagery and vision.

“We know mental imagery is in some ways very similar to vision, but it can’t be exactly identical,” explained Naselaris. “We wanted to know specifically in which ways it was different.”

To explore this question, the researchers used a form of artificial intelligence known as machine learning and insights from machine vision, which uses computers to view and process images.

“There’s this brain-like artificial system, a neural network, that synthesizes images,” Naselaris explained. “It’s like a biological network that synthesizes images.”

The Naselaris team trained this network to see images and then took the next step of having the computer imagine images. Each part of the network is like a group of neurons in the brain. Each level of the network or neuron has a different function in vision and then mental imagery.

While inside the MRI, participants viewed images on a screen and were also asked to imagine images at different points on the screen. MRI imaging enabled researchers to define which parts of the brain were active or quiet while participants viewed a combination of animate and inanimate objects.

Once these brain areas were mapped, the researchers compared the results from the computer model to human brain function.

They discovered that both the computer and human brains functioned similarly. Areas of the brain from the retina of the eye to the primary visual cortex and beyond are both activated with vision and mental imagery. However, in mental imagery, the activation of the brain from the eye to the visual cortex is less precise, and in a sense, diffuse. This is similar to the neural network. With computer vision, low-level areas that represent the retina and visual cortex have precise activation. With mental imagery, this precise activation become diffuse. In brain areas beyond the visual cortex, the activation of the brain or the neural network is similar for both vision and mental imagery. The difference lies in what’s happening in the brain from the retina to the visual cortex.

“When you’re imagining, brain activity is less precise,” said Naselaris. “It’s less tuned to the details, which means that the kind of fuzziness and blurriness that you experience in your mental imagery has some basis in brain activity.”

Naselaris hopes these findings and developments in computational neuroscience will lead to a better understanding of mental health issues.

The fuzzy dream-like state of imagery helps us to distinguish between our waking and dreaming moments. In people with PTSD, invasive images of traumatic events can become debilitating and feel like reality in the moment. By understanding how mental imagery works, scientists may better understand mental illnesses characterized by disruptions in mental imagery.

“When people have really invasive images of traumatic events, such as with PTSD, one way to think of it is mental imagery dysregulation,” explained Naselaris. “There’s some system in your brain that keeps you from generating really vivid images of traumatic things.”

A better understanding of how this works in PTSD could provide insight into other mental health problems characterized by mental imagery disruptions, such as schizophrenia.

“That’s very long term,” Naselaris clarified.

For now, Naselaris is focusing on how mental imagery works, and more research needs to be done to address the connection to mental health.

A limitation of the study is the ability to recreate fully the mental images conjured by participants during the experiment. The development of methods for translating brain activity into viewable pictures of mental images is ongoing.

This study not only explored the neurological basis of seen and imagined imagery but also set the stage for research into improving artificial intelligence.

“The extent to which the brain differs from what the machine is doing gives you some important clues about how brains and machines differ,” said Naselaris. “Ideally, they can point in a direction that could help make machine learning more brainlike.”