Visual and auditory stimuli that elicit high levels of engagement and emotional response can be linked to reliable patterns of brain activity, a team of researchers from The City College of New York and Columbia University reports. Their findings could lead to new ways for producers of films, television programs and commercials to predict what kinds of scenes their audiences will respond to.
“Peak correlations of neural activity across viewings can occur in remarkable correspondence with arousing moments of the film,” the researchers said in an article published in the journal “Frontiers in Human Neuroscience.” “Moreover, a significant reduction in neural correlation occurs upon a second viewing of the film or when the narrative is disrupted by presenting its scenes scrambled in time.”
EEG reading indicates high engagement during tense moments of the Alfred Hitchcock suspense thriller “Bang! You’re Dead.”
Credit: CCNY
The researchers used EEG (electroencephalography), which measures electrical activity across the scalp, to collect data on brainwaves of 20 human subjects, who were shown scenes from three films with repeat viewings. Two films, Alfred Hitchcock’s “Bang! You’re Dead” and Sergio Leone’s “The Good, the Bad and the Ugly,” contained moments of high drama expected to trigger responses. The third, an amateur film of people walking on a college campus, was used as a control.
“We found moments of high correlation (between brainwave activity during separate viewings) and moments when this did not occur,” said Dr. Lucas C. Parra, Herbert G. Kayser Professor of Biomedical Engineering in CCNY’s Grove School of Engineering, and a corresponding author. “By looking at patterns of oscillation we could tell at which moments a person was particularly engaged. Additionally, we could see whether the correlation occurred across subjects and repeated viewings.”
Measurements along the EEG alpha activity scale show the degree of attentiveness in a person, he explained. When the oscillations are strong, a person is relaxed, i.e. not engaged. When a person is very attentive, alpha activity is low.
Peaks in engagement were correlated to three kinds of scenes, said Dr. Jasek Dmochowski, a post-doctoral fellow in the Grove School and a corresponding author. They included moments with powerful visual cues, such as a close-up on the gun in “Bang! You’re Dead,” scenes with ominous music in which the visual component was not significant, and meaningful scene changes.
The researchers found significantly less neural correlation on participants’ second viewings and when scenes were scrambled and shown out of sequence. “Following a narrative is complex and involves a lot of distributed processing. When a person doesn’t have a sense of the narrative there is much less correlation (across views of the same or another subject),” Dr. Dmochowski said.
Having demonstrated the correlations between intense stimuli and brainwave reliability, the research team now wants to locate where in the brain the response occurs, Professor Parra said. He wants to deploy a combination of EEG and magnetic resonance imaging to “get the best of both worlds:” the fine temporal resolution of EEG and the detailed imagery of MRI.
The team sees several potential applications for the ability to quantify levels of engagement, including neuro-marketing, quantitative assessment of entertainment, measuring the impact of narrative discourse and the study of attention deficit disorders. “Advertisers would love to know where and when an ad is engaging,” he noted.
“The potential to measure engagement is huge since this provides an objective way to collect data,” added Dr. Dmochowski, who currently is investigating whether there is a correlation between social media usage and brain activity in young people while watching “The Walking Dead,” a drama series on the American Movie Classics cable network.
“We are mining Twitter to measure the depth of watching,” he continued. “We think there will be many correlations between scenes that elicit social media responses and neural signatures, and we can look at both positive and negative responses.”
The study was funded by a Defense Advanced Research Program Agency grant. Joao Dias of City College and Paul Sajda of Columbia University collaborated with Professor Parra and Dr. Dmochowski on the research.
Contacts and sources:
Ellis Simon
City College of New York
