Scientists have trained a computer to analyze the brain activity of someone listening to music and, based only on those neuronal patterns, recreate the song.

The research produced a recognizable, if muffled version of Pink Floyd’s 1979 song, “Another Brick in the Wall (Part 1).”

Before this, researchers had figured out how to use brain activity to reconstruct music with similar features to the song someone was listening to. Now, “you can actually listen to the brain and restore the music that person heard,” said Gerwin Schalk, a neuroscientist who directs a research lab in Shanghai and collected data for this study. [1]

Figure 1. Scientists Recreated a Pink Floyd Song From Listeners’ Brain Waves

The recreated bars sound garbled and hazy—a distorted echo of the original track. But unmistakable elements of the song’s rhythm, melody and harmony shine through.

“It’s a technical tour de force,” Robert Zatorre, a neuroscientist at McGill University in Canada who did not contribute to the findings, says of the research to the New York Times’ Hana Kiros.

“These exciting findings build on previous work to reconstruct plain speech from brain activity,” Shailee Jain, a neuroscientist at the University of California, San Francisco, who was not involved in the research, tells Scientific American’s Lucy Tu. “Now, we’re able to really dig into the brain to unearth the sustenance of sound.”

Researchers have been working at decoding brain activity with artificial intelligence for years. They’ve tried reading brain scans to determine which words people are listening to, and they’ve even attempted to translate entire stories. Another study aimed to reproduce images that participants looked at.

Such technology could one day be used to help people who are unable to communicate with spoken words. Ludovic Bellier, a neuroscientist at the University of California, Berkeley, and co-author of the new study, tells Science’s Phie Jacobs that he hopes the findings could eventually help people who have trouble speaking due to strokes, injuries or diseases, by making sense out of their brain activity.

For the new study, researchers played the Pink Floyd song to 29 participants with epilepsy. As treatment for their epilepsy, the participants already had electrodes implanted in their brains, per the Times. The song played in the operating room while the patients underwent surgery meant to prevent seizures, according to Fortune’s Erin Prater.

The researchers trained a computer model on the brain data from participants as they listened to about 90 percent of the Pink Floyd song. But the remaining 10 percent—a 15-second clip from the middle of the track—was left out of the training data, writes Science. Instead, the team asked the algorithm to recreate this section of the music from the brain activity based on patterns it had learned. The team trained 128 models, each operating at a different frequency, and together, they matched specific electrode signals to certain characteristics of music, per the Times.

In the future, the researchers hope their insights could help devices translating brain signals into words to incorporate the more musical elements of speech. Language, like music, includes changes in pace, pitch and volume that are a vital part of communicating, per Scientific American.

“These elements, which we call prosody, carry meaning that we can’t communicate with words alone,” Bellier tells the publication.

The researchers chose the Pink Floyd song for this study, in part because it contains a mix of sung words and instrumental sections, according to the Times. But they had another reason, too: The participants “just love Pink Floyd,” Bellier tells Science.[2]

Electrodes implanted onto the brain’s surface

While promising, there are major hurdles to be passed before the kind of technology used can come to the hands of consumers – perhaps none greater than placement of electrodes on the brain itself.

Currently available technology to record the requisite brain waves is simply not sensitive enough to work on the scalp itself. This makes surgical implants a must, something that would dissuade the average consumer.

The 29 test subjects were all suffering from epilepsy and already had implants inserted onto their brains to determine the cause of the seizures. The researchers were able to get consent from these patients to carry out this study.

Moreover, even after placing electrodes directly on to the brain, the music was still somewhat garbled due to the number of electrodes that were packed in. The researchers hope that by increasing the density of electrodes on the brain (number of electrodes/surface area), they will be able to obtain even better resolution.[3]

References:

1. https://www.nibib.nih.gov/news-events/newsroom/scientists-recreate-pink-floyd-song-reading-brain-signals-listeners
2. https://www.smithsonianmag.com/smart-news/scientists-recreated-a-pink-floyd-song-from-listeners-brain-waves-180982746/
3. https://indianexpress.com/article/explained/explained-sci-tech/pink-floyd-reconstructed-brain-activity-8897242/

Cite this article:

Gokula Nandhini K (2023), Scientists Recreated a Pink Floyd Song From Listeners’ Brain Waves, AnaTechmaz, pp.556