Music: Sound or Sight?

With Wayne J. Staab, PhD

It is said that science has not been able to clearly define music. Still, whatever it is, music has been reported to have unique effects on the brain related to learning, memory, emotion, and spirituality. Playing, listening, and creating music involves most cognitive skills. While playing and creating music are important, and although this post prefers to concentrate on the listening aspects of music, it seems that the listening and visual features associated with music are difficult to isolate when it comes to perception.

Some think it is reasonable to consider vision as the major sense because it seems to be the way our view of the world is organized. And, it acts rapidly.

However, those of us in the hearing discipline are more likely to believe that hearing may be the most influential sense. This is because hearing is sensitive to the shortest and most rapid sounds – everything in nature vibrates and produces sounds. In other words, sound vibrations are everywhere, and the ear does not close its eyes and sleep. Real silence never occurs as can be evidenced by experiences in an anechoic chamber and/or other types of isolation tanks. We continue to hear the sounds of the body – heartbeat, breathing, air movement in the ear canals moving the tympanic membrane, and even electrical sounds of the nervous system.

Regardless of one’s ranking of senses, sound and vision are very intertwined. A study by Smith and his colleagues showed that when the brain considers both sight and sound while processing speech, if the two are slightly different, visual cues dominate sounds.¹ This phenomena is called the McGurk effect, named for Scottish cognitive psychologist Harry McGurk, who pioneered studies on the link between hearing and vision in speech perception in the 1970s. The overall message is that the brain requires vision to hear.

Another study further suggests that our judgments about a musical performance might have more to do with sight than sound.² When both trained and untrained musicians observed music competition, judgments about the quality of a music performance depended more on how the musician moved and looked, rather than on how they sounded.

The First Rock Star?

Imagine a performer whipping his long hair around as he played, beads of sweat flying into the audience, and women would swoon or throw their clothes on to the stage. Sound familiar? This would not be Mick Jagger, but Franz Liszt, the nineteenth-century Hungarian pianist who whose theatrical recitals made the composer Robert Schumann say that “a great deal of poetry would be lost” had Liszt played behind a screen.

The Effect of Rhythm

It is known that melody and harmony have wide ranging effects on human behaviour because in the brain, they, along with quality and lyrics, are perceived as movement, meaning, and emotion. But rhythm has a language of its own and seems to have a unique power for healing and neuroplasticity. Examples of this effect are seen in tribal uniting ceremonies, how armies march into battle, how dancers move, how gospel music propels congregations, and how speech punctuation is understood.

In addition, studies have shown that learning and memory can be linked to specific beats in a rhythm. An example is to tie a learning or perception task to the fourth beat to increase cognitive performance. Brain alpha and beta EEG rhythms synchronized to a beat and visual cortex sensory stimulus was greater on a specific beat. In perception tests, details were more likely to be seen on the emphasized beats.

Sound is Complex

Sound waves are extraordinarily complex, to the point that we still have difficulty determining how the brain can comprehend the many factors that contribute to an individual note. Fortunately, the brain makes sound seem simple.

Each musical note consists of a complex series of vibrations (fundamentals and harmonics) that blend into one sound in the brain. Additionally, each type of musical source has physical characteristics related to the production quality of the sound. The sound itself can be analyzed according to numerous physical and psychoacoustical measurements.

The Perception of Sound

A recent study offers a unique insight into the perception of sound.³ When a weak tone is heard, the brain strengthens that tone and suppresses the other frequencies near it, in order to hear it better. Exactly how this “auditory attention filter” functions is not clear, but the end result of being able to pick up relevant cues in the environment, even if they are soft, is commendable. Likewise, certain frequencies of an octave are heard better than are other tones, preparing the brain to hear certain harmonics, even if the fundamental is present or not.

Musical Training and Brain Plasticity

In 1993, a small study by Rauscher, et al. advanced the notion that listening to Mozart’s Sonata for Two Pianos in D Major could boost spatial task performance.⁴ It was soon trademarked the “Mozart effect” and products appeared catering to those interested in fast-tracking their toddlers to the best schools. However, researchers from other laboratories around the world reported difficulty in replicating the effect.

Since then, various studies have demonstrated that music training can have effects on learning, perception, performance, and language. Reports suggest that music training increases brain efficiency with fewer neuronal units required to encode information. It has been reported also to increase other abilities, an effect called metaplasticity.⁵

Music training affects structures more strongly in children (prior to age 7) and are said to last a lifetime. Random samples show structural changes in auditory and motor regions in 15 months of training in children, and those who played musical instruments for 10 years stayed better cognitively. The benefit was not reliant on continued play. Adults involved in music training heard music and speech better,⁶ even in old age, but to a lesser degree. Exercise, plus music, increased cognition with elderly.

Changes in brain function can be observed even within three hours of training, but without continued training, short-term changes are reversible. Lifetime musical practice decreased memory loss. The brain changes that occurred in highly trained musicians included better pitch perception, more cells in motor regions and cerebellum, and a larger corpus callosum connecting the right and left brains.

More recently, the bidirectional effects of music and language were uncovered.⁷ Music training was found to increase language abilities, and certain languages were shown to help musical ability.

It has been reported that people speaking tonal languages (such as Vietnamese and Cantonese that have six different tones) having many high and low pitch patterns that affect meaning, can learn music more rapidly.⁷

References

1. Smith E, Duede S, Hanrahan S, et al. Seeing is believing: neural representations of visual stimuli in human auditory cortex correlate with illusory auditory perceptions. PLoS ONE 2013;8 (9):e73148 DOI: 10.1371/journal.pone.0073148
2. Ball P. Musicians’ appearances matter more than their sound. Nature News; August 19, 2013.
3. Tobias B, Versnel H, Kemnerd C, et al. Octave effect in auditory attention. Proc Nat Acad Sci USA 2013;110(8):15225–230.
4. Rauscher FH, Shaw GL, and Ky KN. Music and spatial task performance. Nature 1993;365(6447):611.
5. Lieff J. Neuroplasticity primer and update. Jon Lieff MD Homepage; 2013. http://jonlieffmd.com/blog/neuroplasticity-primer-and-update.
6. Lieff J. Unique effects of music on the brain. Searching for the mind. Jon Lieff MD Blog; 2013. http://jonlieffmd.com/blog/unique-effects-of-music-on-the-brain.
7. Bidelman G, Moreno S, and Hutka S. Speaking a tonal language (such as Cantonese) primes the brain for musical training. PHYS org.com. 2013. http://medicalxpress.com/news/2013-04-tonal-language-cantonese-primes-brain.html.