What’s New at the Auditory Neuroscience Research Laboratory at the University of Montreal

By François Champoux, MSc, MScS (A), PhD Marie-Soleil Houde, MScS (A) Simon P. Landry, MSc Maxime Maheu, PhD Marie Simon, MSc (P) Sara Pagé, MPA

---

Hearing and Deafness: Crossmodal Perception and Plasticity

By Marie-Soleil Houde, MScS (A), François Champoux MSc, MScS (A), PhD

 At the crossroads of clinical audiology and scientific research lie many important questions on the brain and hearing. It is these questions that fuel research at the Auditory Neuroscience Research Laboratory of the University of Montreal. The laboratory members’ various areas of expertise in clinical audiology and cutting-edge neuroscientific research complement one-another as we seek a deeper understanding of multisensory integration and the mechanism underlying cortical auditory plasticity. By combining audiological, electrophysiological, oculometric, optoelectrical, posturographical and imaging research methods, we can investigate these clinically relevant issues with intensive scientific rigor. Since its inception, the investigations performed in our laboratory have helped expand understanding of the interaction between the auditory system and the other senses in hearing and deaf individuals.

Multisensory Interaction and Plasticity in the Normally Hearing

A decade ago, a member of our research team helped uncover the multisensory role of the inferior colliculus, a subcortical structure then largely considered unisensory.¹ This finding revealed this structure’s role in multisensory speech processing for the first time. Following this, we investigated the divergent characteristics of audiovisual speech and non-speech processing through the use of various illusory tasks.^2,3 We then furthered these investigations on multisensory interactions by looking beyond the domain of audiovisual integration. These investigations continue to this day as we further explore the influence of auditory input on motor and sensory systems.⁴⁻¹⁰

Crossmodal Perception and Neuroplasticity in the Deaf

Many studies have shown that blindness may lead to functional changes in the remaining intact senses. These researches in the blind have shown that neuroplasticity leads to positive and advantageous outcomes for most of the remaining sensory modalities (e.g., from our research team members and collaborators.¹¹⁻¹⁶ Studies in the deaf, however, suggest that crossmodal neuroplastic changes may vary across modalities.¹⁷ Sensory changes seem to lack uniformity in the deaf, contrary to changes seen in the blind, as they vary between the heightening and lowering of abilities.¹⁴ Though fragmentary, these data have revealed the consequences of crossmodal neuroplasticity in the deaf and have highlighted the consequences of faulty adaptation for rehabilitation.¹⁸ This is particularly true in light of rehabilitative efforts following cochlear implantation. For decades, hundreds of studies have explored auditory performance in cochlear implant (CI) users with no interest for the influence other sensory modalities may have had on these results. The general consensus brought forth by these studies stated that auditory performances were practically unpredictable.¹⁹ However, over the past decade an increasing number of researches have shown that crossmodal plasticity appears to be the key factor in limiting hearing and triggering multisensory conflicts for CI users.²⁰⁻²⁸ These findings have lead to the understanding that crossmodal perception and plasticity can no longer be ignored in rehabilitation strategies in CI users. As of late, there is a critical need for perception-training programs to be adapted based on the nature and extent of crossmodal changes in CI users.²⁵ Unfortunately, knowledge regarding the mechanisms underlying sensory-motor changes following deafness is still strikingly lacking. Even more troublesome, we know even less of what remains of these alterations after the restoration of the auditory inputs through a CI and of the conflicts that they may generate across the senses. In fact, there is still much debate over the identity of the systems that are altered and the mechanisms that mediate adaptive or maladaptive neuroplastic changes following deafness.¹⁷ Needless to say, the factors that may constrain or promote these changes are less understood in the deaf,^14,17,18 and have been practically overlooked in CI users.²⁵ In light of these evidences, or lack thereof, it becomes crucial to unravel the consequence of deafness on both unisensory and multisensory processing in CI users.

The study of crossmodal perception and neuroplasticity is an extremely broad field that requires thorough investigation on multiple levels ranging from neural systems to behaviour.¹⁸ There is a still debate on research concerning the anatomical and structural changes following auditory deafferentation.²⁹ We also know very little regarding functional implications of these neuroplastic changes. Our research aims to rectify this lack of knowledge for the many characteristics of hearing loss in deaf signers and cochlear implant users. With our collaborators, we have intensively investigated these topics at the Auditory Neuroscience Research Laboratory of the University of Montreal, with a particular emphasis on sensory deprived individuals.^{23,24,26,27,28,30−34} We plan to continue and expand on these investigations in the coming years.

The Auditory Neuroscience Research Laboratory of the University of Montreal is currently composed of 5 doctoral students. These students have all demonstrated a high potential for research and are benefactors of some of the most prestigious provincial and federal scholarships (FRQS, CIHR, NSERC). Moreover, their varied backgrounds, which include clinical audiology, neuroscience, and fundamental and clinical neuropsychology, allows for a rich and dynamic mixture of perspectives. The laboratory itself also receives funding from the IRD, CRIR, NSERC, and FRQS.

References

1. Champoux F, Tremblay C, Mercier C, et al. A role for the inferior colliculus in multisensory speech integration. Neuroreport 2006;17:1607–10.
2. Tremblay C, Champoux F, Bacon BA, Théoret H. Evidence for a generic process underlying multisensory integration. Open Behav Sci J 2007;1:1–4.
3. Tremblay C, Champoux F, Voss P, et al. Speech and non-speech audio-visual illusions: a developmental study. PLoS One 2007;2:e742.
4. Champoux F, Lepage JF, Daisy MC, et al. The Neurophysiology of Early Motor Resonance. In: Pineda A (ed). Mirror neuron system: The role of mirroring processes in social cognition. Humana Press 2009;63–76.
5. Houde MS, Bélanger M, Shiller D, Champoux F. Visual deprivation improves auditory scene analysis. J Acoust Soc Am 2012;131:3514.
6. Lepage JF, Lortie M, Champoux F. Action-coding neurons in primary motor cortex: making sense of M1 activity during action perception. J Neurosci 2008; 28:1995–96.
7. Lepage JF, Tremblay S, Théoret H. Early non-specific modulation of corticospinal excitability during action observation. Eur J Neurosci 2010;31:931–37.
8. Lepage JF, Morin-Moncet O, Beaulé V, et al. Occlusion of LTP-Like Plasticity in human primary motor cortex by action observation. PLoS One 2012;7:e38754.
9. Landry SP, Shiller DM, Champoux F. Short-term visual deprivation improves the perception of harmonicity. J Exp Psychol Hum Percept Perform 2013; 39:1503–07.
10. Landry SP, Pagé S, Shiller DM, et al. Auditory imagery forces motor action. Neuroreport 2015;26:101–106.
11. Kauffman T, Théoret H, Pascual-Leone A. Braille character discrimination in blindfolded human subjects. Neuroreport 2002;13:571–574.
12. Gougoux F, Zatorre RJ, Lassonde M, et al. A functional neuroimaging study of sound localization: visual cortex activity predicts performance in early-blind individuals. PLoS Biol 2005;3:e27.
13. Théoret H, Merabet L, Pascual-Leone A. Behavioral and neuroplastic changes in the blind: evidence for functionally relevant cross-modal interactions. J Physiol Paris 2004;98:221–33.
14. Collignon O, Champoux F, Voss P, Lepore F. Sensory rehabilitation in the plastic brain. Prog Brain Res 2011;191:211–31.
15. Voss P, Zatorre RJ. Occipital cortical thickness predicts performance on pitch and musical tasks in blind individuals. Cereb Cortex 2012;22:2455–65.
16. Dormal G, Lepore F, Collignon O. Plasticity of the dorsal "spatial" stream in visually deprived individuals. Neural Plast 2012;2012:687–59.
17. Bavelier D, Neville HJ. Cross-modal plasticity: where and how? Nat Rev Neurosci 2002;3:443–52.
18. Merabet LB, Pascual-Leone A. Neural reorganization following sensory loss: the opportunity for change. Nature Reviews Neuroscience 2012;11:44–52.
19. Peterson NR, Pisoni DB, Miyamoto RT. Cochlear implants and spoken language processing abilities: review and assessment of the literature. Restor Neurol Neurosci 2010;28:237–250.
20. Lee DS, Lee JS, Oh SH, et al. Cross-modal plasticity and cochlear implants. Nature 2001;409:149–50.
21. Lee HJ, Giraud AL, Kang E, et al. Cortical activity at rest predicts cochlear implantation outcome. Cereb Cortex 2007;17:909–17.
22. Doucet ME, Bergeron F, Lassonde M, et al. Cross-modal reorganization and speech perception in cochlear implant users. Brain 2006;129:3376–83.
23. Champoux F, Lepore F, Gagne JP, Theoret H. Visual stimuli can impair auditory processing in cochlear implant users. Neuropsychologia 2009;47:17–22.
24. Landry S, Bacon BA, Leybaert J, et al. Audiovisual segregation in cochlear implant users. PLoS One 2012;7:e33113.
25. Landry S, Lévesque J, Champoux F. Plasticity an obstacle for cochlear implant rehabilitation. Hear J 2012;65:18–20.
26. Landry SP, Guillemot JP, Champoux F. Temporary deafness can impair multisensory integration: a study of cochlear-implant users. Psychol Sci 2013;24:1260–68.
27. Landry SP, Guillemot JP, Champoux F. Audiotactile interaction can change over time in cochlear implant users. Front Hum Neurosci 2014;8:316.
28. Tremblay C, Champoux F, Lepore F, Théoret H. Audiovisual fusion and cochlear implant proficiency. Restorat Neurol Neuropsychol 2010;28:283–91.
29. Frasnelli J, Collignon O, Voss P, Lepore F. Crossmodal plasticity in sensory loss. Prog Brain Res 2011;191:233–49.
30. Champoux F, Shiller D, Zatorre RJ. Feel what you say: An auditory effect on somatosensory perception. PLoS One 2011; 6:e22829.
31. Shiell MM, Champoux F, Zatorre RJ. Enhancement of visual motion detection thresholds in early deaf people. PLoS One 2014;9:e90498.
32. Shiell MM, Champoux F, Zatorre RJ. Reorganization of auditory cortex in early-deaf people: functional connectivity and relationship to hearing aid use. J Cogn Neurosci 2015;27:150–63.
33. Turgeon C, Champoux F, Lepore F, Ellemberg D. Reduced visual discrimination in cochlear implant users. Neuroreport 2012;23:385–89.
34. Turgeon C, Champoux F, Lepore F, Ellemberg D. Deficits in auditory frequency discrimination and speech recognition in cochlear implant users. Cochlear Implants Int 2015;16:88–94.

---

Examining, Predicting, and Improving Cochlear Implant Proficiency

By Marie-Soleil Houde, MScS (A); Maxime Maheu, MScS (A); Marie Simon, MSc (P)

In recent years, increasing attention has been paid to sensory changes in individuals having undergone sensory deprivation. Amongst these individuals are the deaf, some of who have regained partial hearing through the use of a cochlear implant (CI). The current body of research studying how deafness alters the perception of the external world suggests that a prolonged period of deafness can lead to significant alterations in sensory processing.^1,2 This can in turn have a dramatic impact on CI proficiency as greater sensory changes prior to cochlear implantation lead to poorer performances with the device.^3–8 Modern deafness rehabilitation methods integrate these findings and have generally been met with success. Indeed, research data have helped predict functional outcomes prior to cochlear implantation on an individual basis, and have allowed more effective patient counselling and expectation management. These data have also improved our understanding of why some people make better use of their CI while others struggle in specific perceptual situations.

Many researchers have studied the impact of auditory experiences on visual perception, yet very few have investigating how auditory deprived individuals experience other sensory or motor modalities.⁹ It has been proposed that several unexplained daily difficulties observed in the deaf could be related to deficits in these domains (for examples in the tactile and motor domain.^10–12

One of the areas of research at the Auditory Neuroscience Research Laboratory of the University of Montreal is to examine, predict, and optimize CI performance for everyday use. For this, we use behavioural, oculometric, optoelectronic, posturographic, electrophysiological and neuroimaging techniques. These studies are performed in collaboration with the CI programming team at the Raymond-Dewar Institute and can be segmented along two axes.

The first axis aims at a better understanding of the perceptual, anatomical, and structural changes in deaf individuals. Moreover, we aim at uncovering the impact of these changes following auditory recovery by cochlear implantation. The second axis aims at optimizing CI performance. These studies investigate methods to increase CI performance through optimization of CI processor parameters as well as rehabilitation strategies to help promote beneficial neural plasticity.

These data will not only provide valuable fundamental insight on the effects of deafness on the brain, but will also further our understanding of some difficulties experienced by these individuals. For instance, much of the understanding of speech occurs in a multisensory environment in which sensory-motor and auditory cues are present. We believe that identifying behavioural changes in CI users has direct and significant implications for recognizing the speech-related difficulties experienced in day-to-day life. More specifically, our research will help identify the systems altered by deafness and the mechanisms and factors that mediate adaptive or maladaptive changes in CI users. We believe these data to be essential for predicting the functional outcomes of cochlear implantation. Knowledge stemming from this research will allow more effective patient counselling and expectation management, and enable more individualized post-implant rehabilitation strategies.

References

1. Bavelier D, Neville HJ. Cross-modal plasticity: where and how? Nat Rev Neurosci 2002;3:443–52.
2. Collignon O, Champoux F, Voss P, Lepore F. Sensory rehabilitation in the plastic brain. Prog Brain Res 2011;191:211–31.
3. Lee DS, Lee JS, Oh SH, et al. Cross-modal plasticity and cochlear implants. Nature 2001;409:149–50.
4. Lee HJ, Giraud AL, Kang E, et al. Cortical activity at rest predicts cochlear implantation outcome. Cereb Cortex 2007;17:909–17.
5. Doucet ME, Bergeron F, Lassonde M, et al. Cross-modal reorganization and speech perception in cochlear implant users. Brain 2006;129:3376–83.
6. Champoux F, Lepore F, Gagne JP, Théoret H. Visual stimuli can impair auditory processing in cochlear implant users. Neuropsychologia 2009;47:17–22.
7. Landry S, Lévesque J, Champoux F. Plasticity an obstacle for cochlear implant rehabilitation. Hear J 2012;65:18–20.
8. Tremblay C, Champoux F, Lepore F, Théoret H. Audiovisual fusion and cochlear implant proficiency. Restorat Neurol Neuropsychol 2010;28:283–91.
9. Petkova VI, Zetterberg H, Ehrsson HH. Rubber hands feel touch, but not in blind individuals. PLoS One 2012;7:e35912.
10. Nasir SM, Ostry DJ. Speech motor learning in profoundly deaf adults. Nat Neurosci 2008;11:1217–22.
11. Landry SP, Guillemot JP, Champoux F. Temporary deafness can impair multisensory integration: a study of cochlear-implant users. Psychol Sci 2013;24:1260–68.
12. Landry SP, Guillemot JP, Champoux F. Audiotactile interaction can change over time in cochlear implant users. Front Hum Neurosci 2014;8:316.

---

Exploring the Vestibular System: From the Laboratory to the Clinic

By Maxime Maheu, MScS (A), Marie-Soleil Houde, MScS (A)

 The vestibular system, a sort of sixth sense, is often taken for granted when it functions properly.¹ However, for someone with a vestibular disorder, living without proper input from this sense can be devastating. Despite the growing interest in vestibular research, many questions remain unanswered to this day. This human factor is the driving force for the vestibular research performed at Auditory Neuroscience Research Laboratory of the University of Montreal.

One of our main objectives when conducting vestibular research is: “How can this research be easily and effectively translated for clinical application?” To achieve this, we primarily perform scientific investigations using widely available clinical evaluation methods such as the video head impulse test (vHIT) and the vestibular evoked myogenic potential (VEMP).

The vHIT is a recent technique allowing us to assess the function of all six semi-circular canals² and is based on the head impulse technique (HIT). It uses an infrared camera designed to track pupillary movement and a patient-worn gyroscope mounted on goggles in order to compare eye and head velocity.³ This technique is less intrusive than caloric testing and is as reliable as the much more invasive scleral magnetic search coil.⁴

The vestibular evoked myogenic potential (VEMP) is an electrophysiological technique used to assess otolith function and can be evoked using different stimuli such as: sounds, vibrations and electrical stimuli.^5–7 Two types of VEMP responses are used in clinical and research settings: ocular-VEMP (oVEMP) and cervical-VEMP (cVEMP).

The cVEMP is an inhibitory response measured from the ipsilateral sternocleidomastoïd muscle and originates from the saccule. oVEMP is an excitatory response of the inferior oblique eye muscle controlateral to the stimulated ear and is thought to originate from the utricle.^9,10 VEMP responses are proven to be robust indicators of vestibular system integrity and are independent from hearing abilities,^11,12 which allow us to use these techniques to assess peripheral vestibular function for patients suffering from hearing loss.

We are currently using these methods in our laboratory to investigate vestibular functions in various populations with sensorineural hearing loss. For instance, between 20% and 70% of children who present sensorineural hearing loss have concomitant vestibular deficit.⁷ We believe vestibular research in the paediatric population with sensorineural hearing loss to be an important area of scientific investigation for its clinical relevance. Moreover, congenital vestibular deficits can impact gross infant motor development.¹⁴ With the widespread adoption of natal auditory screening program, pediatric vestibular research becomes crucial to help with early identification and development of vestibular compensatory strategies to ensure healthy motor development.

On the other end of the spectrum, it is important to assess vestibular function in an aging population. Risks of falls from loss of balance are more prevalent in hearing impaired older adults.15 This is a research priority not only from an audiology perspective, but also to help alleviate the important financial burden on the global healthcare system associated with falls.¹⁶

Our research team is currently working on projects with rehabilitation centers across the province of Quebec with the focus of helping to increase quality of life of patients suffering from vestibular disorders.

References

1. Yu XJ, Dickman JD, Angelaki DE. Detection thresholds of macaque otolith afferents. J Neurosci 2012;32:8306–16.
2. MacDougall HG, McGarvie LA, Halmagyi GM, et al. The video Head Impulse Test (vHIT) detects vertical semicircular canal dysfunction. PLoS One 2013;8:e61488.
3. Patterson JN, Bassett AM, Mollak CM, Honaker JA. Effects of Hand Placement Technique on the Video Head Impulse Test (vHIT) in Younger and Older Adults. Otol Neurotol 2015;36:1061–68.
4. Rosengren SM, Welgampola MS, Colebatch JG. Vestibular evoked myogenic potentials: past, present and future. Clin Neurophysiol 2010;121:636–51.
5. Singh NK, Kashyap RS, Supreetha L, Sahana V. Characterization of age-related changes in sacculocolic response parameters assessed by cervical vestibular evoked myogenic potentials. Eur Arch Otorhinolaryngol 2014;7:1869–77.
6. Zhou G, Kenna MA, Stevens K, Licameli G. Assessment of saccular function in children with sensorineural hearing loss. Arch Otolaryngol Head Neck Surg 2009;135:40–44.
7. MacDougall HG, McGarvie LA, Halmagyi GM, et al. Application of the video head impulse test to detect vertical semicircular canal dysfunction. Otol Neurotol 2013;34:974–79.
8. Rosengren SM, Govender S, Colebatch JG. Ocular and cervical vestibular evoked myogenic potentials produced by air- and bone-conducted stimuli: comparative properties and effects of age. Clin Neurophysiol 2011;122:2282–89.
9. Curthoys IS. A critical review of the neurophysiological evidence underlying clinical vestibular testing using sound, vibration and galvanic stimuli. Clin Neurophysiol 2010;121:132–44.
10. Colebatch JG, Halmagyi GM. Vestibular evoked potentials in human neck muscles before and after unilateral vestibular deafferentation. Neurology 1992;42:1635–36
11. Colebatch JG, Halmagyi GM, Skuse NF. Myogenic potentials generated by a click-evoked vestibulocollic reflex. J Neurol Neurosurg Psychiatry 1994;57:190–97
12. Rine RM, Cornwall G, Gan K, LoCascio C, O'Hare T, Robinson E, Rice M. Evidence of progressive delay of motor development in children with sensorineural hearing loss and concurrent vestibular dysfunction. Percept Mot Skills 2000;90:1101–12.
13. Lin FR, Ferrucci L. Hearing loss and falls among older adults in the United States. Arch Intern Med 2012;172:369–71.
14. World Health Organization. WHO Global Report on Falls Prevention in Older Age. France: WHO Press: 2007; p. 47.

---

Exploring Brain Mechanisms Modulating Auditory Scene Analysis

By Sara Pagé, MPA, Simon P. Landry, MSc

 Brain plasticity is the process by which structural and functional reorganization occurs in the brain in response to physiological and environmental changes.¹ Since deafness or hearing loss can cause compensatory changes at various levels of the central auditory pathway,² audiologists regularly encounter this seemingly fundamental process. Plasticity can also occurs for individuals who use hearing aids or cochlear implants since the brain must adjust to the new sensory input and compensate for the effects of auditory deprivation to achieve successful rehabilitation.³ For this reason, it is important for audiologist to be aware of this phenomenon, but also for audiology research to investigate the effects and underlying mechanisms of auditory plasticity to keep providing the best possible clinical care.

The process of brain plasticity is not exclusive to hearing impairments and can also occur in individuals who have undergone auditory training such as musicians. In fact, a better understanding of the processes that can lead to superior auditory abilities could prove to be useful for clinical audiologists. Since clinical intervention often aims at improving auditory perception, this better understanding could lead to novel therapeutic approaches. Many research teams around the world are investigating the benefits of long-term musical training on real-world hearing abilities. One important finding suggests that musicians have the ability to better understand speech-in-noise.^4–6 Similar results have been found for older adults where long-term musical training enhanced hearing abilities for musical and non-musical tasks lead to improve speech-in-noise abilities.⁷

Brain plasticity is also responsible for the superior auditory processing typically associated with blindness. In this case, the unused blind visual cortex is recruited by other modalities for non-visual sensory processing.⁸ This type of plasticity gives important insights into the brain’s potential. In fact, research suggests that as little as 90 minutes of visual deprivation could improves frequency and intensity discrimination,⁹ reduces sound localization inaccuracies,¹⁰ and improves perception of concurrent sounds.¹¹ However, the neural substrates underlying these enhancements are currently unknown. Current research at the Auditory Neuroscience Research Laboratory of the University of Montreal is aiming to deepen our understanding of this phenomenon so that the auditory benefits of temporary blindness can be transferred to a clinical setting.

Understanding brain plasticity is a fundamental aspect of audiology. Through it, we can better understand auditory pathologies and optimize rehabilitation approaches. The Auditory Neuroscience Research Laboratory of the University of Montreal has recently acquired a high-density electroencephalograph to this end. This cutting edge tool combines the superior temporal resolution of electroencephalography with high spatial resolution from numerous electrodes to provide a comprehensive recording of brain activity. With this, we aim to shed light on the processes underlying certain enhanced auditory abilities, namely the ones following musical training and temporary short-term visual deprivation, to benefit clinical audiology.

References

1. Pascual-Leone A, Amedi A, Fregni F, Merabet L. The plastic human brain cortex. Annu Rev Neurosci 2005;28:377–401.
2. Butler BE, Lomber SG. Functional and structural changes throughout the auditory system following congenital and early-onset deafness: implications for hearing restoration. Front Syst Neurosci 2013;7:92.
3. Berrettini S, Baggiani A, Bruschini L, et al. Systematic review of the literature on the clinical effectiveness of the cochlear implant procedure in adult patients. Acta Otorhinolaryngol Ital 2011;31:299–310.
4. Parbery-Clark A, Strait DL, Kraus N. Context-dependent encoding in the auditory brainstem subserves enhanced speech-in-noise perception in musicians. Neuropsychologia 2011;49:3338–45.
5. Bidelman GM, Ananthanarayan K. Effects of reverberation on brainstem representation of speech in musicians and non-musicians. Brain Res 2010;1355:112–25.
6. Parbery-Clark A, Skoe E, Kraus N. Musical experience limits the degradative effects of background noise on the neural processing of sound. J Neurosci 2009;29:14100–14107.
7. Alain C, Zendel BR, Hutka S, Bidelman G. Turning down the noise: the benefit of musical training on the aging auditory brain. Hear Res 2014;308:162–73.
8. Frasnelli J, Collignon O, Voss P, Lepore F. Crossmodal plasticity in sensory loss. Prog Brain Res 2011;191:233–49.
9. Gibby RG, Gibby RG, Townsend JC. Short-term visual restriction in visual and auditory discrimination. Percep Motor Skills 1970;30:15–21.
10. Lewald J. More accurate sound localization induced by short-term light deprivation. Neuropsychologia 2007;45:1215–22.
11. Landry SP, Shiller DM, Champoux F. Short-term visual deprivation improves the perception of harmonicity. J Exp Psychol Hum Percept Perform 2013;39:1503–1507.

---

Multisensory Integration: From Senses to Perception

Simon P. Landry, MSc, Sara Pagé, MPA

Tap a pen on a hard surface. Did you hear a sound? Do you know where the sound came from? Of course you did. For all of your life, without you knowing it, your brain has been associating information from all of your senses in a cohesive percept.¹ This is called multisensory integration and it is the process by which our brain takes the sound of something tapping a hard surface and associates it with the sight of a pen tapping a table. Our brains know that if relevant sounds and sights come from the same area in space, they are likely from the same event.² Without multisensory integration, the world would hardly be more than a cacophony of random sights and sounds!

Everyday processes require multisensory integration. For instance, we typically regard speech as an auditory process, but when you speak with someone in a noisy room you are likely to unconsciously use your vision to complement auditory information.³ Why? Because throughout your life, every time you’ve seen someone speak, you’ve heard the words they were speaking. Your brain, through a lifetime of learning, has associated auditory speech with labial movement.

If we learn to integrate senses throughout a lifetime of sensory experiences, then what of people such as cochlear implant users who do not have such experiences? If our interaction with the world is based on the implicit knowledge that a tapping sound is associated with seeing a pen hit a surface, then how does a person who just gained the ability to hear experience the world? This is why multisensory integration, though seemingly extremely fundamental, is essential for a sensory-centered field such as audiology.

For someone who just recently gained the ability to hear, the entire dimension of hearing is independent from everything else. Sound does not have the robust association with vision and touch that it does for most. Watching a dubbed film becomes a challenge. Without auditory experience, all communication comes from vision. Ignoring visual speech information in a dubbed film to focus exclusively on the auditory information becomes a very challenging task.3 Even something as innocuous as shaving requires the integration of sensory information.⁴ How much pressure applied to the skin comes from integrating tactile feedback from holding a razor and auditory information from the blade cutting the whiskers. Does someone who just regained his or her hearing integrate information from touch and hearing in a different way? We have recently investigated this question and found the answer to informative but also more complex than expected.^5,6

Having a deeper awareness of human sensory reality allows us to better understand other people’s realities. Though crucial, multisensory integration risks being taken for granted in the auditory rehabilitation process because of its pervasiveness. The fundamental multisensory research performed at the Auditory Neuroscience Research Laboratory of the University of Montreal aims at understanding the effects of deafness on the brain, but also its ramifications on individual perception and daily life. Through our laboratory’s seemingly fundamental investigations, we can gain great understanding on the way others experience the world.

References

1. Wallace MT, Stein BE. Early experience determines how the senses will interact. J Neurophysiol 2007;97:921–26.
2. Meredith MA, Stein BE. Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 1986;365:350–54.
3. McGurk H, MacDonald J. Hearing lips and seeing voices. Nature 1976;264:746–48.
4. Champoux F, Lepore F, Gagné JP, Théoret H. Visual stimuli can impair auditory processing in cochlear implant users. Neuropsychologia 2009;47:17–22.
5. Foxe JJ. Multisensory integration: Frequency tuning of audio-tactile integration. Curr Biol 2009; 19:R373–R375.
6. Landry SP, Guillemot JP, Champoux F. Temporary deafness can impair multisensory integration: A study of cochlear-implant users. Psychol Sci 2013;24:1260–68.