To the Brain and Back: The Role of Visual Neuroplasticity in Cochlear Implant Users’ Speech Outcomes

By Brandon T. Paul, PhD

Cochlear implants (CIs) can restore hearing function from deafness or profound hearing loss, but CI recipients’ long-term speech outcomes can vary widely. Some recover speech perception rapidly and show good listening performance in noisy environments. Others may rehabilitate slowly, struggle with simple words in quiet, or even fail to recover functional speech perception. Research has shown that several factors, such as the cause or duration of deafness, the amount of residual hearing, or whether deafness occurred before or after developing language, correlate with CI users’ long-term speech perception ability. However, much variation in CI users’ speech perception remains unexplained.¹

Researchers have speculated that neural plasticity –  the brain’s ability to change or alter its structure or function – may be a major factor underlying CI recipients’ speech outcomes. CI users’ brains undergo plasticity across two major stages. First, the onset of deafness or profound hearing loss, whether congenital or occurring later in life, alters brain communication. If information is not flowing from the ear to the auditory cortex, these neurons will respond to input from other sources. A second stage of neural plasticity happens after a person receives a CI. Once the CI is activated, the brain changes as it adapts to the input driven by electrical stimulation of the auditory nerve.² Factors such as the CI mapping, how often a person uses their device, and a person’s age may influence how the brain responds to CI use.¹

One major source of neuroplasticity may involve non-auditory sources, such as vision or touch. Interestingly, several studies since 2001 have shown that CI users’ auditory-only speech ability correlates with the size or speed of brain responses to visual-only events.^3,4 Most researchers agree that these correlations could be related to how the brain compensates for hearing loss, such as by relying on the sense of vision to navigate the world or communicate. However, this plasticity is not limited to the visual system itself. Visual processing may emerge in the auditory cortex after it is disconnected from the ear. More specifically, neurons outside of the primary auditory cortex (often referred to as “higher-order” auditory cortex) are not always exclusively “auditory.” They connect to other senses such as vision or touch, but these outside sensory inputs are usually “masked” by normal auditory activity in typical hearing individuals. In other words, these higher-order auditory cortical neurons preferentially respond to sound information but have weak connections to other senses. When input stops flowing from the ear due to deafness or hearing loss, these latent connections become “unmasked.” For example, neurons deprived of auditory input may begin to prefer visual inputs (see Figure 1). This process, which can be described as a form of cross-modal plasticity, showcases the remarkable adaptability of the brain.⁵

The contention, however, is whether this cross-modal plasticity is harmful or helpful for speech perception.⁶ If auditory neurons are busy with visual processing, they may no longer be available for sound processing once a person receives a CI. In other words, visual processing may “take over” auditory cortical areas, at the detriment of sound perception. Several studies show that CI users with larger or faster cross-modal brain responses (responses coming from auditory cortex but elicited by visual events) have worse long-term speech outcomes.^4,7 This would agree that cross-modal plasticity is detrimental to speech perception. As a result, there have been recommendations for CI candidates to limit visual language use if they want to reduce the risk of poor long-term speech perception.⁸ However, the body of evidence suggesting that cross-modal plasticity is maladaptive consists mainly of observational, cross-sectional, and correlational studies that cannot establish causal relationships nor reveal how stages of neural plasticity could interact. Furthermore, these studies typically evoke visual brain responses by presenting unnatural stimuli such as flickering checkerboards or artificial motion stimuli that are not relevant to human speech.

Results from more recent studies argue that cross-modal plasticity may be helpful for long-term speech recovery. Anderson and colleagues performed a longitudinal study and used functional near-infrared spectroscopy (fNIRS) to measure cross-modal responses before and after patients received their CI.⁹ The stimuli used were silent videos of real human speech, which is more relevant for speech processing than artificial visual patterns. They found that CI users with comparatively more cross-modal plasticity in the superior temporal lobe had better long-term speech perception scores, arguing that cross-modal plasticity could aid long-term speech outcomes. If auditory areas undergo cross-modal changes that support visual speech perception, it may help to create a template to process auditory speech delivered through a CI. Similar results using this method have been shown in children.¹⁰ Furthermore, my work with Sunnybrook Hospital agrees with these studies. We used EEG to measure brain responses to a silent video of a mouth speaking a single-syllable word. Individuals with larger cross-modal responses had better speech scores, reinforcing the view that cross-modal plasticity for speech-relevant stimuli may benefit speech outcomes.¹¹

When these conflicting findings are taken together, we cannot firmly conclude that visual brain activity “takes over” the auditory system in deafness and hearing loss, leading to poor speech function. Therefore, no evidence basis from neuroscience supports limiting a person’s use of visual language if they intend to receive a CI. More research is needed to address the following questions: Does cross-modal plasticity in CI users differ for speech-relevant and speech-irrelevant visual stimuli?¹² Can visual speech training help CI rehabilitation?¹³ Would intervention that shapes visual plasticity before CI surgery help long-term speech outcomes? Is there any role for the somatosensory (touch) system in speech recovery?

The bottom line is that CI recipients’ speech outcomes are influenced by neuroplastic interactions between the auditory and visual systems, and these interactions unfold both before and after CI surgery. Neuroimaging like fNIRS and EEG may be essential to monitor visual neural responsiveness in CI candidates. A priority is understanding if we can shape visual plasticity, for example, through audiovisual speech training or lip reading, to maximize speech communication in CI recipients.

---

References

1. Blamey P, Artieres F, Başkent D, et al. Factors affecting auditory performance of postlinguistically deaf adults using cochlear implants: An update with 2251 patients. Audiology and Neurotology. 2012;18(1). doi:10.1159/000343189
2. Kral A. Unimodal and cross-modal plasticity in the “deaf” auditory cortex. Int J Audiol. 2007;46(9). doi:10.1080/14992020701383027
3. Lee DS, Lee JS, Oh SH, et al. Cross-modal plasticity and cochlear implants. Nature. 2001;409(6817). doi:10.1038/35051653
4. Sandmann P, Dillier N, Eichele T, et al. Visual activation of auditory cortex reflects maladaptive plasticity in cochlear implant users. Brain. 2012;135(2). doi:10.1093/brain/awr329
5. Makin TR, Krakauer JW. Against cortical reorganisation. Elife. 2023;12. doi:10.7554/eLife.84716
6. Wallace MT. Cooperation between hearing and vision in people with cochlear implants. Proc Natl Acad Sci U S A. 2017;114(38). doi:10.1073/pnas.1712810114
7. Buckley KA, Tobey EA. Cross-modal plasticity and speech perception in pre- and postlingually deaf cochlear implant users. Ear Hear. 2011;32(1). doi:10.1097/AUD.0b013e3181e8534c
8. Lyness CR, Woll B, Campbell R, Cardin V. How does visual language affect crossmodal plasticity and cochlear implant success? Neurosci Biobehav Rev. 2013;37(10). doi:10.1016/j.neubiorev.2013.08.011
9. Anderson CA, Wiggins IM, Kitterick PT, Hartley DEH. Adaptive benefit of cross-modal plasticity following cochlear implantation in deaf adults. Proc Natl Acad Sci U S A. 2017;114(38). doi:10.1073/pnas.1704785114
10. Zhou XQ, Zhang QL, Xi X, et al. Cortical responses correlate with speech performance in pre-lingually deaf cochlear implant children. Front Neurosci. 2023;17. doi:10.3389/fnins.2023.1126813
11. Paul BT, Bajin MD, Uzelac M, et al. Evidence of visual crossmodal reorganization positively relates to speech outcomes in cochlear implant users. Sci Rep. 2022;12(1). doi:10.1038/s41598-022-22117-z
12. Fullerton AM, Vickers DA, Luke R, et al. Cross-modal functional connectivity supports speech understanding in cochlear implant users. Cerebral Cortex. 2023;33(7). doi:10.1093/cercor/bhac277
13. Tye-Murray, N, et al. Teaching children with hearing loss to recognize speech: Gains made with computer-based auditory and/or speechreading training. Ear and hearing. 2022 43(1): 181-191.