The Consumerization of Hearing Health Comes of Age
The consumerization of hearing health technology has arrived and is maturing rapidly. This should surprise no one.
Microphones used to facilitate hands–free phone and video calls have been a standard component in many, if not most, earphones for years. Also, wireless consumer earphones have increasingly incorporated advanced chips with onboard amplifiers, signal processing, and sensors. The similarities to digital hearing aid technology are apparent. Additionally, hearing test apps intended for self-testing by consumers (as well as professional assessment platforms like Shoebox) have been developed for mobile consumer devices. It was only a matter of time before consumer technology manufacturers took the obvious next steps to provide testing, amplification, and hearing enhancement for live sounds in the user's environment. The consumerization of hearing health is likely to speed up once the United States ' Over the Counter Hearing Aid Act of 2017 is implemented (the FDA's publication of proposed guidelines has been pushed back from the August 2020 deadline) rapidly.
Consumer technology companies have taken different approaches to develop hearing health products that run entirely on mobile consumer devices. Recently, major tech companies paid very little attention to providing a database for reliable hearing health data, empirically-validated testing, or hearing enhancement that was openly based on hearing health science. Instead, many companies simply offered hi-fi style audio personalization features of varying degrees of complexity and sophistication. While claims for efficacy in hearing health were not explicitly made, they were often strongly implied but not backed by empirical data.
This article examines different approaches to hearing health by three major mobile consumer technology companies, approaches that display varying degrees of comprehensiveness and audiological rigor. A recent announcement of collaboration between Qualcomm, one of the world’s largest companies and Jacoti, a medically-based hearing software technology company point to near-future products that could serve as effective live hearing devices for people with hearing loss.
Samsung
Nearly 75% of the world’s smartphones run on the open-source Android operating system.1 Android can run on a wide range of hardware from different manufacturers. As a result, capabilities and specifications can vary widely between different models of Android-powered smartphones. The absence of a tightly controlled digital ecosystem was one of many factors that delayed the rollout of direct wireless connectivity between Android phones and hearing aids/cochlear implants a la Apple’s “Made for iPhone” (MFI) hearing aids program. Today, however, a wide range of Android phones can be used wirelessly with compatible hearing instruments for phone calls, audio streaming, or both (“Made for Android” or MFA hearing aids).
Samsung, an enormous multinational corporation with its headquarters in Seoul, South Korea, makes the Android-powered Galaxy, the most popular smartphone in the world.1 Samsung has also been interested in producing hearing health products for quite some time. Rumors that the company was about to announce a hearing aid have circulated for at least 5 years.2
At present, while the company does provide several hearing-health-related options in its accessibility section, including MFA capability, subtitles, and sound recognition, it does not explicitly support hearing health functionality such as the storage of audiograms or sound level monitoring in the Samsung Health app. Nevertheless, the company has focused considerable attention on developing several different ways that users can personalize audio.
Within Galaxy’s Settings app (under "Sound quality and effects"), users are provided with several options to tailor sound that are familiar to anyone who has operated a stereo hi-fi system. These include simple bass/treble controls, a nine-channel equalizer, and other controls. Also, Samsung provides a unique sound personalization feature called Adapt Sound, which can, Samsung claims, "[f]ind the best sound for you and uses it for calls, music, and video."
Adapt sound can be used to personalize audio in one of two ways. The user can select one of several Samsung-provided audio personalization presets labeled by the user's age. Alternately, the user can take a short beep threshold test that, when completed, customizes the audio output of the phone. No information is publicly available regarding how Samsung's presets are derived or how audio compensation is calculated from the test.
While Adapt Sound works with any brand, model, or style of earphones for streaming sound, live environmental amplification and enhancement (“Ambient Sound“) is only available on the Galaxy Buds, Samsung’s true wireless earbuds. (“True wireless earbuds” require neither a cable to connect to a computer, smartphone, or tablet nor to each other).
Apple
Much has been written in the hearing health press regarding the highly successful "Made for iPhone Hearing Aids" program (MFI), which provides direct wireless audio connectivity for audio streaming and control of compatible third-party hearing aids and cochlear implants via Apple's iPhone. Additionally, Apple has shown an increasing public-facing interest in addressing hearing health entirely within its proprietary hardware/software ecosystem.
Unlike Samsung, Apple's Health app provides explicit hearing health support. Inside Apple Health, users can track headphone audio levels to ensure they are within safe limits, store audiograms generated by third-party programs, track data on environmental sound levels, and receive notifications when the background noise is at dangerous levels. Apple Health also provides basic information on understanding hearing loss and why it matters.
Additional functionality directly relevant to people with hearing loss can be found in various places within the general iOS Settings app. Features include RTT capability, vibrating alerts, and even sound recognition. Additional capabilities are offered inside other official Apple apps. For example, FaceTime can recognize whether a given user uses sign language and highlights their screen, a feature that may provide significant benefits to many members of the Deaf community (surprisingly, however, FaceTime supports neither automatic captioning nor CART).
AirPods are Apple’s immensely popular line of true wireless earbuds.3 Until recently, AirPods’ hearing health-related functions were limited to a basic “Transparency Mode” — essentially a pass-through of ambient sounds — and support for Live Listen, a feature which enables a user to set up an iPhone as a remote microphone to pick up, amplify and stream live sound directly into the AirPods or MFI hearing aids.
In October, an update to iOS tapped previously underutilized features of Apple’s H1 audio chip and significantly extended the hearing enhancement capabilities of the AirPods Pro, Apple’s flagship hearable model. There is now a new option in iOS Settings called “Headphone Accommodations.” Here, the user can select an audiogram stored in Apple Health (or Apple’s presets) as a data source for personalizing all sound heard on the AirPods Pro.
Transparency Mode now provides amplification of environmental sounds, similar to a Personal Sound Amplifier Product (PSAP). The audio profile of live amplification (as well as phone calls and audio streaming) can be optionally “tuned" via a user's audiogram or one of three Apple-provided audio profiles.
A user can also decide to forgo the audiogram or presets and customize the Sound of the AirPods Pro herself. Rather than utilizing a test based on pure-tone audiometry, customization is accomplished via a simple set of A/B comparisons of speech and processed, recorded music from which the user chooses her preference.
There is no public information available on how Apple configures the sound enhancement for either phone calls, audio streams, or Transparency mode. Measurements by audiologists of Apple's audio customization suggest that Apple is not relying on common fitting rules for customization, nor are there published data regarding the usefulness of Apple's customization initiatives for people with hearing loss.4,5
In short, the Apple hearing health ecosystem enables the user to passively store hearing health data generated in third party apps that meet Apple's privacy rules for health data storage. Apple does not verify the accuracy of audiological data generated by these apps. These third-party audiograms (or, alternately, a simple self-test) can be used to configure the AirPods Pro with proprietary customization algorithms that appear to differ from traditional audiological fittings.4,5
The Jacoti/Qualcomm Collaboration
The multinational technology company Qualcomm, based in San Diego, is one of the largest players in the market of Systems on a Chip (SOCs) in the world. SOCs are chips that include nearly all the major components of a computer. Specialized SOCs are at the heart of the smartphone, and Qualcomm's offering can be found inside phones from companies like LG, Samsung, Sony, and Nokia.
Qualcomm also markets a line of wireless audio SOCs, including the QCC5100 series, which is intended for companies wishing to manufacture hearables, true wireless earbuds, and other "smart audio" devices. In addition to low power consumption, extensive Bluetooth functionality, sensor support for fitness tracking, and high sound quality, including active noise cancelling, Qualcomm's SOCs include powerful digital signal processing for audio that enables manufacturers to offer extensive sound customization within the earbuds themselves.
In late October, 2020, Qualcomm announced a collaboration with Jacoti, a hearing software technology company. Based in Belgium, Jacoti is registered as a Medical Device Manufacturer by the FDA in the U.S. and is similarly certified in Europe.6 Jacoti was formed with a social mission to make hearing health technology accessible to as many people as possible. The company believes that by leveraging the power and economies of scale that modern consumer audio technology provides, hearing technology can benefit a far wider number of people worldwide than are reached by the current hearing health delivery model.
Jacoti's collaboration with Qualcomm provides a regulated and validated software platform for Qualcomm's QCC5100 SOCs to create next-generation wireless earbuds or hearables with extensive hearing health capabilities. In addition to essential sound enhancement, companies can create devices that can, if the user chooses, be configured to function as hearing devices for live sound amplification and hearing loss compensation. Also, hearing aid technology companies can use the new Qualcomm hearable chips with Jacoti software to create a new line of affordable hearing aids suitable for mild to moderate losses that feature extensive connectivity and lifestyle options beyond current MFI and MFA hearing aid capabilities.
For many years, Jacoti has been known for a suite of iOS and cloud-based applications. Jacoti’s core software hearing technology has been successfully ported to the QCC5100 SOCs. In Jacoti-enabled devices, users will be able to:
- Take a validated hearing self-test in any quiet room (Hearing Center);
- Store the resulting audiogram in the cloud (earCloud) or enter data from an audiologist-run PTA hearing test;
- Use these stored audiograms to generate hearing loss compensation based on standard audiological fitting algorithms (ListenApp).
Testing and fitting will occur not in a user's smartphone, tablet, or computer but in the in-ear device itself. The phone will be used for control functions and to communicate remotely with hearing experts.
Unlike Samsung and Apple, the Qualcomm/Jacoti collaboration explicitly addresses hearing health functionality based on traditional audiological science. For example, earCloud, a full suite of cloud-based audiological tools, is provided for audiologists to personalize audio on Jacoti-enabled consumer devices. All data are stored within a Quality Management System “in compliance with international medical device regulations and standards.”7
The Last Word
Smartphones and other audio-enabled consumer devices are ubiquitous and users, including people with hearing loss, want to take phone calls and stream audio easily wirelessly. Thus, nearly all future hearing devices, even those developed within the traditional hearing health model, are likely to interconnect with audio-enabled consumer technology. Furthermore, since consumer hardware platforms like wearables have advanced audio processing, companies will seek to develop affordable hearing assistance based on consumer technology.8,9
The three companies profiled here illustrate different approaches towards the deployment of audio enhancement features that could help people with hearing loss. As of this writing, Samsung offers several features that may be useful in hearing assistance, but the company is not publicly committed explicitly to calling them "hearing health" options.
Apple appears to be more aggressively pursuing an explicit hearing health strategy for its consumer devices. Furthermore, the recent addition of enhanced Transparency Mode, as well as other new features for the AirPods Pro, makes it reasonable to conclude that Apple is slowly entering the hearing health market with an integrated suite of hardware and software products that could easily be leveraged to address the over-the-counter (OTC) hearing aid market. However, at present, Apple does not attempt to verify the audiological data that can be generated by third-party apps and stored in Apple Health. Also, audiologists cannot access this data within Apple Health or modify it. When it comes to amplification and sound enhancement, Apple appears to be forgoing traditional fitting algorithms in favor of proprietary sound enhancement for live and streamed audio that appears not to be based on audiology.
By contrast, the Jacoti/Qualcomm collaboration is based on an explicit audiological model that supports an internationally regulated hearing health ecosystem. Users can cloud-store audiological data, interact in real-time with a hearing expert who can fine-tune a fitting, and upload those fittings to any Qualcomm hearable that is Jacoti-enabled audiologically valid hearing assistance. This provides hearing health experts (and their clients) with a robust personal database of hearing health information as well as efficacious and safe hearing fittings, all on a consumer platform.
The consumerization of hearing health technology has come of age.
Disclosure
Richard Einhorn has consulted with Jacoti since 2014.
References
1. O’Dea S. Mobile operating systems' market share worldwide from January 2012 to July 2020. Available at: https://www.statista.com/statistics/272698/global-market-share-held-by-mobile-operating-systems-since-2009/ Statista.
2. Hearing Health and Technology Matters, Samsung Hearing Aid Rumors Heat Up: Reports Claim Company Nearing Official Launch, unsigned article, December 28, 2015. Available at: https://hearinghealthmatters.org/hearingnewswatch/2015/samsung-hearing-aid-rumors-heat-up-reports-claim-company-nearing-official-launch-1227/
3. Tankovska H. Global sales volume for true wireless hearables 2018-2020, Statista. Available at: https://www.statista.com/statistics/985608/worldwide-sales-volume-true-wireless-hearables/ 2019.
4. Bailey A. AirPods Pro: REMs test with Headphone Accommodations. Available at: https://forum.hearingtracker.com/t/airpods-pro-real-ear-measurements-on-my-ear/54942 Hearing Tracker. September 24, 2020
5. Haag S. Hearable vs Medical Device: Competition or Opportunity? online presentation. Available at: https://www.ubivent.com/htmlstart/index.html?t=35c49663-ef8f-4f5a-8f6b-93f0a88168fe&te=1# EUHA October 09, 2020.
6. Jacoti BV. Regulatory page from Jacoti’s website, accessed November 4, 2020 at https://www.jacoti.com/regulatory. 2020; Jacoti.
7. Havell C. Qualcomm Technologies support Jacoti hearing technology to help deliver personalized audio experiences, OnQ Blog on the Qualcomm. Available at: https://www.qualcomm.com/news/onq/2020/10/21/qualcomm-technologies-support-jacoti-hearing-technology-help-deliver Qualcomm.
8. Alsop T. Smartphone application processor (AP) vendor market share worldwide 2014-2020, Available at: https://www.statista.com/statistics/233415/global-market-share-of-applications-processor-suppliers/ Statista.
9. Scansen D. Applications Processors – The Heart of the Smartphone. Available at: https://www.engineering.com/ElectronicsDesign/ElectronicsDesignArticles/ArticleID/5791/Applications-Processors-The-Heart-of-the-Smartphone.aspx May 31, 2013 engineering.com