An Introduction to the Whisper Approach
Editor’s Note: Dr. Schum was specifically invited to submit this article to CanadianAudiologist.ca. While this hearing aid is not yet available in Canada, we are sure to receive questions about it. Like a few other examples, the Whisper Hearing Aid system is only available by subscription which may be problematic for how some provincial health care schemes work, but like the latest generation of computer software, subscriptions rather than a one-time fee may be much more common in the future.
Donald J Schum, PhD
Head of Audiology
Whisper AI
For many decades, the hearing aid industry was pretty much on its own in that the technologies that were required for amplification were pretty much up to us to develop. Hearing aids have processing requirements, fine-tuning resources, reliability needs, and all being limited by a 1-volt technology, such that it was difficult to borrow these technologies from other industries. As an industry, we developed loudspeakers, microphones, amplifiers, batteries, T-coils, etc., to meet our specific demands.
Over the past decade, however, we have been seeing technologies that have been initially developed in other industries being adapted and integrated into hearing aids. One of the significant examples is in the area of connectivity. Wireless techniques that were developed by other industries have allowed hearing aids to go far beyond the traditional approaches of T-coils and FM transmission to connect the hearing aids to audio signals from other sources.
Although this development has allowed traditional hearing aids to expand their function, one area of convergence that speaks right to the heart of why a person would seek out amplification is the incorporation of Artificial Intelligence (AI) into signal processing. Many other areas of healthcare and industry have turned towards AI to solve problems that, until that point in time, could not be optimally addressed. And although various applications that fall under the broad definition of AI have been released into the hearing aid market over the past couple of years (e.g., motion detectors or cloud-based analyses of sound scenes), the powerful technique of Deep Learning is more limited in the marketplace.
Andrew Song and Dwight Crow, the founders of Whisper and experts in AI, started with two basic ideas. First, hearing aids could be more effective by harnessing the power of state-of-the-art AI. And second, hearing aids, like so many other modern AI-based devices, should be able to get better over time. There is no reason why a set of hearing aids purchased today have to work precisely the same way until they wear out 5 or 6 years down the road.
AI has had a significant impact in many other areas of everyday life, and it is beginning to demonstrate its value in hearing health care. Whisper is harnessing its promise and power to address the number one unsolved problem for people with hearing impairment: communication in challenging environments - the sort of environments that bring us so much enjoyment, information, and connection with our favorite people.
What is unique about the Whisper approach? Well, it is focused on going beyond what traditional noise reduction has been able to do in hearing aids. Although adaptive directionality leads to improvements in speech understanding in noise when noise was coming from the back or sides, it has not solved all speech in noise problems. There are many challenging situations in which directionality can provide limited benefit because there is no clear separation between where the noise is generated and where the talker(s) of interest are located. And this problem becomes even more complicated when any amount of reverberation is present.
Traditional noise reduction in hearing aids is based on a long-term change in the response of the devices if there are stable sources of noise detected, especially in frequency regions away from the primary speech information region (approximately 1000-4000 Hz). But these response changes tend to be broadband (over one or more octaves) and are applied and removed gradually (over many seconds). They basically can be seen as overall changes in the frequency response for a given listening environment. And these changes are only updated if the overall nature of the sound scene changes. The decision to apply noise reduction in a given frequency region is based on simple assessments of the listening environment - basically, are there frequency regions in which the ongoing noise level is more stable than the natural moment-to-moment fluctuations in speech? The noise reduction processing algorithms used by most hearing aid manufacturers have not changed significantly over the past two decades.
This is where advanced AI techniques come into play. The Whisper signal processing approach used deep learning to analyze many sound samples to identify the patterns that uniquely constitute human speech. The acoustic patterns that comprise human speech are unique in the world of sound. Human listeners can innately identify speech in contrast to other sounds. But traditional approaches to describe these acoustic differences and then apply processing to separate the two have met with limited success. The Whisper approach employs a real-time AI processor that can execute 300 billion operations per second to use what it has learned about the unique characteristics of speech to separate speech from noise. The amount of processing resources that Whisper has dedicated to speech-in-noise processing is 200 to 300 times greater than other approaches in the hearing aid market.
The Whisper Hearing System is comprised of two RIC earpieces and the Whisper Brain - a palm-sized device that can be easily carried in a pocket, purse, or backpack. The Brain is the hub of the system where the AI-based solution is housed. The Sound Separation Engine in The Brain works to identify the patterns that make human speech unique and pull those signals out of the background noise. The earpieces represent state-of-the-art hearing aids in and of themselves and can function without using the Brain in less challenging, everyday environments. But when the full power of AI processing is required to allow the user to perform to the best of their capability, the Brain works to enhance the performance of the earpieces by directly analyzing and enhancing the audio signal that is passing through the earpieces.
One unique aspect of the Whisper approach is that the system will be upgraded multiple times per year to improve the AI solution and other performance factors of the system. The Whisper system debuted late in 2020, but already in March of this year, the first upgrade was released. That upgrade improved the AI-based approach to speech enhancement in noisy environments and the compression and feedback systems, among other features. The improvements were significant, as over 40% of the users and their loved ones reported improved listening performance.
When we decide on what to include in upgrades, we take input from a variety of sources. Two significant sources of inspiration are the audiologists who are currently fitting the Whisper Hearing System and, of course, the end-users. This allows us to not focus our efforts just on what we would like to improve but also on what professionals and patients have suggested.
Whisper partners with audiologists to bring the highest level of care to those struggling with communication difficulties. A unique aspect of the Whisper Hearing System is that it is provided to patients on a 3-year subscription basis that includes all upgrades and a full warranty and loss/damage coverage. This allows you to reach out to patients who may see this approach as a more affordable way to receive quality hearing care. More importantly, providing the Whisper Hearing System can help frame a practice as being on the leading edge of the newest developments in the hearing care world - offering a system using the newest, most powerful concepts in AI along with the ability to improve over time consistently.
Unlike most new entries into the world of hearing aids, Whisper has taken an approach that emphasizes both the need for excellent solutions to treat hearing loss and that these devices need to be provided by professionals as part of a complete hearing care package. Although the Whisper Hearing System is not currently available in Canada, you may receive inquiries from patients about the system or the company. Hopefully this article provided some insight into the world of Whisper.
For further information, contact Don@Whisper.ai.