A Comparison of Signal Quality of Direct Streaming Hearing Aids
Advertorial
The design, algorithms, and overall processing of hearing instruments often is focused on methods to improve speech understanding, especially speech understanding in background noise. It is important, however, that while this important task is being accomplished that sound quality is not sacrificed. Hence, benchmark studies frequently are conducted to assess the speech quality of a given instrument, and to compare this instrument to competing products from other manufacturers. We have previously reported on such a comparative study, conducted at an independent research site, the DELTA SenseLab in Hørsholm, Denmark.1 The premier hearing aids of five other major manufacturers were compared to the Siemens/Signia product. All devices were fitted for a mild-to-moderate hearing loss (IEC-N3 audiogram; 35 dB in lows sloping to 65 dB in the highs). The hearing aids were rated on a bi-polar “like-least” to “like-most” scale, using a MUSHRA methodology (MUltiple Stimuli with Hidden Reference and Anchor). Quality ratings were conducted for several conditions including speech in quiet, in noise, in several environmental background scenarios, and also for music. At the conclusion, ratings were averaged among conditions to obtain an overall preference rating.
Figure 1 shows the preference ratings for the six different hearing aids for overall preference of sound quality, including an average rating for the five competitive models. Observe that the quality ratings for the Signia product were the highest, by a substantial amount when compared to HA-1, HA-2 and HA-4. The mean rating for the Signia product also was considerably higher than the average of the competitive models (8.7 vs. 6.5).
Figure 1. Sound quality preference ratings for the Signia compared to five premier instruments from other leading manufacturers. Preference ratings obtained using MUSHRA methodology.
Sound Quality for Streamed Signals
Since the time of the research findings shown in Figure 1, most major manufacturers, including Signia, have introduced direct streaming using 2.4 GHz technology.2 This advancement provides many daily benefits for the user including the direct streaming from mobile phones, listening to audio signals from the TV, or music from mobile devices—all possible without the use of an intermediary relay. While it is tempting to assume that a hearing aid that has excellent sound quality in general will also have excellent sound quality for a streamed signal, this is not necessarily true. The processing and algorithms required for incorporating a direct streamed signal involve compromises in the encoding and the subsequent signal processing, which can result in negatively alternating the sound quality. This potentially can result in distortions, smearing, artifacts, narrowed-bandwidth, and/or a signal lacking in clarity or richness. It is important, therefore, to conduct baseline studies regarding the sound quality of the streamed signal, just as we do for traditional hearing aid processing. That was the purpose of this research.
Experimental Design
The participants in this research were eight listeners with normal hearing who were experts at audio quality, and making judgment of audio signals. The hearing aids used in the comparative testing were the Signia Pure 13 BT and the current (March, 2017) premier mini-BTE RIC product from four other leading manufacturers. The hearing aids were all programmed for a flat 40 dB hearing loss, for an experienced listener, using the manufacturer-specific proprietary formula. Double domes/power domes were used for the fitting.
Recordings were conducted with the 5 different hearing aid models fitted to the KEMAR, which was located in an anechoic chamber. The different stimuli were directly streamed to the hearing aids using an Apple iPhone 6s (iOS: 10.2). The signals streamed were telephone speech consisting of a male voice reading a story and five different types of music: classical, instrumental, jazz, piano, and pop.
The ratings by the expert listeners were conducted blinded using a randomized approach. As a reference, the participants listened to the original sound file, without hearing aid processing or streaming. Ratings were conducted for the following attributes:
- Speech intelligibility via telephone: Ratings conducted on a 6-point scale—0 =Not Good to 5=Very Good.
- Overall quality music: Ratings conducted on a 5-point scale—0=Bad to 4=Excellent.
Overall sound quality telephone: Ratings conducted on a 5-point scale—0=Bad to 4=Excellent. - Naturalness of speech via telephone: Ratings conducted on a 6-point scale—0=Not Natural to 5=Very Natural.
- Artifact annoyance for music: Ratings conducted on a 6-point scale—0=Audible and Annoying to 5=Not Audible.
- Artifact annoyance telephone speech: Ratings conducted on a 6-point scale—0=Audible and Annoying to 5=Not Audible.
Results
The following series of Figures display the mean findings for the quality ratings for the Signia Pure 13 BT, compared to the other four premier products tested. To facilitate the comparisons, we have averaged the ratings for the four other products, which is labeled “Industry Average” on the charts.
In Figure 2, we show the mean ratings for intelligibility, overall sound quality and naturalness for the streamed signal of speech via the mobile phone. While the results are plotted on a 5-point scale, the intelligibility and naturalness ratings were obtained on a 6-point scale, so only relative, not absolute differences are meaningful. As expected, these expert listeners with normal hearing did not rate intelligibility or quality at the highest level possible, however, for all three measures ratings were significantly higher for the Signia product—for both intelligibility and sound quality the mean for the Signia was nearly 1 point higher than the Industry Average (fair/good versus poor/fair). If we correct for the 5-point versus 6-point scale difference we find that intelligibility and quality ratings were very similar for all products. The biggest advantage for Signia was observed in the “naturalness” domain, where the rating was 3.2 vs. only 1.7 for the Industry Average.
Figure 2. Mean ratings for Signia Pure 13 BT compared to the mean average of four competitive products. Speech intelligibility ratings for telephone streamed speech were conducted using a 6-point scale—0=Not Good to 5=Very Good. The sound quality ratings of telephone speech were conducted using a 5-point scale—0=Bad to 4=Excellent. The naturalness ratings were obtained using a 6-point scale—0=Not Natural to 5=Very Natural.
For many patients, steaming music is the most popular benefit of the 2.4 GHz technology. Given the diversity of music itself, and individual preferences for different types of music, we selected five different sample genres for the ratings: classical, instrumental, jazz, piano, and pop. Shown in Figure 3 are the mean ratings from the experts for the five different music samples for Signia and the Industry Average. Again, we are not seeing mean values for any products that are close to “excellent” (#4 Rating). Recall, as part of the experimental design, the listeners used an original unprocessed/non-streamed sound file as a reference. However, the average values for the Signia product were substantially above the Industry Average. In general, these mean ratings were at or above “Fair” (#3 Rating). By comparison, the Industry Average was only somewhat above “Poor” (#2 Rating). Interestingly, the superiority of Signia appears to be the greatest for listening to piano music, which could be related to a more artifact-free signal, as we discuss in the next section.
Figure 3. Mean ratings for Signia Pure 13 BT compared to the mean average of four competitive products for five different genres of music. Ratings conducted using a 5-point scale—0=Bad to 4=Excellent.
As we mentioned earlier, adding direct streaming using 2.4GHz technology requires alterations of the signal processing which has the potential to create annoying artifacts. The final subjective ratings therefore were related to the presence or absence of these annoying artifacts. Shown in Figure 4 are the mean ratings regarding annoying artifacts for the five different music samples, and also for the telephone transmitted speech signal (0=Audible and Very Annoying, 5=Not Audible). As shown, the Signia product was consistently more “artifact free” than the Industry Average, a full 2.0 scale points better for the important condition of listening on the telephone—1.3 vs 3.4—a 160% improvement!
Figure 4. Mean ratings for Signia Pure 13 BT compared to the mean average of four competitive products for presence of artifacts. Ratings conducted using a 6-point scale—0=Audible and Annoying to 5=Not Audible.
To this point, we have shown that the mean ratings for the Signia product have been substantially higher than the Industry Average for a variety of subjective metrics. The skeptic might question, however, if perhaps there was one competitive product that performed very poorly, and as a result was driving down the Industry Average. To provide a better indication of product differentiation, we examined the mean values for each variable to determine when the Signia was better (or worse) than a competitor’s product. The critical difference value used was .5 (e.g. “Fair/Good” vs. “Good”). This value was based on a Cohen’s d analysis of the data, showing that a difference of .5 would be considered a “moderate” effect for nearly all the comparisons.
Figure 5 shows the distribution of the ratings for the different categories. As shown, the Signia product was always rated superior to Hearing Aids B and D, and nearly always rated superior to Hearing Aid A. In general, ratings for Hearing Aid C were similar, except for annoying artifacts for the telephone conversation, where Signia was rated higher (3.5 vs. 2.6). Most importantly, for the six categories studied, and the 13 different head-to-head comparisons, the Signia product was never rated poorer than any of the other premier products.
Figure 5. Chart illustrating the performance of the Signia product compared to the other four products for 6 different subjective categories. The ratings for Signia are categorized as either “Better Than,” “Equal To” or “Poorer Than” based on a .5 critical difference.
Discussion and Summary
It’s important to mention, that each product initially was programmed to each manufacturer’s proprietary fitting algorithm. It is possible, therefore, that the fitting algorithm itself could have influenced some of these ratings. Our goal, however, was to create a real-world use case. The majority of health care providers use the manufacturer’s proprietary algorithm for the initial fitting. Hence, we believe that these findings are a representative prediction of what is occurring in daily practice. Moreover, it’s unlikely that variances in the programmed frequency-specific gain were responsible for the poor ratings of the other products concerning annoying artifacts.
Overall, using expert listeners, we studied several components of intelligibility and sound quality of the Signia Pure BT direct-streamed signals to ratings for premier instruments of other manufacturers. The Signia product consistently received the highest mean ratings compared to the average of the other four products. This advantage held true for five different genres of music and for a streamed mobile phone speech signal. The Signia product also was rated considerably higher than the other products regarding annoying artifacts. In general, we can conclude that the earlier research showing an industry-high level of sound quality for conventional processing also holds true for directly streamed signals.
References
- Powers T, Beilin J. True advances in hearing aid technology: What are they and where’s the proof? Hear Rev 2013;20(1):32-39.
- Hoydal E. Pure 13 BT, Motion Detection and myControl App - the next step in connectivity. Signia White Paper; 2017.