 Empirical Research
 Open Access
 Published:
Robust single and multiloudspeaker leastsquaresbased equalization for hearing devices
EURASIP Journal on Audio, Speech, and Music Processing volume 2022, Article number: 15 (2022)
Abstract
To improve the sound quality of hearing devices, equalization filters can be used to achieve acoustic transparency, i.e., listening with the device in the ear is perceptually similar to the open ear. The equalization filter needs to ensure that the superposition of the equalized signal played by the device and the signal leaking through the device into the ear canal matches a processed version of the signal reaching the eardrum of the open ear. Depending on the processing delay of the hearing device, combfiltering artifacts can occur due to this superposition, which may degrade the perceived sound quality. In this paper, we propose a unified leastsquaresbased procedure to design single and multiloudspeaker equalization filters for hearing devices aimed at achieving acoustic transparency. To account for nonminimum phase components, we utilize a socalled group delay compensation. To reduce combfiltering artifacts, we propose to use a frequencydependent regularization. Experimental results using measured acoustic transfer functions from a multiloudspeaker earpiece show that the proposed equalization filter design procedure results in robust acoustic transparency and reduces the impact of combfiltering artifacts. A comparison between single and multiloudspeaker equalization shows that for both cases a robust equalization performance can be achieved for different desired open ear transfer functions.
Introduction
Despite major improvements in hearing device technology in the past decades, the acceptance of hearing aids and assistive listening devices is still rather limited, partly due to a suboptimal sound quality [1–3]. This is most prominent in firsttime users and users with normal hearing or mildtomoderate hearing loss. While these users would benefit from advanced hearing device processing like noise reduction, dereverberation and dynamic range compression, they usually do not accept degradations of the sound quality. In order to improve the sound quality, equalization algorithms have been proposed that aim at achieving socalled acoustic transparency [4–8], i.e., listening with the device inserted in the ear achieves a similar perceptual impression as listening without the device inserted.
Generally, equalization algorithms for acoustic transparency aim at matching the sound pressure reaching the eardrum when the device is inserted in the ear (aided ear) with the sound pressure at the eardrum when the device is not inserted (open ear) [4, 6]. For the open ear, the sound pressure at the eardrum only consists of the direct sound. In contrast, for the aided ear the sound pressure at the eardrum consists of the superposition of the direct sound leaking into the (partially) occluded ear canal and the sound picked up by the microphone(s) of the device, processed and played back by the loudspeaker(s) of the device. Since the sound played back by the device is typically delayed compared to the direct sound, socalled combfiltering effects frequently occur which may degrade the perceived sound quality [3, 9, 10]. Several equalization algorithms for hearing devices have been proposed in the literature [4, 5, 7, 11–14]. However, often either the direct sound component was neglected in the equalization filter design, e.g., [5], or electroacoustic components were neglected, e.g., [7]. Additionally, it may be desirable to include knowledge about the advanced hearing device processing when designing the equalization filter, e.g., when a hearing loss needs to be compensated. However, including knowledge about advanced hearing device processing has often been neglected in previous research, e.g., [5, 7, 13, 14]. In this paper we propose to include information about both the direct sound component as well as the hearing device processing in the equalization filter design.
Equalization in hearing devices is commonly performed using a single loudspeaker [6], i.e., a single equalization filter is computed to match the sound pressure of the aided ear and the open ear. Computing this equalization filter usually requires the inversion of the (estimated) acoustic transfer function (ATF) between the hearing device loudspeaker and the eardrum. However, since this ATF typically has zeros inside and outside the unit circle, perfect inversion with a stable and causal filter cannot be achieved [15, 16]. Hence, approximate solutions are required to obtain a good equalization filter when using a single loudspeaker [4, 5, 7, 13, 14], e.g., equalizing only the minimum phase component [4, 7] or by including a socalled acausal delay [13, 14]. On the contrary, using multiple loudspeakers perfect equalization can be achieved when the conditions of the multipleinput/output inverse theorem (MINT) are satisfied [16]. Briefly, MINT states that perfect inversion of a multichannel system can be achieved if all channels are coprime, i.e., they do not share common zeros, and the equalization filters are of sufficient length. However, since multiloudspeaker equalization using MINT is known to be very sensitive to small changes in the ATFs [17], regularization is commonly applied to increase the robustness [18, 19] or other optimization criteria are considered [20, 21]. Multiloudspeaker equalization for acoustic transparency in hearing devices was considered in [12], where the equalization filters were shown to exhibit system common zeros introduced by the system design, rendering the application of MINT difficult.
In this paper we propose a unified procedure to design an equalization filter that can be applied when using either a single loudspeaker or multiple loudspeakers to achieve acoustic transparency. The equalization filter is computed by minimizing a leastsquares cost function, where we show that for the considered scenario the multiloudspeaker system exhibits common zeros introduced by the system design. Since these system common zeros are, however, exactly known, we propose to exploit this knowledge and reformulate the optimization problem accordingly. In order to account for potential nonminimum phase components, we incorporate an acausal delay for group delay compensation in the filter computation, similarly as proposed for singleloudspeaker equalization in [13, 14]. Furthermore, to counteract combfiltering effects we include a frequencydependent regularization term to reduce the hearing device playback when the leakage signal and the desired signal at the eardrum are of similar magnitude, similarly as proposed for singleloudspeaker equalization in [13]. While regularization can also be used to increase the robustness of the equalization filters to unknown acoustic transfer functions, in this paper we improve the robustness by considering multiple sets of measurements in the optimization. While the idea of combining group delay compensation and frequencydependent regularization for singleloudspeaker equalization was already presented in [13] and a leastsquaresbased design procedure for multiloudspeaker equalization was already presented in [12], the main objective of this paper is to present a unified procedure (incorporating group delay compensation, frequencydependent regularization, multiple measurements) that can be used both for singleloudspeaker as well as multiloudspeaker equalization.
Experimental results using measured ATFs from a multiloudspeaker earpiece depicted in Fig. 1 show that the proposed single and multiloudspeaker equalization approach is able to achieve almost perfect equalization. Furthermore, we show that the equalization performance depends on the gain and the processing delay of the hearing device. By incorporating the frequencydependent regularization, the effect of combfiltering in the lower frequency region can be considerably reduced. Furthermore, robust equalization can be achieved by considering multiple sets of measurements when computing the equalization filter. A performance comparison between single and multiloudspeaker equalization shows that robust equalization can be achieved independent of the number of loudspeakers.
The remainder of this paper is organized as follows. In Section 2 we describe the considered hearing device setup. In Section 3 we analyze the singlemicrophonemultipleloudspeaker scenario with respect to the processing parameters of the hearing device. In Section 4 we present the robust single and multiloudspeaker equalization filter design procedure using a regularized leastsquares cost function. In Section 5 the proposed equalization filters are experimentally evaluated using either a single loudspeaker or using multiple loudspeakers.
Scenario and problem statement
Consider a singlemicrophonemultiloudspeaker hearing device with N loudspeakers as depicted in Fig. 2. For simplicity we assume that all transfer functions are linear and timeinvariant and that they can be modeled as polynomials in the variable q [24]. Furthermore, we assume the absence of acoustic feedback, i.e., the coupling of the loudspeaker signal (u[k]) into the microphone signal y[k]. We assume that the signal y[k] picked up by the microphone of the hearing device is the signal emitted from a single directional sound source s[k], i.e.,
where k denotes the discretetime index and H_{m}(q) denotes the ATF between the source and the microphone of the hearing device, i.e.,
with q a vector of delay elements and the L_{H}dimensional impulse response (IR) vector h_{m} of H_{m}(q) given by
where (·)^{T} denotes the transpose operation. The microphone signal y[k] is processed by the forward path G(q) of the hearing device, which accounts for potential advanced processing and the processing delay of the device, yielding the intermediate signal \(\tilde {u}[k]\), i.e.,
with the L_{G}dimensional IR vector g of G(q) defined similary as h_{m} in (3). The intermediate signal \(\tilde {u}[k]\) is used as the input to N equalization filters A_{n}(q),n=1,…,N, yielding the Ndimensional loudspeaker signal vector u[k], i.e.,
with the L_{A}dimensional equalization filter coefficient vector a_{n} of A_{n}(q) given by
Furthermore, we define the NL_{A}dimensional vector of stacked equalization filter coefficient vectors as
For the aided ear, i.e., when the device is inserted and playing back the processed microphone signal, the signal t_{aid}[k] at the eardrum of the listener is the superposition of the loudspeaker signals and the signal leaking into the (partially) occluded ear canal, i.e.,
where H_{occ}(q) denotes the ATF between the source and the eardrum for the occluded ear, i.e., with the hearing device inserted and processing off, with h_{occ} the L_{H}dimensional IR vector of H_{occ}(q), defined similarly as h_{m} in (3). The Ndimensional vector D(q) contains the ATFs between the loudspeakers of the hearing device and the eardrum, i.e.,
with d_{n} the L_{D}dimensional IR vector of D_{n}(q), defined similarly as h_{m} in (3).
The desired signal at the eardrum t_{des}[k] is the signal reaching the eardrum of the listener when the device is not inserted (open ear), processed with the forward path of the device, i.e.,
where H_{open}(q) denotes the ATF between the source and the open ear, with h_{open} the L_{H}dimensional IR vector of H_{open}(q). It should be noted that the forward path G(q) is included in t_{des}[k] as otherwise the equalization filter would (partially) compensate for any additional signal processing applied by G(q). In order to achieve acoustic transparency, the goal is to obtain the equalization filters a in (7) such that the signal t_{aid}[k] in (8) is perceptually not distinguishable from the signal t_{des}[k] in (10), accounting for small variations in the ATFs, e.g., due to different positions of the hearing device in the ear.
Transfer function analysis
In this section, we analyze the considered singlemicrophone multiloudspeaker system in terms of its system transfer function between the source and the eardrum. For the aided ear, the system transfer function is obtained by combining (1), (4), (5), and (8), leading to
Similarly, for the desired open ear transfer function, the system transfer function is obtained from (10) as
By equating (11) and (12), we observe that the optimal equalization filter needs to fulfill
which corresponds to
It should be noted that the optimal equalization filter in (14) depends on the relative transfer functions (RTFs) \(\frac {H_{open}(q)}{H_{m}(q)}\) and \(\frac {H_{occ}(q)}{H_{m}(q)}\), i.e., the RTF between the eardrum and the microphone when the device is not inserted (open ear) and the RTF between the eardrum and the microphone when the device is inserted and switched off (occluded ear). Furthermore, the optimal filter in (14) depends on the forward path G(q). In order to analyze the dependency on the forward path, in the following we consider two extreme cases:

1.
Assuming no leakage component (H_{occ}(q)=0), e.g., when the ear canal entrance is blocked completely by the device, the optimal equalization filter needs to fulfill
$$\begin{array}{*{20}l} \mathbf{D}^{T}(q)\mathbf{A}(q) = \frac{H_{open}(q)}{H_{m}(q)}, \end{array} $$(15)such that H_{aid}(q)=G(q)H_{open}(q).

2.
Assuming that t_{des}[k]=0, i.e., H_{open}(q)=0, e.g., when sound pressure minimization at the eardrum is desired, the optimal equalization filter aims at actively suppressing the leaking component at the eardrum, i.e.,
$$\begin{array}{*{20}l} \mathbf{D}^{T}(q)\mathbf{A}(q) =  \frac{H_{occ}(q)}{H_{m}(q)}\frac{1}{G(q)}, \end{array} $$(16)such that H_{aid}(q)=0.
The above analysis shows that for large forward path gains, equalization to the open ear ATF becomes more important than suppression of the leakage component, whereas for small forward path gains the leakage component dominates and needs to be actively suppressed. Furthermore, depending on the delay of G(q), the equalized transfer function in (14) may become increasingly acausal due to the term \(\frac {1}{G(q)}\), which may impact the equalization performance. Additionally, the processing delay that can be allowed for these two cases is substantially different. For large forward path gains as in case 1, a large processing delay of a few milliseconds can be tolerated [9], while to achieve the desired active suppression in case 2, only very low processing delays of only a few microseconds can be tolerated [27].
Equalization filter design procedure
In this section, we present a regularized leastsquaresbased design procedure to compute the equalization filter A(q) in the timedomain. It should be noted that the same design procedure can be applied using either a single (N=1) or multiple (N>1) loudspeakers. In the following we assume knowledge of all required ATFs, e.g., by measurement. Alternatively some ATFs could also be estimated, e.g., an estimate of the open ear ATF between the source and the eardrum H_{open}(q) could be obtained by an appropriate correction function of the ATF between the source and the microphone H_{m}(q) [26] or an estimate of the ATFs between the loudspeakers and the eardrum D(q) could be obtained using an inear microphone and an electroacoustic model [28]. However, the investigation of such estimation procedures is beyond the scope of this paper.
In Section 4.1, we formulate the leastsquares cost function for the equalization filter according to (13) and show that the transfer functions to be equalized share common zeros introduced by the system design. Since these system common zeros are exactly known, in Section 4.2, we exploit this knowledge and reformulate the leastsquares cost function for the equalization filter according to (14), i.e., based on RTFs instead of ATFs. To account for potential acausalities in the filter design, in Section 4.3, we incorporate an acausal delay in the optimization. In addition, in Section 4.4, we include a frequencydependent regularization to reduce combfiltering effects. Finally, in Section 4.5, we include multiple measurements to increase the robustness of the equalization filters to variations, e.g., due to different positions of the hearing device in the ear.
Optimal equalization filter using ATFs
The expression of the optimal equalization filter A(q) in (13) can be reformulated using matrixvector notation as
where C is an (L_{C}+L_{A}−1)×NL_{A}dimensional matrix, with L_{C}=L_{G}+L_{H}+L_{D}−2, defined as
where D is the (L_{D}+L_{A}−1)×NL_{A}dimensional matrix of concatenated (L_{D}+L_{A}−1)×L_{A}dimensional convolution matrices D_{n} of the IR vector d_{n}, i.e.,
H_{m} is the (L_{H}+L_{D}+L_{A}−2)×(L_{D}+L_{A}−1)dimensional convolution matrix of the IR vector h_{m}, and G is the (L_{C}+L_{A}−1)×(L_{H}+L_{D}+L_{A}−2)dimensional convolution matrix of the IR vector g. Furthermore, v is the (L_{C}+L_{A}−1)dimensional vector of the desired equalization hearing device output
where \(\tilde {\mathbf {h}}_{open}\) is the (L_{H}+L_{D}+L_{A}−2)dimensional zeropadded vector of the IR vector h_{open} and \(\tilde {\mathbf {h}}_{occ}\) is the (L_{C}+L_{A}−1)dimensional zeropadded coefficient vector of the IR vector h_{occ}.
The NL_{A}dimensional equalization filter coefficient vector a is then obtained by minimizing the following leastsquares cost function
The optimal solution minimizing (22) is equal to
where (·)^{†} denotes the pseudoinverse of a matrix.
Optimal equalization filter using RTFs
Since the rows of the matrix C in (18) are linearly related by the matrix GH_{m}, the matrix C is not of full rowrank. In order to mitigate this rankdeficiency^{Footnote 1}, we propose to left multiply both C and v by the pseudoinverse of GH_{m} (assumed to be of full columnrank), i.e.,
which is equivalent to writing (14) using matrixvector notation. It should be noted that \(\tilde {\mathbf {v}}\) in (24) represents an RTF, i.e., an infinite impulse response filter, which cannot be perfectly modeled using a finite impulse response filter and hence perfect equalization is not possible. Nevertheless, the leastsquares cost function in (22) can now be reformulated using D and \(\tilde {\mathbf {v}}\) instead of C and v, i.e.,
However, since the ATFs between the loudspeakers and the eardrum D(q) are likely to share nearcommon zeros due the close proximity of the loudspeakers in the considered hearing device (cf. Fig. 1), the matrix inversion when using D in (27) is typically illconditioned. In order to mitigate this illconditioning, we add a regularization term to the leastsquares cost function in (27) [18, 19], i.e.,
where λ is a realvalued nonnegative regularization parameter. The optimal solution minimizing (28) is equal to
where I is the identity matrix and λ is chosen to guarantee a numerically stable inversion of D^{T}D.
Group delay compensation using modeling delay
While computing the equalization filter using (29) may yield a reasonable performance, it has been shown in, e.g., [13, 14], for singleloudspeaker equalization and in, e.g., [25], for multiloudspeaker equalization that allowing the filter design to account for acausalities can improve the equalization performance. This can be explained by the fact that accounting for acausalities allows for (partial) equalization of nonminimum phase components of the RTFs, and the inverse forward path gain \(\frac {1}{G(q)}\) in (14). In the proposed single and multiloudspeaker equalization approach we, therefore, account for such potential nonminimum phase components by delaying the transfer functions H_{open}(q) and H_{occ}(q) by d_{H} samples, such that (14) can be rewritten as
This corresponds to reformulating the cost function in (28) as
where \(\tilde {\mathbf {v}}_{\Delta }\) is defined similarly as \(\tilde {\mathbf {v}}\) in (26) but using the delayed open ear IR \(\tilde {\mathbf {h}}_{open,\Delta }\) and the delayed occluded ear IR \(\tilde {\mathbf {h}}_{occ,\Delta }\), i.e.
with
and defining the convolution matrices using zeropadded IRs, i.e.,
The optimal solution minimizing (31) is equal to
Frequencydependent regularization
Combfiltering effects may occur due to constructive and destructive interference of the leakage component and the processed signal, which is delayed due to the processing delay of the hearing device. These effects are usually most pronounced in frequency regions where the leakage component H_{occ}(q)s[k] is of similar level compared to the desired signal at the eardrum t_{des}[k]. Based on this observation, we propose to use a frequencydependent regularization that aims at reducing combfiltering effects by penalizing frequency regions where the magnitude of the leakage component is similar to the magnitude of desired signal, i.e., where
where ω_{l} denotes the lth angular frequency.
A frequencydependent weighting factor is then computed using a zero mean logarithmic normal distribution with variance \(\sigma ^{2} = \frac {\log 10}{20}\beta \), i.e.,
where the parameter β enables to control the amount of regularization depending on the relative level of the leakage component and the desired signal and P(·) is a 1/6octave smoothing with a rectangular smoothing window [29]. Using this weighting, we replace the frequencyindependent regularization in (31) with a frequencydependent regularization, i.e.,
where F is a NL_{FFT}×NL_{A}dimensional blockdiagonal matrix consisting of N L_{FFT}×L_{A}dimensional DFT matrices and W is a blockdiagonal matix consisting of N L_{FFT}×L_{FFT}dimensional diagonal matrices containing the weighting factors W(ω_{l}),l=0,…,L_{FFT}−1. The optimal solution to (41) is equal to
It should be noted that a similar frequencydependent regularization was proposed in [13]. However, the regularization in [13] also limited the filter output when the desired signal at the eardrum t_{des}[k] was much smaller than the leakage component, such that it is not applicable when active suppression of the leakage component is desired. On the contrary, the proposed weighting in (40) can also be used with small forward path gains, e.g., when the leakage component should be suppressed (cf. Section 3).
Increased robustness
While the frequencydependent regularization allows to counteract combfiltering effects, the obtained equalization filter may still be sensitive to variations in the ATFs, e.g., due to different positions of the hearing device in the ear. In order to increase the robustness to such variations, we propose to consider multiple sets of measured ATFs in the optimization, similarly as for singleloudspeaker equalization in [14].
Assuming that I different sets of ATFs are available, we propose to extend the cost function in (41) as
where \(\tilde {\mathbf {v}}_{\Delta,i}\) and D_{Δ,i} are defined similarly as in (32) and (36) for the ith set of ATFs, i=1,…,I. The optimal solution minimizing (43) is equal to
with \(\bar {\mathbf {D}}_{\Delta }\) the I(L_{D}+L_{A}−1)×NL_{A}dimensional matrix of stacked matrices D_{Δ,i} and \(\bar {\mathbf {v}}_{\Delta }\) the I(L_{D}+L_{A}−1)dimensional vector of stacked vectors \(\tilde {\mathbf {v}}_{\Delta,i}\), i.e.,
The equalization filter in (44) is optimal in the mean across the ATFs considered in the optimization and thus is expected to be more robust to frequently occuring variations in the ATFs of the hearing device.
Experimental evaluation
In this section, we evaluate the performance of the proposed equalization design procedure, using a single loudspeaker (N=1) and using multiple loudspeakers (N=2). After introducing the considered setup and performance measures in Section 5.1, we perform four different experiments. In Section 5.2, we evaluate the impact of the group delay compensation. In Section 5.3, we investigate the impact of the frequencydependent regularization. In Section 5.4, we investigate the robustness against unknown ATFs due to reinsertion of the hearing device in the ear. In Section 5.5, we evaluate the influence of different forward path gains on the equalization performance.
Setup and performance measures
All required ATFs were measured for the earpiece depicted in Fig. 1 (see also [23, 30]), which was inserted into the left ear of a GRAS 45BB12 KEMAR Head & Torso with lownoise ear simulators. It should be noted here, that this earpiece consist of four microphones and two loudspeakers. For the present evaluation we only used the microphone located on the outside close to the vent. The IRs of the ATFs were sampled at f_{s}=16,000 Hz and truncated to length L_{H}=130 for the ATFs between the source and the earpiece and the eardrum and L_{D}=100 for the ATFs between the loudspeakers of the earpiece and the eardrum. Measurements were performed in an anechoic chamber with a distance of approximately 2.3 m between the frontal source and the dummy head. Each measurement was performed I=5 times after reinserting the earpiece to investigate reinsertion variability. The forward path was set to \(\phantom {\dot {i}\!}G(q) = 10^{G_{0}/20}q^{d_{G}}\) with G_{0} a broadband gain in dB and d_{G} a delay in samples. Different broadband gains and delays were considered in the experiments.
To analyze the performance of the proposed equalization design procedure, we use the magnitude response of the aided ear transfer function H_{aid}(q) in (11) and the magnitude response of the desired open ear transfer function H_{des}(q) in (12). To quantify the differences between both magnitude responses, we use a perceptually motivated auditory spectral distance, i.e.,
where f_{low}=200 Hz and f_{up}=8000 Hz, and F(f_{l}) is a frequencydependent weighting function. To counteract overrepresentation of high frequencies, we have used the normalized inverse of the frequencydependent equivalent rectangular bandwidth [31] as weighting function, i.e.,
where c is a constant to ensure that the summation of the weighting function over the considered frequency range is equal to one.
In all experiments, the equalization filter was computed using a filter length of L_{A}=99, which is the optimal filter length for N=2. Note that for N=1 the optimal filter length of L_{A}=∞ is obviously not realizable and does not guarantee perfect equalization.
Experiment 1: Group delay compensation
In the first experiment, we investigate the impact of the group delay compensation proposed in Section 4.3. For different values of the introduced acausal delay d_{H}, we computed the equalization filter using (38) for N=1 and N=2 loudspeakers, using a small regularization parameter λ=10^{−8} to avoid numerical inversion problems. We used a broadband gain of G_{0}=0 dB and a hearing device delay of either d_{G}=1 or d_{G}=96, corresponding to a delay of 0.0625 ms and 6 ms, respectively, which is well within the range of typical delays for commercial hearing devices with transparency features [32]. It should be noted that for N=1 and a low hearing device latency the resulting equalization filter is computed similarly as in [14]. The same ATFs were used for computing the equalization filter and for evaluating its performance. Note that the sensitivity to unknown ATFs will be investigated in Experiment 3 (cf. Section 5.4).
For a hearing device delay of d_{G}=1 and N=1 loudspeaker, Fig. 3a shows magnitude responses of the aided ear transfer function for different values of the acausal delay d_{H} as well as the desired open ear transfer function and the occluded ear transfer function. As can be observed, using no group delay compensation (d_{H}=0) leads to strong deviations of the aided ear transfer function from the desired open ear transfer function. By introducing an acausal delay (d_{H}>0), a better match between both transfer functions can be achieved for frequencies above approximately 2 kHz. This is in line with results observed in [13, 14]. It should be noted that using a larger acausal delay may result in combfiltering effects, in particular in frequency regions where the occluded ear transfer function H_{occ}(q) and the desired open ear transfer function H_{des}(q) are of similar magnitude (here the frequency region below approximately 500Hz). In order to investigate the impact of the acausal delay for a larger hearing device delay, Fig. 3b depicts the magnitude responses for d_{G}=96 and N=1 loudspeaker. As can be observed, combfiltering effects now occur for all aided ear transfer functions. In addition to the combfiltering effects, again strong deviations between the aided ear transfer function and the desired open ear transfer function occur for d_{H}=0, while a better match is obtained for d_{H}>0. Comparing the results for d_{G}=1 and d_{G}=96, despite the more pronounced combfiltering effects for d_{G}=96 only a small impact of the hearing device delay is observed, demonstrating that when considering singleloudspeaker equalization an acausal delay with d_{H}≥1 is crucial. In the following experiments the optimal value for d_{H} will be determined.
For N=2 loudspeakers, Fig. 4a and b show magnitude responses of the aided ear transfer function for different values of the acausal delay d_{H} as well as the desired open ear transfer function and the occluded ear transfer function. In contrast to using N=1 loudspeaker, introducing an acausal delay (d_{H}≥1) does not yield a benefit compared to using no group delay compensation (d_{H}=0), but even leads to some deviations from the desired open ear transfer function in the lower frequencies due to combfiltering effects. This can be explained by the fact that allowing for some acausality in a singleloudspeaker system makes it easier to equalize a nonminimum phase system, while for a multiloudspeaker system a nonminimum phase system can be perfectly equalized without additional delays in case the MINT conditions are satisfied [16].
In order to investigate the impact of the acausal delay for a larger hearing device delay, Fig. 4b depicts the magnitude responses for d_{G}=96 and N=2 loudspeakers. As can be observed, combfiltering effects now occur for all aided ear transfer functions. Comparing the results for d_{G}=1 and d_{G}=96, despite the more pronounced combfiltering effects for d_{G}=96, only a small impact of the hearing device delay is observed. These results demonstrate that when considering multiloudspeaker equalization, an acausal delay is generally not necessary. However, as will be shown in the next experiment a larger d_{H} may be beneficial.
Experiment 2: Influence of regularization
In the second experiment, we investigate the impact of the frequencydependent regularization proposed in Section 4.4. We will analyze the performance of the equalization filters computed for different values of the tradeoff parameter λ in (42) and the control parameter β in (40). In this experiment we used a broadband gain of G_{0}=0 dB and a hearing device delay of d_{G}=96. If not mentioned otherwise, we used an acausal delay of d_{H}=32 (this optimal value will be determed later in this section).
For N=1 loudspeaker, Fig. 5a shows magnitude responses of the aided ear transfer function for different values of λ and β=1 as well as the desired open ear transfer function and the occluded ear transfer function. As can be observed, for high frequencies no major differences can be observed for the different considered values of λ, while differences are visible in the lower frequencies especially for f≤500 Hz, which is even clearer in the zoomed in portion in Fig. 5c. This is due to the fact that regularization is mostly affecting frequency regions where the occluded ear transfer function H_{occ}(q) and the desired open ear transfer function H_{des}(q) are of similar magnitude. Therefore, in the following we will focus on the frequency region below 1 kHz to assess the impact of the regularization parameter λ and the control parameter β. As can be observed in Fig. 5c, increasing λ reduces undesirable combfiltering effects but increases the similarity between the aided ear transfer function and the occluded ear transfer function. For example, for the largest considered value of λ=10 no visible combfiltering artifacts occur, but larger deviations between the aided ear transfer function and the desired open ear transfer function occur for frequencies between 500 and 700 Hz compared to the smaller values of λ. In general, the parameter λ introduces a tradeoff between a reduction of combfiltering artifacts in the lower frequencies and a good equalization performance in frequency regions where the magnitude responses of the occluded ear transfer function and the desired open ear transfer function begin to deviate.
In order to investigate a potential interaction between the acausal delay d_{H} and the regularization parameter λ, Fig. 6a depicts the auditory spectral distance ΔH_{aud} in (47) as a function of λ for different values of d_{H} and β=1. In general, increasing the regularization parameter results in a larger auditory spectral distance. The proposed frequencydependent regularization yields the lowest auditory spectral distance for d_{H}=32 and λ=0.1. To investigate the impact of the control parameter β, Fig. 7a depicts the auditory spectral distance as a function of λ for different values of β using d_{H}=32. As can be observed, the auditory spectral distance generally increases with increasing β. The lowest auditory spectral distance is obtained for β=1 and λ=0.1. These results show that when using the proposed approach with a single loudspeaker and a delay of d_{G}=96, using λ=0.1 and β=1 are reasonable and allow to reduce combfiltering effects in the lower frequency region while maintaining accurate equalization results.
For N=2 loudspeakers, Fig. 5b shows magnitude responses of the aided ear transfer function for different values of λ and β=1, as well as the desired open ear transfer function and the occluded ear transfer function. Similarly as for N=1, for high frequencies no major differences can be observed for the different considered values of λ, while differences are visible in the lower frequencies, e.g., especially for f≤1 kHz (cf. Fig. 5d). Again, this is due to the fact that regularization is mostly affecting frequency regions where the occluded ear transfer function H_{occ}(q) and the desired open ear transfer function H_{des}(q) are of similar magnitude. Similarly as for N=1, Figs. 6b and 7b show the auditory spectral distance as a function of λ for different values of d_{H} and β when using N=2 loudspeakers. It can be observed that the lowest auditory spectral distance is obtained for the same parameters as for N=1, i.e., d_{H}=32,λ=0.1, and β=1. We will hence use these parameter values in the following two experiments.
Experiment 3: Robustness against unknown ATFs
While in the previous experiments the same acoustic ATFs were used for computing and evaluating the performance of the equalization filter, in this experiment we investigate the impact of unknown ATFs on the performance of the equalization filter. To this end, we use five different sets of measured ATFs obtained after reinserting the earpiece into the ear of the dummy head and compute the equalization filter using the cost function defined in (44) with I=4 sets of ATFs. We evaluate the performance using the fifth set of ATFs that was not used for the computation of the equalization filter. This procedure is repeated for each of the five available sets of measurements, i.e., we use a leaveoneout crossvalidation approach. In this experiment we used a broadband gain of G_{0}=0 dB and hearing device delay of d_{G}=96.
Figure 8 shows the magnitude responses of the aided ear transfer function for N=1 and N=2 loudspeakers, respectively, as well as the desired open ear transfer function and the occluded ear transfer function. For both single and multipleloudspeaker equalization it can be observed that the results obtained by using multiple sets of measurements to compute the equalization filter (gray curves) are much closer to the desired open ear transfer function than the range of results obtained using only a single set of measurements to compute the equalization filter (light grayshaded area in the background). This is particularly the case for multiloudspeaker equalization, where huge deviations occur for unknown ATFs when using only a single set of measurements to compute the equalization filter. Comparing the results for N=1 and N=2, in general a slightly better approximation of the desired open ear transfer function is achieved using N=1, especially in the frequency range from 3500 to 6000 Hz. These results demonstrate that both using a single loudspeaker as well as using multiple loudspeakers a robust equalization can be achieved when considering multiple sets of measurements in the filter optimization, where singleloudspeaker equalization is slightly more robust than multiloudspeaker equalization.
Experiment 4: Influence of forward path gain
While in the previous experiments we used a forward path gain of G_{0}=0 dB, in practice also larger gains are obviously relevant. Therefore, in this experiment we investigate the impact of the forward path gain on the performance of the equalization filter. To this end, we consider 3 different broadband gains, i.e., G_{0}=0 dB, G_{0}=10 dB and G_{0}=20 dB. Similarly as in Experiment 3, for each considered forward path gain we compute 5 different equalization filters using I=4 sets of measured ATFs and use the fifth set of ATFs for evaluation in a leaveoneout crossvalidation approach. We use the same parameter settings as in Experiment 3, i.e., d_{G}=96,d_{H}=32,λ=0.1, and β=1.
For all considered forward path gains, Fig. 9 shows the magnitude responses of the aided ear transfer function for N=1 and N=2 loudspeakers, respectively, as well as the desired open ear transfer function and the occluded ear transfer function. As can be observed, a similar equalization performance is achieved for the different forward path gains. Furthermore, as expected combfiltering effects are reduced with larger forward path gains due to the reduced impact of the leakage component on the aided ear transfer function (see Section 3). In addition, it can be observed that the effect of the forward path gain is similar for N=1 and N=2 loudspeakers. In conclusion, these results demonstrate that the proposed approach enables to achieve a good equalization performance for different forward path gains, independent of the number of loudspeakers used without changing the design parameters, i.e., the acausal delay d_{H}, the regularization constant λ and the control parameter β.
Conclusion
In this paper we considered a leastsquaresbased procedure to design single and multiloudspeaker equalization filters for hearing devices aiming at achieving acoustic transparency. We proposed a unified design procedure for both single and multiple loudspeakers to compute the equalization filter by minimizing a leastsquares cost function. We showed that for the considered scenario the multiloudspeaker system exhibits common zeros introduced by the system design and proposed to exploit the exact knowledge about these system common zeros to reformulate the optimization problem accordingly. Since with increasing delay of the hearing device processing combfiltering artifacts are one of the major limitations to achieve a high quality of the sound at the ear drum, we proposed to reduce the hearing device playback when the leakage signal and the desired signal at the eardrum are of similar magnitude by incorporating a frequencydependent regularization in the equalization filter design. In order to improve the robustness to unknown acoustic transfer functions, we proposed to consider multiple sets of measured ATFs in the design of the equalization filter. Experimental results using measured ATFs from a multiloudspeaker earpiece show that both using a single loudspeaker as well as using multiple loudspeakers a robust equalization can be achieved when considering a robust filter optimization based on multiple sets of measurements, where singleloudspeaker equalization is slightly more robust than multiloudspeaker equalization. Furthermore, the results show that the proposed frequencydependent regularization is able to reduce combfiltering artifacts mainly in the lower frequency regions. Future research could include analyzing the effect of approximation errors of all required transfer functions, including estimation of the individual transfer function D(q), e.g., similar as in [28, 33, 34], interactions with acoustic feedback and feedback cancelation algorithms, e.g., similar as in [10], as well as subjective evaluation of the different equalization filters.
Availability of data and materials
Not applicable.
Notes
Note that regularization could also be used to mitigate this rankdeficiency. However, since we have perfect knowledge (in terms of the convolution matrices) of the system common zeros, we decided to exploit this knowledge.
References
M. C. Killion, Myths that discourage improvements in hearing aid design. Hear. Rev.11(4), 32–70 (2004).
R. Sockalingam, J. Beilin, D. L. Beck, Sound quality considerations of hearing instruments. Hear. Rev.16(9), 22–28 (2009).
H. Schepker, F. Denk, B. Kollmeier, S. Doclo, Acoustic transparency in hearables – perceptual sound quality evaluations. J. Audio Eng. Soc.68(7/8), 495–507 (2020).
F. Denk, M. Hiipakka, B. Kollmeier, S. M. A. Ernst, An individualised acoustically transparent earpiece for hearing devices. Int. J. Aud.57(3), 62–70 (2018).
P. Hoffmann, F. Christensen, D. Hammershøi, in Proc. AES Conf.: Loudspeaker & Headphones. Insert earphone calibration for hearthrough options (Audio Engineering Society (AES)Aalborg, 2013), pp. 3–4. http://www.aes.org/elib/browse.cfm?elib=16875.
V. Välimäki, A. Franck, J. Rämö, H. Gamper, L. Savioja, Assisted Listening using a Headset. Signal Process. Mag.32(2), 92–99 (2015).
J. Rämö, V. Välimäki, in Proc. Europ. Signal. Process. Conf. (EUSIPCO). An allpass hearthrough headset (IEEELisbon, 2014), pp. 1123–1127. https://ieeexplore.ieee.org/document/6952384.
J. Liski, R. Väänänen, S. Vesa, V. Välimäki, in Proc. AES Int. Conf. Headphone Techn.Adaptive equalization of acoustic transparency in an augmented reality headset (Audio Engineering Society (AES)Aalborg, 2016). https://www.aes.org/elib/browse.cfm?elib=18343.
M. A. Stone, B. C. J. Moore, K. Meisenbacher, R. P. Derleth, Tolerable Hearing Aid Delays. V. Estimation of Limits for Open Canal Fittings. Ear Hear. 29(4), 601–617 (2008).
H. Schepker, F. Denk, B. Kollmeier, S. Doclo, in Proc. AES Conference on Headphone Technology. Subjective sound quality evaluation of an acoustically transparent hearing device (Audio Engineering Society (AES)San Francisco, 2019). https://www.aes.org/elib/browse.cfm?elib=20517.
R. Gupta, R. Ranjan, J. He, G. WoonSeng, in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Process. (ICASSP). Parametric hear through equalization for augmented reality audio (IEEEBrighton, 2019), pp. 1587–1591. https://doi.org/10.1109/ICASSP.2019.8683657.
H. Schepker, F. Denk, B. Kollmeier, S. Doclo, in Proc. ITG Conf. Speech Comm.Multiloudspeaker equalization for acoustic transparency in a custom hearing device (VDEOldenburg, 2018), pp. 36–40. https://ieeexplore.ieee.org/abstract/document/8577990.
F. Denk, H. Schepker, S. Doclo, B. Kollmeier, in Proc. ITG Conf. Speech Comm.Equalization filter design for achieving acoustic transparency in a semiopen fit hearing device (VDEOldenburg, 2018), pp. 226–230. https://ieeexplore.ieee.org/abstract/document/8578028.
J. Fabry, F. König, S. Liebich, P. Jax, in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Process. (ICASSP). Acoustic equalization for headphones using a fixed feedforward filter, (IEEEBrighton, 2019), pp. 980–984. https://doi.org/10.1109/ICASSP.2019.8682167.
I. Kodrasi, T. Gerkmann, S. Doclo, in Proc. IEEE Int. Conf. Acoustics, Speech Signal Process. (ICASSP). FrequencyDomain SingleChannel Inverse Filtering for Speech Dereverberation: Theory and Practice (IEEEFlorence, 2014), pp. 5214–5218. https://doi.org/10.1109/ICASSP.2014.6854590.
M. Miyoshi, Y. Kaneda, Inverse Filtering of Room Acoustics. IEEE Trans. Acoustics, Speech Signal Process.36(2), 145–152 (1988).
B. D. Radlovic, R. C. Williamson, R. A. Kennedy, Equalization in an acoustic reverberant environment: robustness results. IEEE Trans. Speech Audio Process.8(3), 311–319 (2000).
T. Hikichi, M. Delcroix, M. Miyoshi, Inverse filtering for speech dereverberation less sensitive to noise and room transfer function fluctuations. EURASIP J. Adv. Signal Process., vol. 2007 (Springer, 2007). https://doi.org/10.1155/2007/34013.
I. Kodrasi, S. Goetze, S. Doclo, Regularization for partial multichannel equalization for speech dereverberation. IEEE Trans. Audio Speech Lang. Process.21(9), 1879–1890 (2013).
F. Lim, W. Zhang, E. A. P. Habets, P. A. Naylor, Robust Multichannel dereverberation using relaxed multichannel least squares. IEEE/ACM Trans. Audio Speech Lang. Process.22(9), 1379–1390 (2014).
A. Mertins, T. Mei, M. Kallinger, Room impulse response shortening/reshaping with infinity and pnorm optimization. IEEE Trans. Audio Speech Lang. Process.18(2), 249–259 (2010).
F. Denk, S. Vogl, H. Schepker, B. Kollmeier, M. Blau, S. Doclo, in Proc. International Workshop on Challenges in Hearing Assistive Technology (CHAT). The Acoustically Transparent Hearing Device: Towards Integration of Individualized Sound Equalization, ElectroAcoustic Modeling and Feedback Cancellation (Stockholm, 2017).
F. Denk, M. Lettau, H. Schepker, S. Doclo, R. Roden, M. Blau, J. H. Bach, J. Wellmann, B. Kollmeier, in Proc. AES Conference on Headphone Technology. A onesizefitsall earpiece with multiple microphones and drivers for hearing device research (Audio Engineering Society (AES)San Francisco, 2019), pp. 1–10. https://www.aes.org/elib/online/browse.cfm?elib=20523.
L. Ljung, T. Söderström, Theory and Practice of Recursive Identification (M.I.T. Press, Cambridge, 1983).
O. Kirkeby, P. A. Nelson, H. Hamada, F. OrdanuBustamante, Fast deconvolution of multichannel system using regularization. IEEE Trans. Speech Audio Process.6(2), 189–194 (1998).
F. Denk, S. M. A. Ernst, S. D. Ewert, B. Kollmeier, Adapting hearing devices to the individual ear acoustics: Database and target response correction functions for various device styles. Trends Hear.22:, 1–9 (2018).
S. M. Kuo, D. R. Morgan, Active noise control: A tutorial review. Proc. IEEE. 87(6), 943–973 (1999).
S. Vogl, M. Blau, Individualized prediction of the sound pressure at the eardrum for an earpiece with integrated receivers and microphones. J. Acoust. Soc. Am.145(2), 917–930 (2019).
P. D. Hatizantoniou, J. N. Mourjopoulos, Generalized fractionaloctave smoothing of audio and acoustic responses. J. Audio Eng. Soc.48(4), 259–280 (2000).
F. Denk, B. Kollmeier, The Hearpiece database of individual transfer functions of an intheear earpiece for hearing device research. Acta Acustica. 5(2), 1–16 (2021).
B. R. Glasberg, B. C. Moore, Derivation of auditory filter shapes from notchednoise data. Hear. Res.47(1), 103–138 (1990).
F. Denk, H. Schepker, S. Doclo, B. Kollmeier, Acoustic transparency in hearables  Technical evaluation. J. Aud. Eng. Soc.68(7/8), 508–521 (2020).
H. Schepker, R. Rohden, F. Denk, B. Kollmeier, M. Blau, S. Doclo, Individualized sound pressure equalization in hearing devices exploiting an electroacoustic model. arXiv:2110.01422 [eess.AS] (2021). https://arxiv.org/abs/2110.01422.
W. Jin, T. Schoof, H. Schepker, Individualized Hearthrough for Acoustic Transparency using PCAbased sound pressure estimation at the eardrum. arXiv:2110.04385 [eess.AS] (2021). https://ieeexplore.ieee.org/abstract/document/9746142. https://doi.org/10.1109/ICASSP43922.2022.9746142.
Acknowledgements
Not applicable.
Funding
This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project ID 352015383 (SFB 1330 A4 and C1), and Project ID 390895286 (EXC 2177/1) under Germany’s Excellence Strategy. Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Contributions
H.S. contributed in developing the main algorithmic idea, deriving the mathematical analysis, performing simulations, analyzing the simulation results, and drafting the article. F.D. contributed in developing the main algorithmic idea, analyzing the simulation results and revising the article. B.K. and S.D. contributed in critically discussing the mathematical analysis, the simulation results and revising the article. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Schepker, H., Denk, F., Kollmeier, B. et al. Robust single and multiloudspeaker leastsquaresbased equalization for hearing devices. J AUDIO SPEECH MUSIC PROC. 2022, 15 (2022). https://doi.org/10.1186/s13636022002476
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13636022002476
Keywords
 Sound pressure equalization
 Acoustic transparency
 Hearing device
 Multiloudspeaker