Robust single- and multi-loudspeaker least-squares-based equalization for hearing devices

Abstract

To improve the sound quality of hearing devices, equalization filters can be used to achieve acoustic transparency, i.e., listening with the device in the ear is perceptually similar to the open ear. The equalization filter needs to ensure that the superposition of the equalized signal played by the device and the signal leaking through the device into the ear canal matches a processed version of the signal reaching the eardrum of the open ear. Depending on the processing delay of the hearing device, comb-filtering artifacts can occur due to this superposition, which may degrade the perceived sound quality. In this paper, we propose a unified least-squares-based procedure to design single- and multi-loudspeaker equalization filters for hearing devices aimed at achieving acoustic transparency. To account for non-minimum phase components, we utilize a so-called group delay compensation. To reduce comb-filtering artifacts, we propose to use a frequency-dependent regularization. Experimental results using measured acoustic transfer functions from a multi-loudspeaker earpiece show that the proposed equalization filter design procedure results in robust acoustic transparency and reduces the impact of comb-filtering artifacts. A comparison between single- and multi-loudspeaker equalization shows that for both cases a robust equalization performance can be achieved for different desired open ear transfer functions.

1 Introduction

Despite major improvements in hearing device technology in the past decades, the acceptance of hearing aids and assistive listening devices is still rather limited, partly due to a suboptimal sound quality [1–3]. This is most prominent in first-time users and users with normal hearing or mild-to-moderate hearing loss. While these users would benefit from advanced hearing device processing like noise reduction, dereverberation and dynamic range compression, they usually do not accept degradations of the sound quality. In order to improve the sound quality, equalization algorithms have been proposed that aim at achieving so-called acoustic transparency [4–8], i.e., listening with the device inserted in the ear achieves a similar perceptual impression as listening without the device inserted.

Generally, equalization algorithms for acoustic transparency aim at matching the sound pressure reaching the eardrum when the device is inserted in the ear (aided ear) with the sound pressure at the eardrum when the device is not inserted (open ear) [4, 6]. For the open ear, the sound pressure at the eardrum only consists of the direct sound. In contrast, for the aided ear the sound pressure at the eardrum consists of the superposition of the direct sound leaking into the (partially) occluded ear canal and the sound picked up by the microphone(s) of the device, processed and played back by the loudspeaker(s) of the device. Since the sound played back by the device is typically delayed compared to the direct sound, so-called comb-filtering effects frequently occur which may degrade the perceived sound quality [3, 9, 10]. Several equalization algorithms for hearing devices have been proposed in the literature [4, 5, 7, 11–14]. However, often either the direct sound component was neglected in the equalization filter design, e.g., [5], or electro-acoustic components were neglected, e.g., [7]. Additionally, it may be desirable to include knowledge about the advanced hearing device processing when designing the equalization filter, e.g., when a hearing loss needs to be compensated. However, including knowledge about advanced hearing device processing has often been neglected in previous research, e.g., [5, 7, 13, 14]. In this paper we propose to include information about both the direct sound component as well as the hearing device processing in the equalization filter design.

Equalization in hearing devices is commonly performed using a single loudspeaker [6], i.e., a single equalization filter is computed to match the sound pressure of the aided ear and the open ear. Computing this equalization filter usually requires the inversion of the (estimated) acoustic transfer function (ATF) between the hearing device loudspeaker and the eardrum. However, since this ATF typically has zeros inside and outside the unit circle, perfect inversion with a stable and causal filter cannot be achieved [15, 16]. Hence, approximate solutions are required to obtain a good equalization filter when using a single loudspeaker [4, 5, 7, 13, 14], e.g., equalizing only the minimum phase component [4, 7] or by including a so-called acausal delay [13, 14]. On the contrary, using multiple loudspeakers perfect equalization can be achieved when the conditions of the multiple-input/output inverse theorem (MINT) are satisfied [16]. Briefly, MINT states that perfect inversion of a multi-channel system can be achieved if all channels are co-prime, i.e., they do not share common zeros, and the equalization filters are of sufficient length. However, since multi-loudspeaker equalization using MINT is known to be very sensitive to small changes in the ATFs [17], regularization is commonly applied to increase the robustness [18, 19] or other optimization criteria are considered [20, 21]. Multi-loudspeaker equalization for acoustic transparency in hearing devices was considered in [12], where the equalization filters were shown to exhibit system common zeros introduced by the system design, rendering the application of MINT difficult.

In this paper we propose a unified procedure to design an equalization filter that can be applied when using either a single loudspeaker or multiple loudspeakers to achieve acoustic transparency. The equalization filter is computed by minimizing a least-squares cost function, where we show that for the considered scenario the multi-loudspeaker system exhibits common zeros introduced by the system design. Since these system common zeros are, however, exactly known, we propose to exploit this knowledge and reformulate the optimization problem accordingly. In order to account for potential non-minimum phase components, we incorporate an acausal delay for group delay compensation in the filter computation, similarly as proposed for single-loudspeaker equalization in [13, 14]. Furthermore, to counteract comb-filtering effects we include a frequency-dependent regularization term to reduce the hearing device playback when the leakage signal and the desired signal at the eardrum are of similar magnitude, similarly as proposed for single-loudspeaker equalization in [13]. While regularization can also be used to increase the robustness of the equalization filters to unknown acoustic transfer functions, in this paper we improve the robustness by considering multiple sets of measurements in the optimization. While the idea of combining group delay compensation and frequency-dependent regularization for single-loudspeaker equalization was already presented in [13] and a least-squares-based design procedure for multi-loudspeaker equalization was already presented in [12], the main objective of this paper is to present a unified procedure (incorporating group delay compensation, frequency-dependent regularization, multiple measurements) that can be used both for single-loudspeaker as well as multi-loudspeaker equalization.

Experimental results using measured ATFs from a multi-loudspeaker earpiece depicted in Fig. 1 show that the proposed single- and multi-loudspeaker equalization approach is able to achieve almost perfect equalization. Furthermore, we show that the equalization performance depends on the gain and the processing delay of the hearing device. By incorporating the frequency-dependent regularization, the effect of comb-filtering in the lower frequency region can be considerably reduced. Furthermore, robust equalization can be achieved by considering multiple sets of measurements when computing the equalization filter. A performance comparison between single- and multi-loudspeaker equalization shows that robust equalization can be achieved independent of the number of loudspeakers.

Fig. 1

Schematic overview of considered single-microphone multi-loudspeaker earpiece setup [23]

The remainder of this paper is organized as follows. In Section 2 we describe the considered hearing device setup. In Section 3 we analyze the single-microphone-multiple-loudspeaker scenario with respect to the processing parameters of the hearing device. In Section 4 we present the robust single- and multi-loudspeaker equalization filter design procedure using a regularized least-squares cost function. In Section 5 the proposed equalization filters are experimentally evaluated using either a single loudspeaker or using multiple loudspeakers.

2 Scenario and problem statement

Consider a single-microphone-multi-loudspeaker hearing device with N loudspeakers as depicted in Fig. 2. For simplicity we assume that all transfer functions are linear and time-invariant and that they can be modeled as polynomials in the variable q [24]. Furthermore, we assume the absence of acoustic feedback, i.e., the coupling of the loudspeaker signal (u[k]) into the microphone signal y[k]. We assume that the signal y[k] picked up by the microphone of the hearing device is the signal emitted from a single directional sound source s[k], i.e.,

$$\begin{array}{*{20}l} y[k] = H_{m}(q)s[k], \end{array} $$

(1)

Fig. 2

Generic single-microphone multi-loudspeaker hearing device setup considered in this work

where k denotes the discrete-time index and H_m(q) denotes the ATF between the source and the microphone of the hearing device, i.e.,

$$\begin{array}{*{20}l} H_{m}(q) = \sum\limits_{i = 0}^{L_{H}-1}h_{m,i}q^{-i} = \mathbf{h}_{m}^{T}\mathbf{q}, \end{array} $$

(2)

with q a vector of delay elements and the L_H-dimensional impulse response (IR) vector h_m of H_m(q) given by

$$\begin{array}{*{20}l} \mathbf{h}_{m} = \left[ h_{m,0} \quad h_{m,1} \quad \dots \quad h_{m,L_{H}-1} \right]^{T}, \end{array} $$

(3)

where (·)^T denotes the transpose operation. The microphone signal y[k] is processed by the forward path G(q) of the hearing device, which accounts for potential advanced processing and the processing delay of the device, yielding the intermediate signal $\tilde {u}[k]$, i.e.,

$$\begin{array}{*{20}l} \tilde{u}[k] = G(q)y[k], \end{array} $$

(4)

with the L_G-dimensional IR vector g of G(q) defined similary as h_m in (3). The intermediate signal $\tilde {u}[k]$ is used as the input to N equalization filters A_n(q),n=1,…,N, yielding the N-dimensional loudspeaker signal vector u[k], i.e.,

$$\begin{array}{*{20}l} \mathbf{u}[k] = \left[ A_{1}(q) \quad \dots \quad A_{N}(q) \right]^{T} \tilde{u}[k] = \mathbf{A}(q) \tilde{u}[k], \end{array} $$

(5)

with the L_A-dimensional equalization filter coefficient vector a_n of A_n(q) given by

$$\begin{array}{*{20}l} \mathbf{a}_{n} = \left[ a_{n,0} \quad a_{n,1} \quad \dots \quad a_{n,L_{A}-1} \right]^{T}. \end{array} $$

(6)

Furthermore, we define the NL_A-dimensional vector of stacked equalization filter coefficient vectors as

$$\begin{array}{*{20}l} \mathbf{a} = \left[ \mathbf{a}_{1}^{T} \quad \dots \quad \mathbf{a}_{N}^{T} \right]^{T}. \end{array} $$

(7)

For the aided ear, i.e., when the device is inserted and playing back the processed microphone signal, the signal t_aid[k] at the eardrum of the listener is the superposition of the loudspeaker signals and the signal leaking into the (partially) occluded ear canal, i.e.,

$$\begin{array}{*{20}l} t_{aid}[k] = \mathbf{D}^{T}(q)\mathbf{u}[k]+H_{occ}(q)s[k], \end{array} $$

(8)

where H_occ(q) denotes the ATF between the source and the eardrum for the occluded ear, i.e., with the hearing device inserted and processing off, with h_occ the L_H-dimensional IR vector of H_occ(q), defined similarly as h_m in (3). The N-dimensional vector D(q) contains the ATFs between the loudspeakers of the hearing device and the eardrum, i.e.,

$$\begin{array}{*{20}l} \mathbf{D}(q) = \left[ D_{1}(q) \quad \dots \quad D_{N}(q) \right]^{T}, \end{array} $$

(9)

with d_n the L_D-dimensional IR vector of D_n(q), defined similarly as h_m in (3).

The desired signal at the eardrum t_des[k] is the signal reaching the eardrum of the listener when the device is not inserted (open ear), processed with the forward path of the device, i.e.,

$$\begin{array}{*{20}l} t_{des}[k] = G(q)\underbrace{H_{open}(q)s[k]}_{t_{open}[k]}, \end{array} $$

(10)

where H_open(q) denotes the ATF between the source and the open ear, with h_open the L_H-dimensional IR vector of H_open(q). It should be noted that the forward path G(q) is included in t_des[k] as otherwise the equalization filter would (partially) compensate for any additional signal processing applied by G(q). In order to achieve acoustic transparency, the goal is to obtain the equalization filters a in (7) such that the signal t_aid[k] in (8) is perceptually not distinguishable from the signal t_des[k] in (10), accounting for small variations in the ATFs, e.g., due to different positions of the hearing device in the ear.

3 Transfer function analysis

In this section, we analyze the considered single-microphone multi-loudspeaker system in terms of its system transfer function between the source and the eardrum. For the aided ear, the system transfer function is obtained by combining (1), (4), (5), and (8), leading to

$$\begin{array}{*{20}l} H_{aid}(q) = \frac{t_{aid}[k]}{s[k]} = \mathbf{D}^{T}(q)\mathbf{A}(q)G(q)H_{m}(q) + H_{occ}(q). \end{array} $$

(11)

Similarly, for the desired open ear transfer function, the system transfer function is obtained from (10) as

$$\begin{array}{*{20}l} H_{des}(q) = \frac{t_{des}[k]}{s[k]} = G(q)H_{open}(q). \end{array} $$

(12)

By equating (11) and (12), we observe that the optimal equalization filter needs to fulfill

$$\begin{array}{*{20}l} G(q)H_{m}(q) \mathbf{D}^{T}(q)\mathbf{A}(q) = G(q)H_{open}(q) - H_{occ}(q), \end{array} $$

(13)

which corresponds to

$$\begin{array}{*{20}l} \mathbf{D}^{T}(q)\mathbf{A}(q) = \frac{H_{open}(q)}{H_{m}(q)} - \frac{H_{occ}(q)}{H_{m}(q)} \frac{1}{G(q)}. \end{array} $$

(14)

It should be noted that the optimal equalization filter in (14) depends on the relative transfer functions (RTFs) $\frac {H_{open}(q)}{H_{m}(q)}$ and $\frac {H_{occ}(q)}{H_{m}(q)}$, i.e., the RTF between the eardrum and the microphone when the device is not inserted (open ear) and the RTF between the eardrum and the microphone when the device is inserted and switched off (occluded ear). Furthermore, the optimal filter in (14) depends on the forward path G(q). In order to analyze the dependency on the forward path, in the following we consider two extreme cases:

• 1. Assuming no leakage component (H_occ(q)=0), e.g., when the ear canal entrance is blocked completely by the device, the optimal equalization filter needs to fulfill

  $$\begin{array}{*{20}l} \mathbf{D}^{T}(q)\mathbf{A}(q) = \frac{H_{open}(q)}{H_{m}(q)}, \end{array} $$

  (15)

  such that H_aid(q)=G(q)H_open(q).

• 2. Assuming that t_des[k]=0, i.e., H_open(q)=0, e.g., when sound pressure minimization at the eardrum is desired, the optimal equalization filter aims at actively suppressing the leaking component at the eardrum, i.e.,

  $$\begin{array}{*{20}l} \mathbf{D}^{T}(q)\mathbf{A}(q) = - \frac{H_{occ}(q)}{H_{m}(q)}\frac{1}{G(q)}, \end{array} $$

  (16)

  such that H_aid(q)=0.

The above analysis shows that for large forward path gains, equalization to the open ear ATF becomes more important than suppression of the leakage component, whereas for small forward path gains the leakage component dominates and needs to be actively suppressed. Furthermore, depending on the delay of G(q), the equalized transfer function in (14) may become increasingly acausal due to the term $\frac {1}{G(q)}$, which may impact the equalization performance. Additionally, the processing delay that can be allowed for these two cases is substantially different. For large forward path gains as in case 1, a large processing delay of a few milliseconds can be tolerated [9], while to achieve the desired active suppression in case 2, only very low processing delays of only a few microseconds can be tolerated [27].

4 Equalization filter design procedure

In this section, we present a regularized least-squares-based design procedure to compute the equalization filter A(q) in the time-domain. It should be noted that the same design procedure can be applied using either a single (N=1) or multiple (N>1) loudspeakers. In the following we assume knowledge of all required ATFs, e.g., by measurement. Alternatively some ATFs could also be estimated, e.g., an estimate of the open ear ATF between the source and the eardrum H_open(q) could be obtained by an appropriate correction function of the ATF between the source and the microphone H_m(q) [26] or an estimate of the ATFs between the loudspeakers and the eardrum D(q) could be obtained using an in-ear microphone and an electro-acoustic model [28]. However, the investigation of such estimation procedures is beyond the scope of this paper.

In Section 4.1, we formulate the least-squares cost function for the equalization filter according to (13) and show that the transfer functions to be equalized share common zeros introduced by the system design. Since these system common zeros are exactly known, in Section 4.2, we exploit this knowledge and reformulate the least-squares cost function for the equalization filter according to (14), i.e., based on RTFs instead of ATFs. To account for potential acausalities in the filter design, in Section 4.3, we incorporate an acausal delay in the optimization. In addition, in Section 4.4, we include a frequency-dependent regularization to reduce comb-filtering effects. Finally, in Section 4.5, we include multiple measurements to increase the robustness of the equalization filters to variations, e.g., due to different positions of the hearing device in the ear.

4.1 Optimal equalization filter using ATFs

The expression of the optimal equalization filter A(q) in (13) can be reformulated using matrix-vector notation as

$$\begin{array}{*{20}l} \mathbf{C}\mathbf{a} = \mathbf{v}, \end{array} $$

(17)

where C is an (L_C+L_A−1)×NL_A-dimensional matrix, with L_C=L_G+L_H+L_D−2, defined as

$$\begin{array}{*{20}l} \mathbf{C} = \mathbf{G} \mathbf{H}_{m} \mathbf{D}, \end{array} $$

(18)

where D is the (L_D+L_A−1)×NL_A-dimensional matrix of concatenated (L_D+L_A−1)×L_A-dimensional convolution matrices D_n of the IR vector d_n, i.e.,

$$\begin{array}{*{20}l} \mathbf{D} &= \left[ \mathbf{D}_{1} \quad \dots \quad \mathbf{D}_{N} \right], \end{array} $$

(19)

$$\begin{array}{*{20}l} \mathbf{D}_{n} &= \left[\begin{array}{llll} d_{n,0} & 0 & \dots & 0 \\ d_{n,1} & d_{n,0} & \ddots & \vdots \\ \vdots & \ddots & \ddots & 0 \\ \vdots & \ddots & \ddots & d_{n,0} \\ \vdots & \ddots & \ddots & \vdots \\ d_{n,L_{D}-1} & \ddots & \ddots & \vdots \\ 0 & \ddots & \ddots & \vdots \\ \vdots & \ddots & \ddots & \vdots \\ 0 & \dots & 0 & d_{n,L_{D}-1} \end{array}\right], \end{array} $$

(20)

H_m is the (L_H+L_D+L_A−2)×(L_D+L_A−1)-dimensional convolution matrix of the IR vector h_m, and G is the (L_C+L_A−1)×(L_H+L_D+L_A−2)-dimensional convolution matrix of the IR vector g. Furthermore, v is the (L_C+L_A−1)-dimensional vector of the desired equalization hearing device output

$$\begin{array}{*{20}l} \mathbf{v} = \mathbf{G}\tilde{\mathbf{h}}_{open} - \tilde{\mathbf{h}}_{occ}, \end{array} $$

(21)

where $\tilde {\mathbf {h}}_{open}$ is the (L_H+L_D+L_A−2)-dimensional zero-padded vector of the IR vector h_open and $\tilde {\mathbf {h}}_{occ}$ is the (L_C+L_A−1)-dimensional zero-padded coefficient vector of the IR vector h_occ.

The NL_A-dimensional equalization filter coefficient vector a is then obtained by minimizing the following least-squares cost function

$$\begin{array}{*{20}l} J_{LS}^{atf} (\mathbf{a}) = \Vert \mathbf{C} \mathbf{a} - \mathbf{v} \Vert_{2}^{2}. \end{array} $$

(22)

The optimal solution minimizing (22) is equal to

$$\begin{array}{*{20}l} \mathbf{a}_{LS}^{atf} = \mathbf{C}^{\dag} \mathbf{v}, \end{array} $$

(23)

where (·)^† denotes the pseudo-inverse of a matrix.

4.2 Optimal equalization filter using RTFs

Since the rows of the matrix C in (18) are linearly related by the matrix GH_m, the matrix C is not of full row-rank. In order to mitigate this rank-deficiency¹, we propose to left multiply both C and v by the pseudo-inverse of GH_m (assumed to be of full column-rank), i.e.,

$$\begin{array}{*{20}l} \tilde{\mathbf{C}} &= \left(\mathbf{H}_{m}^{T}\mathbf{G}^{T} \mathbf{G} \mathbf{H}_{m}\right)^{-1} \mathbf{H}_{m}^{T}\mathbf{G}^{T} \mathbf{C} = \mathbf{D}, \end{array} $$

(24)

$$\begin{array}{*{20}l} \tilde{\mathbf{v}} &= \left(\mathbf{H}_{m}^{T}\mathbf{G}^{T} \mathbf{G} \mathbf{H}_{m}\right)^{-1} \mathbf{H}_{m}^{T}\mathbf{G}^{T} \mathbf{v}, \end{array} $$

(25)

$$\begin{array}{*{20}l} &= \left(\mathbf{H}_{m}^{T}\mathbf{G}^{T} \mathbf{G} \mathbf{H}_{m}\right)^{-1} \mathbf{H}_{m}^{T}\mathbf{G}^{T}\left(\mathbf{G}\tilde{\mathbf{h}}_{open} - \tilde{\mathbf{h}}_{occ}\right), \end{array} $$

(26)

which is equivalent to writing (14) using matrix-vector notation. It should be noted that $\tilde {\mathbf {v}}$ in (24) represents an RTF, i.e., an infinite impulse response filter, which cannot be perfectly modeled using a finite impulse response filter and hence perfect equalization is not possible. Nevertheless, the least-squares cost function in (22) can now be reformulated using D and $\tilde {\mathbf {v}}$ instead of C and v, i.e.,

$$\begin{array}{*{20}l} J_{LS}^{rtf} (\mathbf{a}) = \Vert \mathbf{D} \mathbf{a} - \tilde{\mathbf{v}} \Vert_{2}^{2}. \end{array} $$

(27)

However, since the ATFs between the loudspeakers and the eardrum D(q) are likely to share near-common zeros due the close proximity of the loudspeakers in the considered hearing device (cf. Fig. 1), the matrix inversion when using D in (27) is typically ill-conditioned. In order to mitigate this ill-conditioning, we add a regularization term to the least-squares cost function in (27) [18, 19], i.e.,

$$\begin{array}{*{20}l} \boxed{J_{rLS}(\mathbf{a}) = \Vert \mathbf{D} \mathbf{a} - \tilde{\mathbf{v}} \Vert_{2}^{2} + \lambda \Vert \mathbf{a} \Vert_{2}^{2}} \end{array} $$

(28)

where λ is a real-valued non-negative regularization parameter. The optimal solution minimizing (28) is equal to

$$\begin{array}{*{20}l} \boxed{\mathbf{a}_{rLS} = \left(\mathbf{D}^{T} \mathbf{D} + \lambda\mathbf{I}\right)^{-1} \mathbf{D}^{T} \tilde{\mathbf{v}}} \end{array} $$

(29)

where I is the identity matrix and λ is chosen to guarantee a numerically stable inversion of D^TD.

4.3 Group delay compensation using modeling delay

While computing the equalization filter using (29) may yield a reasonable performance, it has been shown in, e.g., [13, 14], for single-loudspeaker equalization and in, e.g., [25], for multi-loudspeaker equalization that allowing the filter design to account for acausalities can improve the equalization performance. This can be explained by the fact that accounting for acausalities allows for (partial) equalization of non-minimum phase components of the RTFs, and the inverse forward path gain $\frac {1}{G(q)}$ in (14). In the proposed single- and multi-loudspeaker equalization approach we, therefore, account for such potential non-minimum phase components by delaying the transfer functions H_open(q) and H_occ(q) by d_H samples, such that (14) can be rewritten as

$$\begin{array}{*{20}l} \mathbf{D}^{T}(q)\mathbf{A}(q) = \frac{H_{open}\left(q^{-d_{H}}\right)}{H_{m}(q)} - \frac{H_{occ}\left(q^{-d_{H}}\right)}{H_{m}(q)} \frac{1}{G(q)}. \end{array} $$

(30)

This corresponds to reformulating the cost function in (28) as

$$\begin{array}{*{20}l} \boxed{J_{r\Delta LS}(\mathbf{a}) = \Vert \mathbf{D}_{\Delta} \mathbf{a} - \tilde{\mathbf{v}}_{\Delta} \Vert_{2}^{2} + \lambda \Vert \mathbf{a} \Vert_{2}^{2}} \end{array} $$

(31)

where $\tilde {\mathbf {v}}_{\Delta }$ is defined similarly as $\tilde {\mathbf {v}}$ in (26) but using the delayed open ear IR $\tilde {\mathbf {h}}_{open,\Delta }$ and the delayed occluded ear IR $\tilde {\mathbf {h}}_{occ,\Delta }$, i.e.

$$ \begin{aligned} \tilde{\mathbf{v}}_{\Delta} &= \left(\mathbf{H}_{m,\Delta}^{T}\mathbf{G}_{\Delta}^{T} \mathbf{G}_{\Delta} \mathbf{H}_{m,\Delta}\right)^{-1} \mathbf{H}_{m,\Delta}^{T}\mathbf{G}_{\Delta}^{T} \left(\mathbf{G}_{\Delta}\tilde{\mathbf{h}}_{open,\Delta} - \tilde{\mathbf{h}}_{occ,\Delta}\right), \end{aligned} $$

(32)

with

$$\begin{array}{*{20}l} \tilde{\mathbf{h}}_{open,\Delta} &= [ \underbrace{\begin{array}{ccc} 0 & \dots & 0 \end{array}}_{d_{H}} \,\, \tilde{\mathbf{h}}_{open}^{T} \,\, ]^{T}, \end{array} $$

(33)

$$\begin{array}{*{20}l} \tilde{\mathbf{h}}_{occ,\Delta} &= [ \underbrace{\begin{array}{ccc} 0 & \dots & 0 \end{array}}_{d_{H}} \,\, \tilde{\mathbf{h}}_{occ}^{T} \,\, ]^{T}, \end{array} $$

(34)

and defining the convolution matrices using zero-padded IRs, i.e.,

$$\begin{array}{*{20}l} \mathbf{G}_{\Delta} &= \left[\begin{array}{cc} \mathbf{G} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0} \end{array}\right], \end{array} $$

(35)

$$\begin{array}{*{20}l} \mathbf{D}_{\Delta} &= \left[\begin{array}{cc} \mathbf{D} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0} \end{array}\right], \end{array} $$

(36)

$$\begin{array}{*{20}l} \mathbf{H}_{m,\Delta} &= \left[\begin{array}{cc} \mathbf{H}_{m} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0} \end{array}\right]. \end{array} $$

(37)

The optimal solution minimizing (31) is equal to

$$\begin{array}{*{20}l} \boxed{\mathbf{a}_{r\Delta LS} = \left(\mathbf{D}_{\Delta}^{T} \mathbf{D}_{\Delta} + \lambda\mathbf{I}\right)^{-1} \mathbf{D}_{\Delta}^{T} \tilde{\mathbf{v}}_{\Delta}} \end{array} $$

(38)

4.4 Frequency-dependent regularization

Comb-filtering effects may occur due to constructive and destructive interference of the leakage component and the processed signal, which is delayed due to the processing delay of the hearing device. These effects are usually most pronounced in frequency regions where the leakage component H_occ(q)s[k] is of similar level compared to the desired signal at the eardrum t_des[k]. Based on this observation, we propose to use a frequency-dependent regularization that aims at reducing comb-filtering effects by penalizing frequency regions where the magnitude of the leakage component is similar to the magnitude of desired signal, i.e., where

$$\begin{array}{*{20}l} V(\omega_{l}) &= \frac{\vert H_{occ}(\omega_{l})\vert }{\vert H_{open}(\omega_{l})G(\omega_{l})\vert} \approx 1, \end{array} $$

(39)

where ω_l denotes the lth angular frequency.

A frequency-dependent weighting factor is then computed using a zero mean logarithmic normal distribution with variance $\sigma ^{2} = \frac {\log 10}{20}\beta$, i.e.,

$$\begin{array}{*{20}l} W(\omega_{l}) = \frac{1}{\sqrt{2\pi}P(V(\omega_{l}))\sigma}e^{-\frac{1}{2} \left(\frac{\log P(V(\omega_{l}))}{\sigma}\right)^{2} }, \end{array} $$

(40)

where the parameter β enables to control the amount of regularization depending on the relative level of the leakage component and the desired signal and P(·) is a 1/6-octave smoothing with a rectangular smoothing window [29]. Using this weighting, we replace the frequency-independent regularization in (31) with a frequency-dependent regularization, i.e.,

$$\begin{array}{*{20}l} \boxed{J_{fr\Delta LS}(\mathbf{a})=\Vert \mathbf{D}_{\Delta} \mathbf{a} - \tilde{\mathbf{v}}_{\Delta} \Vert_{2}^{2} + \lambda \Vert \mathbf{W} \mathbf{F} \mathbf{a} \Vert_{2}^{2}} \end{array} $$

(41)

where F is a NL_FFT×NL_A-dimensional block-diagonal matrix consisting of N L_FFT×L_A-dimensional DFT matrices and W is a block-diagonal matix consisting of N L_FFT×L_FFT-dimensional diagonal matrices containing the weighting factors W(ω_l),l=0,…,L_FFT−1. The optimal solution to (41) is equal to

$$\begin{array}{*{20}l} \boxed{\mathbf{a}_{fr\Delta LS} = \left(\mathbf{D}_{\Delta}^{T} \mathbf{D}_{\Delta} + \lambda\mathbf{F}^{H}\mathbf{W}^{H}\mathbf{W} \mathbf{F}\right)^{-1} \mathbf{D}_{\Delta}^{T} \tilde{\mathbf{v}}_{\Delta}} \end{array} $$

(42)

It should be noted that a similar frequency-dependent regularization was proposed in [13]. However, the regularization in [13] also limited the filter output when the desired signal at the eardrum t_des[k] was much smaller than the leakage component, such that it is not applicable when active suppression of the leakage component is desired. On the contrary, the proposed weighting in (40) can also be used with small forward path gains, e.g., when the leakage component should be suppressed (cf. Section 3).

4.5 Increased robustness

While the frequency-dependent regularization allows to counteract comb-filtering effects, the obtained equalization filter may still be sensitive to variations in the ATFs, e.g., due to different positions of the hearing device in the ear. In order to increase the robustness to such variations, we propose to consider multiple sets of measured ATFs in the optimization, similarly as for single-loudspeaker equalization in [14].

Assuming that I different sets of ATFs are available, we propose to extend the cost function in (41) as

$$\begin{array}{*{20}l} \boxed{J_{mfr\Delta LS}(\mathbf{a}) = \sum\limits_{i=1}^{I}\Vert \mathbf{D}_{\Delta,i} \mathbf{a} - \tilde{\mathbf{v}}_{\Delta,i} \Vert_{2}^{2} + \lambda \Vert \mathbf{W} \mathbf{F} \mathbf{a} \Vert_{2}^{2}} \end{array} $$

(43)

where $\tilde {\mathbf {v}}_{\Delta,i}$ and D_Δ,i are defined similarly as in (32) and (36) for the ith set of ATFs, i=1,…,I. The optimal solution minimizing (43) is equal to

$$\begin{array}{*{20}l} \boxed{\mathbf{a}_{mfr\Delta LS} = \left(\bar{\mathbf{D}}_{\Delta}^{T} \bar{\mathbf{D}}_{\Delta} + \lambda\mathbf{F}^{H} \mathbf{W}^{H} \mathbf{W} \mathbf{F}\right)^{-1} \bar{\mathbf{D}}_{\Delta}^{T} \bar{\mathbf{v}}_{\Delta}} \end{array} $$

(44)

with $\bar {\mathbf {D}}_{\Delta }$ the I(L_D+L_A−1)×NL_A-dimensional matrix of stacked matrices D_Δ,i and $\bar {\mathbf {v}}_{\Delta }$ the I(L_D+L_A−1)-dimensional vector of stacked vectors $\tilde {\mathbf {v}}_{\Delta,i}$, i.e.,

$$\begin{array}{*{20}l} \bar{\mathbf{D}}_{\Delta} &= \left[ \mathbf{D}_{\Delta,1}^{T} \quad \dots \quad \mathbf{D}_{\Delta,I}^{T} \right]^{T}, \end{array} $$

(45)

$$\begin{array}{*{20}l} \bar{\mathbf{v}}_{\Delta} &= \left[ \tilde{\mathbf{v}}_{\Delta,1}^{T} \quad \dots \quad \tilde{\mathbf{v}}_{\Delta,I}^{T} \right]^{T}. \end{array} $$

(46)

The equalization filter in (44) is optimal in the mean across the ATFs considered in the optimization and thus is expected to be more robust to frequently occuring variations in the ATFs of the hearing device.

5 Experimental evaluation

In this section, we evaluate the performance of the proposed equalization design procedure, using a single loudspeaker (N=1) and using multiple loudspeakers (N=2). After introducing the considered setup and performance measures in Section 5.1, we perform four different experiments. In Section 5.2, we evaluate the impact of the group delay compensation. In Section 5.3, we investigate the impact of the frequency-dependent regularization. In Section 5.4, we investigate the robustness against unknown ATFs due to reinsertion of the hearing device in the ear. In Section 5.5, we evaluate the influence of different forward path gains on the equalization performance.

5.1 Setup and performance measures

All required ATFs were measured for the earpiece depicted in Fig. 1 (see also [23, 30]), which was inserted into the left ear of a GRAS 45BB-12 KEMAR Head & Torso with low-noise ear simulators. It should be noted here, that this earpiece consist of four microphones and two loudspeakers. For the present evaluation we only used the microphone located on the outside close to the vent. The IRs of the ATFs were sampled at f_s=16,000 Hz and truncated to length L_H=130 for the ATFs between the source and the earpiece and the eardrum and L_D=100 for the ATFs between the loudspeakers of the earpiece and the eardrum. Measurements were performed in an anechoic chamber with a distance of approximately 2.3 m between the frontal source and the dummy head. Each measurement was performed I=5 times after reinserting the earpiece to investigate reinsertion variability. The forward path was set to $\phantom {\dot {i}\!}G(q) = 10^{G_{0}/20}q^{-d_{G}}$ with G₀ a broadband gain in dB and d_G a delay in samples. Different broadband gains and delays were considered in the experiments.

To analyze the performance of the proposed equalization design procedure, we use the magnitude response of the aided ear transfer function H_aid(q) in (11) and the magnitude response of the desired open ear transfer function H_des(q) in (12). To quantify the differences between both magnitude responses, we use a perceptually motivated auditory spectral distance, i.e.,

$$\begin{array}{*{20}l} \Delta H_{aud} = \sum\limits_{f_{l} = f_{low}}^{f_{up}} F(f_{l}) \Big\vert 10\log_{10}\frac{\vert H_{aid}(f_{l})\vert^{2}}{\vert H_{des}(f_{l})\vert^{2}} \Big\vert, \end{array} $$

(47)

where f_low=200 Hz and f_up=8000 Hz, and F(f_l) is a frequency-dependent weighting function. To counteract over-representation of high frequencies, we have used the normalized inverse of the frequency-dependent equivalent rectangular bandwidth [31] as weighting function, i.e.,

$$\begin{array}{*{20}l} F(f_{l}) &= \frac{c}{24.7(4.37 f_{l} + 1)}, \end{array} $$

(48)

where c is a constant to ensure that the summation of the weighting function over the considered frequency range is equal to one.

In all experiments, the equalization filter was computed using a filter length of L_A=99, which is the optimal filter length for N=2. Note that for N=1 the optimal filter length of L_A=∞ is obviously not realizable and does not guarantee perfect equalization.

5.2 Experiment 1: Group delay compensation

In the first experiment, we investigate the impact of the group delay compensation proposed in Section 4.3. For different values of the introduced acausal delay d_H, we computed the equalization filter using (38) for N=1 and N=2 loudspeakers, using a small regularization parameter λ=10⁻⁸ to avoid numerical inversion problems. We used a broadband gain of G₀=0 dB and a hearing device delay of either d_G=1 or d_G=96, corresponding to a delay of 0.0625 ms and 6 ms, respectively, which is well within the range of typical delays for commercial hearing devices with transparency features [32]. It should be noted that for N=1 and a low hearing device latency the resulting equalization filter is computed similarly as in [14]. The same ATFs were used for computing the equalization filter and for evaluating its performance. Note that the sensitivity to unknown ATFs will be investigated in Experiment 3 (cf. Section 5.4).

For a hearing device delay of d_G=1 and N=1 loudspeaker, Fig. 3a shows magnitude responses of the aided ear transfer function for different values of the acausal delay d_H as well as the desired open ear transfer function and the occluded ear transfer function. As can be observed, using no group delay compensation (d_H=0) leads to strong deviations of the aided ear transfer function from the desired open ear transfer function. By introducing an acausal delay (d_H>0), a better match between both transfer functions can be achieved for frequencies above approximately 2 kHz. This is in line with results observed in [13, 14]. It should be noted that using a larger acausal delay may result in comb-filtering effects, in particular in frequency regions where the occluded ear transfer function H_occ(q) and the desired open ear transfer function H_des(q) are of similar magnitude (here the frequency region below approximately 500Hz). In order to investigate the impact of the acausal delay for a larger hearing device delay, Fig. 3b depicts the magnitude responses for d_G=96 and N=1 loudspeaker. As can be observed, comb-filtering effects now occur for all aided ear transfer functions. In addition to the comb-filtering effects, again strong deviations between the aided ear transfer function and the desired open ear transfer function occur for d_H=0, while a better match is obtained for d_H>0. Comparing the results for d_G=1 and d_G=96, despite the more pronounced comb-filtering effects for d_G=96 only a small impact of the hearing device delay is observed, demonstrating that when considering single-loudspeaker equalization an acausal delay with d_H≥1 is crucial. In the following experiments the optimal value for d_H will be determined.

Fig. 3

Magnitude responses of the occluded ear transfer function H_occ(q), the aided ear transfer function H_aid(q) and the desired open ear transfer function H_des(q) for different values of d_H using N=1 loudspeaker, and G₀=0 dB for a d_G=1 and b d_G=96

For N=2 loudspeakers, Fig. 4a and b show magnitude responses of the aided ear transfer function for different values of the acausal delay d_H as well as the desired open ear transfer function and the occluded ear transfer function. In contrast to using N=1 loudspeaker, introducing an acausal delay (d_H≥1) does not yield a benefit compared to using no group delay compensation (d_H=0), but even leads to some deviations from the desired open ear transfer function in the lower frequencies due to comb-filtering effects. This can be explained by the fact that allowing for some acausality in a single-loudspeaker system makes it easier to equalize a non-minimum phase system, while for a multi-loudspeaker system a non-minimum phase system can be perfectly equalized without additional delays in case the MINT conditions are satisfied [16].

Fig. 4

Magnitude responses of the occluded ear transfer function H_occ(q), the aided ear transfer function H_aid(q) and the desired open ear transfer function H_des(q) for different values of d_H using N=2 loudspeakers, and G₀=0 dB for a d_G=1 and b d_G=96

In order to investigate the impact of the acausal delay for a larger hearing device delay, Fig. 4b depicts the magnitude responses for d_G=96 and N=2 loudspeakers. As can be observed, comb-filtering effects now occur for all aided ear transfer functions. Comparing the results for d_G=1 and d_G=96, despite the more pronounced comb-filtering effects for d_G=96, only a small impact of the hearing device delay is observed. These results demonstrate that when considering multi-loudspeaker equalization, an acausal delay is generally not necessary. However, as will be shown in the next experiment a larger d_H may be beneficial.

5.3 Experiment 2: Influence of regularization

In the second experiment, we investigate the impact of the frequency-dependent regularization proposed in Section 4.4. We will analyze the performance of the equalization filters computed for different values of the trade-off parameter λ in (42) and the control parameter β in (40). In this experiment we used a broadband gain of G₀=0 dB and a hearing device delay of d_G=96. If not mentioned otherwise, we used an acausal delay of d_H=32 (this optimal value will be determed later in this section).

For N=1 loudspeaker, Fig. 5a shows magnitude responses of the aided ear transfer function for different values of λ and β=1 as well as the desired open ear transfer function and the occluded ear transfer function. As can be observed, for high frequencies no major differences can be observed for the different considered values of λ, while differences are visible in the lower frequencies especially for f≤500 Hz, which is even clearer in the zoomed in portion in Fig. 5c. This is due to the fact that regularization is mostly affecting frequency regions where the occluded ear transfer function H_occ(q) and the desired open ear transfer function H_des(q) are of similar magnitude. Therefore, in the following we will focus on the frequency region below 1 kHz to assess the impact of the regularization parameter λ and the control parameter β. As can be observed in Fig. 5c, increasing λ reduces undesirable comb-filtering effects but increases the similarity between the aided ear transfer function and the occluded ear transfer function. For example, for the largest considered value of λ=10 no visible comb-filtering artifacts occur, but larger deviations between the aided ear transfer function and the desired open ear transfer function occur for frequencies between 500 and 700 Hz compared to the smaller values of λ. In general, the parameter λ introduces a trade-off between a reduction of comb-filtering artifacts in the lower frequencies and a good equalization performance in frequency regions where the magnitude responses of the occluded ear transfer function and the desired open ear transfer function begin to deviate.

Fig. 5

Magnitude responses of the occluded ear transfer function H_occ(q), the desired open ear transfer function H_des(q) and the aided ear transfer function H_aid(q) for different values of λ for N=1 and N=2 loudspeakers. (β=1,d_G=96 and G₀=0 dB)

In order to investigate a potential interaction between the acausal delay d_H and the regularization parameter λ, Fig. 6a depicts the auditory spectral distance ΔH_aud in (47) as a function of λ for different values of d_H and β=1. In general, increasing the regularization parameter results in a larger auditory spectral distance. The proposed frequency-dependent regularization yields the lowest auditory spectral distance for d_H=32 and λ=0.1. To investigate the impact of the control parameter β, Fig. 7a depicts the auditory spectral distance as a function of λ for different values of β using d_H=32. As can be observed, the auditory spectral distance generally increases with increasing β. The lowest auditory spectral distance is obtained for β=1 and λ=0.1. These results show that when using the proposed approach with a single loudspeaker and a delay of d_G=96, using λ=0.1 and β=1 are reasonable and allow to reduce comb-filtering effects in the lower frequency region while maintaining accurate equalization results.

Fig. 6

Auditory spectral distance as a function of the regularization parameter λ for different acausal delays d_H for a N=1 loudspeaker and b N=2 loudspeakers (β=1,G₀=0 dB, d_G=96)

Fig. 7

Auditory spectral distance as a function of the regularization parameter λ for different values of control parameter β for a N=1 loudspeaker and b N=2 loudspeakers (d_H=32,G₀=0 dB, d_G=96)

For N=2 loudspeakers, Fig. 5b shows magnitude responses of the aided ear transfer function for different values of λ and β=1, as well as the desired open ear transfer function and the occluded ear transfer function. Similarly as for N=1, for high frequencies no major differences can be observed for the different considered values of λ, while differences are visible in the lower frequencies, e.g., especially for f≤1 kHz (cf. Fig. 5d). Again, this is due to the fact that regularization is mostly affecting frequency regions where the occluded ear transfer function H_occ(q) and the desired open ear transfer function H_des(q) are of similar magnitude. Similarly as for N=1, Figs. 6b and 7b show the auditory spectral distance as a function of λ for different values of d_H and β when using N=2 loudspeakers. It can be observed that the lowest auditory spectral distance is obtained for the same parameters as for N=1, i.e., d_H=32,λ=0.1, and β=1. We will hence use these parameter values in the following two experiments.

5.4 Experiment 3: Robustness against unknown ATFs

While in the previous experiments the same acoustic ATFs were used for computing and evaluating the performance of the equalization filter, in this experiment we investigate the impact of unknown ATFs on the performance of the equalization filter. To this end, we use five different sets of measured ATFs obtained after reinserting the earpiece into the ear of the dummy head and compute the equalization filter using the cost function defined in (44) with I=4 sets of ATFs. We evaluate the performance using the fifth set of ATFs that was not used for the computation of the equalization filter. This procedure is repeated for each of the five available sets of measurements, i.e., we use a leave-one-out cross-validation approach. In this experiment we used a broadband gain of G₀=0 dB and hearing device delay of d_G=96.

Figure 8 shows the magnitude responses of the aided ear transfer function for N=1 and N=2 loudspeakers, respectively, as well as the desired open ear transfer function and the occluded ear transfer function. For both single- and multiple-loudspeaker equalization it can be observed that the results obtained by using multiple sets of measurements to compute the equalization filter (gray curves) are much closer to the desired open ear transfer function than the range of results obtained using only a single set of measurements to compute the equalization filter (light gray-shaded area in the background). This is particularly the case for multi-loudspeaker equalization, where huge deviations occur for unknown ATFs when using only a single set of measurements to compute the equalization filter. Comparing the results for N=1 and N=2, in general a slightly better approximation of the desired open ear transfer function is achieved using N=1, especially in the frequency range from 3500 to 6000 Hz. These results demonstrate that both using a single loudspeaker as well as using multiple loudspeakers a robust equalization can be achieved when considering multiple sets of measurements in the filter optimization, where single-loudspeaker equalization is slightly more robust than multi-loudspeaker equalization.

Fig. 8

Magnitude responses of the occluded ear transfer function H_occ(q), the desired open ear transfer function H_des(q) and the aided ear transfer function H_aid(q) for the robust optimization for a N=1 and b N=2 (λ=0.1,d_G=96,d_H=32,β=1). The light gray-shaded area shows the range of the results based on optimization using a single set of measurements

5.5 Experiment 4: Influence of forward path gain

While in the previous experiments we used a forward path gain of G₀=0 dB, in practice also larger gains are obviously relevant. Therefore, in this experiment we investigate the impact of the forward path gain on the performance of the equalization filter. To this end, we consider 3 different broadband gains, i.e., G₀=0 dB, G₀=10 dB and G₀=20 dB. Similarly as in Experiment 3, for each considered forward path gain we compute 5 different equalization filters using I=4 sets of measured ATFs and use the fifth set of ATFs for evaluation in a leave-one-out cross-validation approach. We use the same parameter settings as in Experiment 3, i.e., d_G=96,d_H=32,λ=0.1, and β=1.

For all considered forward path gains, Fig. 9 shows the magnitude responses of the aided ear transfer function for N=1 and N=2 loudspeakers, respectively, as well as the desired open ear transfer function and the occluded ear transfer function. As can be observed, a similar equalization performance is achieved for the different forward path gains. Furthermore, as expected comb-filtering effects are reduced with larger forward path gains due to the reduced impact of the leakage component on the aided ear transfer function (see Section 3). In addition, it can be observed that the effect of the forward path gain is similar for N=1 and N=2 loudspeakers. In conclusion, these results demonstrate that the proposed approach enables to achieve a good equalization performance for different forward path gains, independent of the number of loudspeakers used without changing the design parameters, i.e., the acausal delay d_H, the regularization constant λ and the control parameter β.

Fig. 9

Magnitude responses of the occluded ear transfer function H_occ(q), the desired open ear transfer function H_des(q) and the aided ear transfer function H_aid(q) for the robust optimization for different forward path gains G₀ and a N=1 and b N=2 (λ=0.1,d_G=96,d_H=32,β=1). The forward path gains are from bottom to top G₀=0 dB, G₀=10 dB, and G₀=20 dB

6 Conclusion

In this paper we considered a least-squares-based procedure to design single- and multi-loudspeaker equalization filters for hearing devices aiming at achieving acoustic transparency. We proposed a unified design procedure for both single and multiple loudspeakers to compute the equalization filter by minimizing a least-squares cost function. We showed that for the considered scenario the multi-loudspeaker system exhibits common zeros introduced by the system design and proposed to exploit the exact knowledge about these system common zeros to reformulate the optimization problem accordingly. Since with increasing delay of the hearing device processing comb-filtering artifacts are one of the major limitations to achieve a high quality of the sound at the ear drum, we proposed to reduce the hearing device playback when the leakage signal and the desired signal at the eardrum are of similar magnitude by incorporating a frequency-dependent regularization in the equalization filter design. In order to improve the robustness to unknown acoustic transfer functions, we proposed to consider multiple sets of measured ATFs in the design of the equalization filter. Experimental results using measured ATFs from a multi-loudspeaker earpiece show that both using a single loudspeaker as well as using multiple loudspeakers a robust equalization can be achieved when considering a robust filter optimization based on multiple sets of measurements, where single-loudspeaker equalization is slightly more robust than multi-loudspeaker equalization. Furthermore, the results show that the proposed frequency-dependent regularization is able to reduce comb-filtering artifacts mainly in the lower frequency regions. Future research could include analyzing the effect of approximation errors of all required transfer functions, including estimation of the individual transfer function D(q), e.g., similar as in [28, 33, 34], interactions with acoustic feedback and feedback cancelation algorithms, e.g., similar as in [10], as well as subjective evaluation of the different equalization filters.