A robust polynomial regression-based voice activity detector for speaker verification

Abstract

Robustness against background noise is a major research area for speech-related applications such as speech recognition and speaker recognition. One of the many solutions for this problem is to detect speech-dominant regions by using a voice activity detector (VAD). In this paper, a second-order polynomial regression-based algorithm is proposed with a similar function as a VAD for text-independent speaker verification systems. The proposed method aims to separate steady noise/silence regions, steady speech regions, and speech onset/offset regions. The regression is applied independently to each filter band of a mel spectrum, which makes the algorithm fit seamlessly to the conventional extraction process of the mel-frequency cepstral coefficients (MFCCs). The k-means algorithm is also applied to estimate average noise energy in each band for spectral subtraction. A pseudo SNR-dependent linear thresholding for the final VAD output decision is introduced based on the k-means energy centers. This thresholding considers the speech presence in each band. Conventional VADs usually neglect the deteriorative effects of the additive noise in the speech regions. Contrary to this, the proposed method decides not only for the speech presence, but also if the frame is dominated by the speech, or the noise. Performance of the proposed algorithm is compared with a continuous noise tracking method, and another VAD method in speaker verification experiments, where five different noise types at five different SNR levels were considered. The proposed algorithm showed superior verification performance both with the conventional GMM-UBM method, and the state-of-the-art i-vector method.

1 Introduction

Automatic speaker recognition systems’ performances are greatly improved in the last two decades, especially with the introduction of the modeling methods such as universal background model (UBM) [1] and i-vectors [2]. For the front end, mel-frequency cepstral coefficients (MFCCs) [3] are extensively preferred by the researchers in speaker or speech recognition systems. Despite the high performance of the MFCCs in controlled environments, their performance degrades in adverse conditions, such as convolutive channel noise, additive background noise, and reverberation. Many different techniques have been developed to overcome this issue such as using a voice activity detector (VAD), extracting robust features, and speech enhancement. Some of them are mentioned in the following paragraphs.

Instead of using the MFCCs, other types of features are proposed by the researchers to increase the robustness of the recognizers [4,5,6,7,8]. Also, since the MFCCs are widely adopted, many researchers have made effort to improve its robustness under noise by modifying, or changing, some processes in the conventional scheme [9,10,11,12,13]. Interested readers may refer to [14] for the recent progress in the feature extraction techniques for robust speaker recognition.

Another way to improve the performance of the recognizers is to enhance the speech. These methods usually require an estimation of the level and frequency response of the noise. When the noise has been estimated, the enhancement can be achieved via methods such as spectral subtraction [15] or Wiener filtering as in [16]. Once the signal is enhanced, it can be treated as a clean signal and can be used in further operations (i.e., feature extraction). A popular method for the noise estimation is based on the minimum statistics as proposed in [17] or based on its improvement [18]. Statistical differences are taken into account in [19] for non-stationary noise power spectral density estimation. Relying on the speech/non-speech decisions of the frames (by considering the harmonics for the voiced speech and durations for the unvoiced speech), a noise tracking algorithm is proposed [20]. Speech presence probability is considered in [21], which depends on the relation of the adjacent spectral components.

The methods referred in the previous paragraph are aiming to track the noise spectrum even under the speech activity. They are fast adapting to the changes in the noise level but may degrade the information during the speech regions. Since both the noise and the speech are present at the same time, exactly separating them is not possible. On the other hand, separating the speech-dominant and the noise-dominant frames can also be beneficial. It is also possible to estimate an average of the noise from the noise-dominant frames, or the noise parameters can be updated between the speech regions. Various voice activity detection algorithms have been developed for this purpose. In [22], long-term signal probabilities are used as the discrimination criteria and the decision is assigned to the frame in the middle of a long window. Similarly, in [23], long-term signal variability is used but the decision is assigned to the long window. Energy-based detection and Gaussian mixture model (GMM)-based statistical model are combined for the VAD in [24]. Modeling the feature distribution with a bi-Gaussian model can be also used as a VAD, where the Gaussian with the lower mean corresponds to noisy frames [25]. Four different voicing measures and a spectral feature are concatenated and mapped to a one-dimensional space via principal component analysis in [26], and the resulting feature distribution is bi-Gaussian. In addition to the four voicing measures used in [26], MFCCs and two pitch trackers are used as features for the VAD in [27]. A comparison of some standard VAD methods for speaker recognition can be found in [28], where the bi-Gaussian modeling-based VAD is reported to be the best performing one. Similar to the VAD, vowel-like regions are used in [29] and improved in [30] by including the non-vowel-like regions. Missing data approach is also investigated in several studies [31,32,33,34], where a binary time-frequency mask is constructed for the noisy spectrum to indicate reliable and unreliable features. The unreliable features are then reconstructed, or marginalized (ignored in score computation).

Usually, VADs look for clues of speech presence (i.e., harmonicity, periodicity, energy, long-term variability) but do not give any idea about if the found speech region is useful for recognition, or how many bands include speech information. As mentioned before, exactly separating the noise and the speech is not possible. Therefore, an output speech of a robust VAD may not cover the same speech/speaker information as its clean equivalent does. Especially for the low SNR signals, directly using the VAD output is not suitable, because of the nonlinear transformation effect of the additive noise on the low energy frames [35]. Missing data approach tries to compensate this issue; however, its performance highly depends on the estimation of the spectral mask, where a poor estimation reduces the recognition performance [36].

In this paper, a polynomial regression-based algorithm is proposed as a solution to the aforementioned problem, that is, separating the useful speaker/speech information from the additive noise. A polynomial is fitted on the energy output over a range of several frames, and the output is not strictly frame-dependent. Also, suppression of sudden energy ripples is possible with this method, which is achieved by smoothing in some studies [27]. Besides the polynomial regression, another important novelty of the proposed algorithm is the pseudo SNR-dependent linear thresholding that takes account of the speech presence in each band. Since the algorithm is unsupervised, it is suitable for any SNR level and any stationary additive noise, unlike supervised methods such as neural networks, which may perform inadequately under unseen environments. To verify the performance of the proposed method, speaker verification experiments with both male and female data were made. Five different noise types at five different overall SNR levels were added to the test files. Both conventional UBM and state-of-the-art i-vector frameworks were investigated.

2 Polynomial regression on mel spectrum

The proposed method can be separated into two parts. As the first part, the polynomial regression, k-means classification, and speech enhancement applied on the filter bank outputs are explained in Sections 2.1, 2.2, 2.3, 2.4, and 2.5. For the second part, based on the results from the first part of the proposed algorithm, the final VAD output decision is given in Section 2.6.

2.1 Motivation

The VADs, in general, use parameters, such as periodicity, harmonicity, and long-term variability, or energy levels to detect speech regions, then make frame-based speech/non-speech decision. For the voiced phonemes, these parameters could be sufficient, because of their harmonic structure and relatively high energy. However, the unvoiced phonemes may be treated as noise since they do not possess harmonic content as the voiced phonemes do and have less energy. To overcome this problem, polynomial regression is applied on the output of each filter band of the mel scale spectrum, and the frames are grouped together by finding the best fitting polynomial over the range of five to ten frames. For a 25-ms frame with a 10-ms overlap, this range is equal to 65–115 ms, which covers the average duration of a vowel-like (i.e., vowel, semivowel, and diphthong) region in a continuous speech [29]. Another important issue is that the temporal contour of speech energy of a filter bank output approximately resembles a bell-like shape in the spectrum, i.e., energy level rising from the speech onset point and vice versa. This phenomenon was also considered in [37], where an end point detection method was proposed with the aid of the aforementioned rising and falling energy levels (called beginning and ending edges, respectively). With a basic energy thresholding for VAD, low energy frames, especially unvoiced phonemes, are likely to be suppressed in low SNR levels. In the proposed algorithm, the grouped frames are represented by the average energy of the fitted polynomial, which can be thought as an energy boost for the low energy frames in a speech region. Also, this representation avoids the misclassifications of sudden energy ripples (i.e., noise frame with a higher energy than its neighbor noise frames, or speech frame with a lower energy than its neighbor speech frames). It should be noted that a single polynomial may not capture the entire bell-like shape of a speech segment, due to the frame range (5–10) limitations. Instead, the minimum expectation is to capture at least the rising and falling edges and the peak regions.

As a preliminary experiment, the proposed algorithm is going to be tested on a sample speech signal to illustrate its effectiveness. The speech file used in the tests is an utterance spoken by a male speaker from the Noizeus corpus [38], with an 8 kHz sampling frequency. The mel spectrum of the signal is shown in Fig. 1a, which is obtained by using 25 ms long frames with 10 ms overlap, windowing with a Hamming window, taking 1024 points Fourier transform, and filtering with 26 filters that are equally spaced on the mel scale between 300 Hz and 4 kHz. Note that these parameters are the same throughout this section and also for the speaker verification experiments in Section 3. The lynx noise from the NOISEX-92 database [39] is mixed with the clean speech to obtain a noisy speech signal with a desired overall SNR level (5 dB in this case). Note that the NOISEX-92 database has a 16 kHz sampling frequency. Noise files were down-sampled to 8 kHz to match the sampling rates of speech and noise data. To achieve this, first, the NOISEX-92 data were filtered by a low pass filter (almost ideal low pass) with a cut-off frequency of 4 kHz. Then, the data were down-sampled by a factor of two. The mel spectrum of the degraded signal is illustrated in Fig. 1b.

Fig. 1

Mel spectra of the a clean signal and b the noisy signal with a 5 dB overall SNR

The effect of the noise can be easily observed by comparing the spectra given in Fig. 1. The speech signals with low energies vanish into the noise. However, the regions with higher energies are still visible in the spectrum. The ultimate goal of the proposed algorithm is to detect and enhance these frames by using the polynomial regression, followed by the spectral subtraction.

2.2 A brief review of polynomial regression

Before detailing the proposed algorithm, reminding the general polynomial regression expressions with least squares sense could be beneficial. A kth order polynomial is defined as a ₀ + a ₁ x + … + a _k x ^k, where x is the intermediate variable, and a’s are the coefficients of the polynomial. The least squares method minimizes the summed difference (error) of the observed value and the estimated value. This error can be defined as in Eq. 1.

$$ E=\sum_{i=1}^n{\left[{y}_i-\left({a}_0+{a}_1{x}_i+\dots +{a}_k{x}_i^k\right)\right]}^2 $$

(1)

where y is the observation vector (filter bank output vector for a given band), n is the length of the vector (number of frames considered for regression, 5 to 10), and i is the index (frame index). In the proposed method, x _i  = i. The optimum value of a coefficient is the value that makes the partial derivative of the error, with respect to the coefficient, equal to zero (Eq. 2).

$$ {\displaystyle \begin{array}{lll}\frac{\partial E}{\partial {a}_0}\hfill & =-2\sum \limits_{i=1}^n\left[{y}_i-\left({a}_0+{a}_1{x}_i+\dots +{a}_k{x}_i^k\right)\right]\hfill & =0\hfill \\ {}\hfill & \vdots \hfill & \vdots \hfill \\ {}\frac{\partial E}{\partial {a}_k}\hfill & =-2\sum \limits_{i=1}^n\left[{y}_i-\left({a}_0+{a}_1{x}_i+\dots +{a}_k{x}_i^k\right)\right]{x}_i^k\hfill & =0\hfill \end{array}} $$

(2)

Taking the terms with y to one side, these equations can be written in a matrix form as given in Eq. 3. Equation 3 can be expanded as X ^T y = X ^T Xa, where X is defined as in Eq. 4. Then, the coefficients of the polynomial can be found as a = ( X ^T X ) ⁻¹ X ^T y.

$$ \left[\begin{array}{llll}n\hfill & \sum \limits_{i=1}^n{x}_i\hfill & \cdots \hfill & \sum \limits_{i=1}^n{x}_i^k\hfill \\ {}\sum \limits_{i=1}^n{x}_i\hfill & \sum \limits_{i=1}^n{x}_i^2\hfill & \cdots \hfill & \sum \limits_{i=1}^n{x}_i^{k+1}\hfill \\ {}\vdots \hfill & \vdots \hfill & \ddots \hfill & \vdots \hfill \\ {}\sum \limits_{i=1}^n{x}_i^k\hfill & \sum \limits_{i=1}^n{x}_i^{k+1}\hfill & \cdots \hfill & \sum \limits_{i=1}^n{x}_i^{2k}\hfill \end{array}\right]\left[\begin{array}{l}\begin{array}{l}{a}_0\\ {}\end{array}\hfill \\ {}\begin{array}{l}{a}_1\\ {}\end{array}\hfill \\ {}\vdots \hfill \\ {}{a}_k\hfill \end{array}\right]=\left[\begin{array}{c}\hfill \sum \limits_{i=1}^n{y}_i\hfill \\ {}\hfill \sum \limits_{i=1}^n{x}_i{y}_i\hfill \\ {}\hfill \vdots \hfill \\ {}\hfill \sum \limits_{i=1}^n{x}_i^k{y}_i\hfill \end{array}\right] $$

(3)

$$ X=\left[\begin{array}{lllll}1\hfill & {x}_1\hfill & {x}_1^2\hfill & \cdots \hfill & {x}_1^k\hfill \\ {}1\hfill & {x}_2\hfill & {x}_2^2\hfill & \cdots \hfill & {x}_2^k\hfill \\ {}\vdots \hfill & \vdots \hfill & \vdots \hfill & \ddots \hfill & \vdots \hfill \\ {}1\hfill & {x}_n\hfill & {x}_n^2\hfill & \cdots \hfill & {x}_n^k\hfill \end{array}\right] $$

(4)

2.3 Application of polynomial regression on mel spectrum

The order of the polynomial, “k,” is chosen as two in the proposed method. The reason behind this choice is that a first-order polynomial is a straight line, which is not suitable to follow the variations of a highly nonstationary signal. On the other hand, as the order of the polynomial increases, the fitted signal takes values closer to the real signal but the computational load increases. For the proposed algorithm, only an approximation of the general trend over the frames is sufficient; hence, the polynomial order was chosen as two. Also, it should be noted that no substantial advantage was found by using a third-degree polynomial.

The regression is applied in each band of the mel spectrum, directly to the smoothed filter bank energies. It can be expanded to the conventional spectrum; however, increased number of the frequency bins will also increase the computation time. Also, the mel spectrum is a part of the MFCC extraction process; hence, the proposed algorithm fits seamlessly in the conventional framework.

Let S(t,m) denote the filter bank output of the noisy speech signal for frame t and filter m. The filter bank outputs are first smoothed to reduce the ripples (Eq. 5).

$$ {S}^s\left(t,m\right)=\sum_{n=-2}^2{p}_nS\left(t+n,m\right) $$

(5)

where, S ^s(t,m) is the smoothed filter bank output, p _n is the smoothing coefficient with p ₋₂ = p ₂ = 0.1, p ₋₁ = p ₁ = 0.2, and p ₀ = 0.4. The smoothed filter bank outputs are then subjected to the polynomial regression in each filter band independently. The regression error is defined as the normalized distance between the smoothed filter bank outputs and the fitted polynomial (Eq. 6).

$$ {e}_{N,m}=\frac{\sqrt{\sum_{i=0}^{N-1}{\left({S}^s\left(t+i,m\right)-{F}_N\left(t+i,m\right)\right)}^2}}{N},N=5,6,\dots, 10 $$

(6)

where F _N (t,m) is the value of the fitted polynomial at frame t and filter bank m. N is the number of the frames used for regression, and e _N,m is the error observed for N-length fitting. The consecutive frames, which give the minimum e _N,m , are grouped and represented by the average of the fitted polynomial (i.e., average energy of the group). Starting from the next frame, this process continues until all frames have been represented by their respective average polynomial energy. As an example, consider first 25 frames. F is calculated from 1st frame to 5th, then 1st to 6th,…1st to 10th. Assume that minimum error is found in the range of 6 frames. So, frames 1, 2,…6 represents a group. Next, F is calculated from 7th frame to 11th, then 7th to 12th, and so on. In the end, the number of the segments is at most D/5, where D is the total number of the frames.

In Fig. 2, the results of the polynomial regression are visualized for a better understanding of the proposed algorithm, using the same degraded signal shown in Section 2.1. The blue-colored line shows the smoothed filter bank outputs of the first filter. The red line shows the frame groups represented by their respective polynomial’s average energy (note the horizontal parts). Same illustration is made for the filter 24, given in Fig. 3.

Fig. 2

Results of the regression and the k-means in filter 1

Fig. 3

Results of the regression and the k-means in filter 24

2.4 Two class k-means algorithm

After representing all of the frames within a band as energy levels, a threshold is necessary to determine the noise-dominant or the speech-dominant groups. In order to define a threshold level, the well-known k-means algorithm is used, influenced from the success of the bi-Gaussian modeling approach [28]. The k-means algorithm can be thought as a simplified version of Gaussian mixture models, where the weights and the variances of the classes are assumed to be equal. Also, since only two classes are needed and the number of the data points (frames) is low (at most D/5 points as described above), a simple classification should be sufficient. The k-means algorithm gives two energy levels as class centroids. The frames that belong to the higher energy class are assumed to be speech-dominant frames. On the other hand, low energy speech regions contribute to the lower energy class. Therefore, the lower energy center is chosen as the threshold and only the frames below this threshold are treated as noise-dominant frames. Then, the average noise energy in the band can be calculated by using these frames.

The horizontal lines in Figs. 2 and 3 show the class centroids found by the k-means algorithm. The lower line (dashed), which is the centroid of the lower energy class, is used as the threshold level for the filter band. The frames below the threshold are used to estimate the average noise energy in each band.

It can be observed from the figures that as the speech energy decreases, the class centroids come closer to each other. This situation is clearly seen by comparing the class centers of the two filter bands. Filter 24 covers the higher frequencies, where less speech energy is found usually. Therefore, the difference between the class centroids is reduced. As the overall SNR is decreased, it is expected that the class centroids in the other filters also come closer. This information will be used in Section 2.6 to estimate how noisy the signal is.

2.5 Speech enhancement

Once the noise energy for a band has been estimated, the spectral subtraction method can be used to enhance the speech information. To avoid the spectrum becoming too small, an energy floor is also included as expressed in Eq. 7.

$$ {S}^e\left(t,m\right)=\max \left({S}^s\left(t,m\right)-N(m),0.001{S}^s\left(t,m\right)\right) $$

(7)

where S ^e(t,m) is the enhanced signal and N(m) is the estimated noise energy for mth filter.

At the end of the regression and k-means processes, each of the filter bank outputs is divided into one of the two classes, indicating the reliable and unreliable components similar to the missing data method as mentioned in Section 1. However, the binary matrix of the proposed algorithm is used in a way that it aids the final decisions on the frames. Let the ones in the matrix denote the reliable components and the zeros denote the unreliable components. If the ones dominated a frame, it is highly probable that the frame is a speech frame. However, as the noise level increases, the number of ones also tends to increase, due to misclassifying the noise frames as speech in filter bands. Therefore, a constant threshold value for all SNR levels is not suitable for the final decision. A pseudo SNR-dependent threshold called “clarity level” is defined to tackle the aforementioned problem, which is detailed in the following subsection. The procedures described above to obtain binary representation of the frames can be summarized as a block diagram given in Fig. 4.

Fig. 4

Block diagram of the processes to observe enhanced speech information and binary representation of the frames in a given band

2.6 Clarity level and VAD output

The clarity level is a parameter that is related to the k-means class centroids in a band. Mathematically, it is defined as in Eq. 8.

$$ L=\frac{\sum_{m=1}^M{\mathrm{log}}_{10}\left({C}^{hi}(m)/{C}^{low}(m)\right)}{M} $$

(8)

where L is the clarity level, C ^hi(m) is the centroid of the speech-dominant class, and C ^low(m) is the centroid of the noise-dominant class, of mth filter, and M is the total number of filters in the filter bank. It should be noted that C ^low(m) is also used as the threshold as described in Section 2.4.

As the noise level in the signal increases, the value of the L decreases. This is mostly due to the fact that the class centroids get closer to each other since the speech vanishes into the noise. Therefore, if the clarity level is high, the signal is treated as a high SNR signal and a small number of ones found in a frame of the binary matrix is sufficient to determine that the frame is dominated by the speech. If the clarity level is low, more evidence is needed to depict a frame as the speech, so the total number of ones in a frame should be more.

The relation between the clarity level and the sufficient speech evidence value for a signal can be expressed as in Eq. 9, which is found by setting the limits for the best and worst SNR cases as 7 and 23, respectively, then simply calculating the equation of the line between them.

$$ {L}^s=\Big\{{\displaystyle \begin{array}{l}7,L>0.8\kern1.00em \\ {}\mathrm{round}\left(28.36-{25.45}^{\ast }L\right),0.8\ge L\ge 0.25\kern1.00em \\ {}23,L<0.25\kern1.00em \end{array}}\operatorname{} $$

(9)

where L ^s is the minimum number of ones required for a frame to be detected as speech (sufficient speech evidence number) and is calculated per utterance. After comparing the ones in the frames of the binary matrix with the sufficient speech evidence value, frames detected as the noise are eliminated and the other frames observed as the final VAD output are subjected to discrete cosine transform to obtain the MFCCs.

The best-case limit of L ^s, 7, is determined by assuming the speech signal should cover at least several bands for a clean signal, and it is found that this limit is not critical as the worst-case limit (choosing 6, 7, or 8 gave similar verification performances). On the other hand, the worst-case limit (L < 0.25) indicates a severely degraded signal; therefore, much more evidence is needed. Eighteen and 23 were examined as the worst-case limits, and L ^s = 23 improved speaker verification performance in terms of equal error rate (EER) for the low SNR levels (− 5 and − 10 dB) as high as absolute 10% compared to L ^s = 18. Hence, L ^s = 23 is chosen as the worst-case limit. This result also supports the assumption made above; more speech evidence in a frame is needed for the low SNR levels. Note that total number of the filters in the bank is 26, and the worst-case limits were not reached for the SNR levels used in the experiments, they rather adjust the slope of the threshold line. The values of L were determined on a sample signal described in Section 2.1, which is not included in any part of the speaker verification process. Values higher than 0.8 indicate almost clean speech (SNR > 15 dB), and values lower than 0.25 correspond to noisy speech (SNR < − 10 dB).

The VAD decision process is summarized as a block diagram in Fig. 5.

Fig. 5

VAD final decision processes as a block diagram

As a preliminary experiment, noisy speech spectrum, enhanced speech spectrum, and final VAD output decisions are shown together in Fig. 6 for the utterance degraded with the lynx noise at 0 dB overall SNR value. It is clear that the proposed algorithm can detect relatively high energy regions, but more proofs are needed to understand if the proposed method adds robustness to recognition systems or deteriorates the speech/speaker information. Therefore, speaker verification experiments are conducted with five different noises and five different SNR levels for each noise in the next section.

Fig. 6

a Noisy speech signal with a 0 dB overall SNR, b enhanced speech signal, and c final VAD output (only the frames detected as speech-dominant) of the proposed algorithm

3 Speaker verification experiments

3.1 Description of the experimental setup

In the experiments, 250 male and 250 female speakers from the NIST SRE 1998 database were used, where there are approximately 5 min training data per speaker. For the tests, speech segments with 30 s duration were used. There are 1308 test speech files for male speakers and 1379 test files for female speakers. For each test file, there is one trial for the target speaker and nine trials for the non-target speakers. A more detailed analysis of the database can be found in [40]. A simple energy-based VAD as given in [41] was applied to clean training data to eliminate silence. Since the training data is clean, the type of VAD will not make much difference on the verification performance. For each speaker, only one model is trained by using the speaker’s training data. The proposed and baseline methods are applied only to the test utterances. In the test phase, the features were extracted only from the speech-dominant frames for all methods. The feature vectors used in the experiments were 26-dimensional (13 MFCCs, excluding the zeroth coefficient and their deltas).

For the back end, GMM-UBM [1] and the i-vector [2] methods were considered. For the GMM-UBM, two gender-dependent UBM models with 1024 mixtures were trained by pooling all available gender-dependent training data. The speaker models were then adapted from their respective gender’s UBM with a relevance factor of 16. For the i-vector framework, same UBMs and the pooled training data were used to train the total variability matrix in 20 iterations, and then, 100 dimensional i-vectors were extracted from each utterance. Linear discriminant analysis (LDA) was used to reduce the channel mismatch effects, and probabilistic LDA was employed for scoring the i-vectors. MSR Identity Toolbox [42] was used in all of the classification and scoring phases.

Two methods were selected to compare the performance of the proposed VAD algorithm. One of these methods was the noise tracking algorithm proposed in [21] (called Rangachari’s method from here on). This algorithm works on the frequency bins of the conventional spectrum, but, to make a fair comparison, it was modified to work on the mel spectrum, similar to the algorithm proposed in this paper. Also, the noise tracking algorithm does not explicitly indicate the speech-active regions (although it gives a speech presence probability, it is not suitable to use it directly as a VAD output). Therefore, after the noise spectrum was estimated, and the speech was enhanced with the spectral subtraction method, the frames that had an energy higher than the average energy of all frames were accepted as speech regions. All of the other parameters were the same as given in [21].

The other method was another recently proposed VAD [27] (called Drugman’s method from here on). Four voicing measures of [26], MFCCs, and two pitch trackers were used as the features, and a neural network with a single hidden layer of 32 neurons was used to obtain posterior speech probabilities of frames. The reason for choosing this method for comparison was that it had achieved superior results against the four state-of-the-art VAD methods, as reported in [27]. Also, its memory and computational demands are low, and it includes smoothing processes before and after the neural network phase to remove spurious values and isolated misclassifications, respectively. The method is implemented by using the code obtained from the author’s website [43]. For the spectral subtraction, frames decided as non-speech were used to estimate average noise energy.

3.2 Experimental results

In the verification phase, the test data was degraded with Lynx, F16, car, babble, and Stitel noises from the NOISEX-92 database. The overall SNR levels were changed from − 10 dB to 10 dB with 5 dB steps. EER was selected as the main performance metric, which is a widely accepted measurement in the speaker verification literature. Also, a detection cost function (DCF) was defined as given in Eq. 10,

$$ DCF={C}_{FP}{P}_{FP\left|N\right.}{P}_N+{C}_{FN}{P}_{FN\left|T\right.}{P}_T $$

(10)

where P _FP|N is the false positive rate (FPR), P _FN|T is the false negative rate (FNR), the cost of the false acceptance is C _FP  = 10, the cost of the false rejection is C _FN  = 1, the a priori probability of target tests is P _T  = 0.1, and the a priori probability of nontarget tests is P _N  = 0.9. The other performance metric was the minimum of the DCF. The results of the experiments with the male speakers using the GMM-UBM method are given in Table 1 for all noise types and all SNR levels. The minDCF values are given in parenthesis. Also, the last two columns of the table show the relative percent EER reduction between the proposed algorithm and the others. Similarly, the experimental results for the female speakers using the GMM-UBM method are given in Table 2. Table 3 shows the relative percent EER reductions for each noise level (averaged over all noise types). In addition to the tables, EER values averaged over all noise types are given as bar graphs in Figs. 7 and 8 for the male speakers and the female speakers, respectively.

Table 1 Male speaker verification results of GMM-UBM method in terms of percent EER (minDCF) for the proposed algorithm, Drugman’s VAD method [27], and Rangachari’s noise tracking method [21]. The last columns show the relative percent EER reduction rates compared to Drugman’s VAD and Rangachari’s method, respectively

Table 2 Female speaker verification results of GMM-UBM method in terms of percent EER (minDCF) for the proposed algorithm, Drugman’s VAD method [27], and Rangachari’s noise tracking method [21]. The last columns show the relative percent EER reduction rates compared to Drugman’s VAD and Rangachari’s method, respectively

Table 3 SNR-based relative percent EER reduction rates for GMM-UBM method compared to Drugman’s VAD (second and fourth column) and Rangachari’s method (third and fifth column). Computed by averaging over all noise types

Fig. 7

EER values averaged over all noise types for the male data with GMM/UBM backend

Fig. 8

EER values averaged over all noise types for the female data with GMM/UBM backend

Before proceeding to the results for the i-vector method, it may be beneficial to verify that the polynomial regression is the main reason for the increased verification performance. For this purpose, the k-means classification is applied directly to the frames in the bands, without using the polynomial regression. All other parts of the algorithm (noise estimation, sufficient speech evidence thresholding, etc.) are the same. Table 4 shows the results of this case for the female data under the lynx noise. It is clear that directly classifying the frames as speech and noise is not as effective as the polynomial representation. The performance of the system without the polynomial regression is worse than the Drugman’s VAD. Therefore, it is clear that the performance of the proposed algorithm can be attributed to both the polynomial regression and the sufficient speech evidence threshold.

Table 4 Results of the proposed algorithm with and without the polynomial regression using GMM-UBM method. Female data degraded with the lynx noise are used

The experimental results for the male and female speakers using the i-vector method are given in Tables 5 and 6, respectively. Table 7 shows the relative percent EER reductions for each noise level (averaged over all noise types). The relative EER reduction rates are also shown in the last two columns of Tables 5 and 6. EER values averaged over all noise types are given as bar graphs in Figs. 9 and 10 for the male speakers and the female speakers, respectively.

Table 5 Male speaker verification results of i-vector method in terms of percent EER (minDCF) for the proposed algorithm, Drugman’s VAD method [27], and Rangachari’s noise tracking method [21]. The last columns show the relative percent EER reduction rates compared to Drugman’s VAD and Rangachari’s method, respectively

Table 6 Female speaker verification results of i-vector method in terms of percent EER (minDCF) for the proposed algorithm, Drugman’s VAD method [27], and Rangachari’s noise tracking method [21]. The last columns show the relative percent EER reduction rates compared to Drugman’s VAD and Rangachari’s method, respectively

Table 7 SNR-based relative percent EER reduction rates for i-vector method compared to Drugman’s VAD (second and fourth column) and Rangachari’s method (third and fifth column). Computed by averaging over all noise types

Fig. 9

EER values averaged over all noise types for the female data with i-vector backend

Fig. 10

EER values averaged over all noise types for the female data with i-vector backend

3.3 Discussion

As seen in the tables given above, the proposed algorithm showed better verification performances than the other methods except a few cases for the GMM-UBM method. It is also shown that all methods’ performances increased by using the i-vector technique, but the proposed method benefits from the modeling capacities of the i-vectors more than the others. The superior performance of the proposed method may be due to the selection of useful speaker/speech information from the noisy speech signal. These regions were effectively extracted with the aid of the polynomial regression, and the sufficient speech evidence thresholding technique. Also, some of the low energy speech regions may be recovered by the polynomial regression if related frames also exceed the threshold. Contrary to this, frames with high energy speech only in a few bands may be discarded, due to the presence of the noise in most of the bands, which are expected to degrade the system’s performance.

It is verified that the linear function chosen for determining the sufficient speech evidence is a good approximation to extract speech/speaker information under noise. Instead of a linear function between the end points, other types of functions, such as an exponential, may increase the verification performance. However, the authors prefer to analyze the performance differences of these functions in a future work to avoid an excessively long paper.

Drugman’s VAD works with a conventional VAD’s principle: detects speech regions, but does not produce any information about their usefulness, which probably deteriorates the verification performance. Hence, for the text-independent verification, a conventional VAD does not offer any advantages over the proposed algorithm.

Rangachari’s noise tracking algorithm gave the worst overall verification performance. As discussed in Section 3.1, since the enhanced speech is treated as a clean speech, only the frames with an energy that is higher than the average energy were used for verification in this method. Instead of this kind of thresholding, a more suitable way to extract the speech-dominant regions may positively affect its results. Also, the speaker information may be damaged while the algorithm was trying to track the noise.

4 Conclusions

A novel algorithm is proposed in this work to extract speaker information in a robust manner. The core of the algorithm is the polynomial regression, applied in each filter band. Also, speech presence in each band is considered for a VAD-like final decision. The thresholding method, called sufficient speech evidence, increases the verification performance, especially for the low SNR levels. The algorithm also fits seamlessly to the conventional MFCC extraction scheme. The algorithm does not strictly search speech on/off points; instead, the focus is to extract most informative frames. Therefore, it is much more suitable for text-independent systems. In the experiments, the proposed algorithm was compared to a recently introduced neural network-based VAD and a speech enhancement algorithm that successfully tracks the additive noise signal. The proposed algorithm produced superior results than the others with both the conventional GMM-UBM system and the state-of-the-art i-vector system. It was verified that the frames selected by the proposed method captured more speaker information than a VAD.