Marvin Tammen, Simon Doclo

To improve speech quality and intelligibility in environments with noise and interfering sounds, binaural speech enhancement algorithms use the microphone signals from both the left and the right hearing device to generate an enhanced output signal for each ear. As a multi-frame extension of the binaural multi-channel Wiener filter, in this paper we consider the binaural spatio- temporal Wiener filter (STWF) in the short-time Fourier transform domain, which requires estimates of the highly time-varying spatio-temporal correlations of the speech and interference components. To this end, the binaural STWF is embedded into an end-to-end supervised learning framework, where temporal convolutional networks estimate the required quantities, i.e., the inverse spatio-temporal correlation matrices of the interference component and the spatio-temporal correlation vectors and power spectral densities of the speech component. In this paper, we investigate the potential of imposing spatio-temporal correlation structure on these quantities and relating these quantities between the left and the right hearing device. Assuming that the spatial correlation of the speech component is stationary over a small number of frames, we propose to decompose the spatio-temporal correlation vectors as the Kronecker product of a relative transfer function vector and a temporal correlation vector, either considering a global reference microphone or a reference microphone for each hearing device. In addition, we consider a deep bilateral STWF by neglecting the spatio-temporal correlations of the speech and interference components between both devices. The imposed spatio-temporal correlation structures greatly differ in the number of parameters that need to be estimated. Simulation results based on simulated binaural room impulse responses and diverse speech and noise sources demonstrate that the proposed spatio- temporal correlation structures significantly reduce the computational complexity of the binaural STWF while yielding a similar speech enhancement performance compared to not imposing any spatio-temporal correlation structure. Furthermore, the results confirm that the deep binaural STWF outperforms the binaural Conv-TasNet algorithm as well as directly estimating the binaural multi-frame filter coefficients.

Code

The implementations of the deep binaural and bilateral STWF algorithms as well as the baseline algorithms will be published upon acceptance of the manuscript.

Audio Demos