Filterbank API¶
Filterbank, Encoder and Decoder¶

class
asteroid.filterbanks.
Filterbank
(n_filters, kernel_size, stride=None)[source]¶ Bases:
sphinx.ext.autodoc.importer._MockObject
Base Filterbank class. Each subclass has to implement a filters property.
Parameters: Variables: n_feats_out (int) – Number of output filters.

filters
¶ Abstract method for filters.


class
asteroid.filterbanks.
Encoder
(filterbank, is_pinv=False, as_conv1d=True, padding=0)[source]¶ Bases:
asteroid.filterbanks.enc_dec._EncDec
Encoder class.
Add encoding methods to Filterbank classes. Not intended to be subclassed.
Parameters:  filterbank (
Filterbank
) – The filterbank to use as an encoder.  is_pinv (bool) – Whether to be the pseudo inverse of filterbank.
 as_conv1d (bool) – Whether to behave like nn.Conv1d. If True (default), forwarding input with shape (batch, 1, time) will output a tensor of shape (batch, freq, conv_time). If False, will output a tensor of shape (batch, 1, freq, conv_time).
 padding (int) – Zeropadding added to both sides of the input.

forward
(waveform)[source]¶ Convolve input waveform with the filters from a filterbank. :param waveform: any tensor with samples along the
last dimension. The waveform representation with and batch/channel etc.. dimension.Returns: torch.Tensor
– The corresponding TF domain signal. Shapes:
>>> (time, ) > (freq, conv_time) >>> (batch, time) > (batch, freq, conv_time) # Avoid >>> if as_conv1d: >>> (batch, 1, time) > (batch, freq, conv_time) >>> (batch, chan, time) > (batch, chan, freq, conv_time) >>> else: >>> (batch, chan, time) > (batch, chan, freq, conv_time) >>> (batch, any, dim, time) > (batch, any, dim, freq, conv_time)

classmethod
pinv_of
(filterbank, **kwargs)[source]¶ Returns an
Encoder
, pseudo inverse of aFilterbank
orDecoder
.
 filterbank (

class
asteroid.filterbanks.
Decoder
(filterbank, is_pinv=False, padding=0, output_padding=0)[source]¶ Bases:
asteroid.filterbanks.enc_dec._EncDec
Decoder class.
Add decoding methods to Filterbank classes. Not intended to be subclassed.
Parameters:  filterbank (
Filterbank
) – The filterbank to use as an decoder.  is_pinv (bool) – Whether to be the pseudo inverse of filterbank.
 padding (int) – Zeropadding added to both sides of the input.
 output_padding (int) – Additional size added to one side of the output shape.
 Notes
 padding and output_padding arguments are directly passed to F.conv_transpose1d.

forward
(spec)[source]¶ Applies transposed convolution to a TF representation.
This is equivalent to overlapadd.
Parameters: spec ( torch.Tensor
) – 3D or 4D Tensor. The TF representation. (Output ofEncoder.forward()
).Returns: torch.Tensor
– The corresponding time domain signal.
 filterbank (

class
asteroid.filterbanks.
make_enc_dec
[source]¶ Creates congruent encoder and decoder from the same filterbank family.
Parameters:  fb_name (str, className) – Filterbank family from which to make encoder
and decoder. To choose among [
'free'
,'analytic_free'
,'param_sinc'
,'stft'
]. Can also be a class defined in a submodule in this subpackade (e.g.FreeFB
).  n_filters (int) – Number of filters.
 kernel_size (int) – Length of the filters.
 stride (int, optional) – Stride of the convolution.
If None (default), set to
kernel_size // 2
.  who_is_pinv (str, optional) – If None, no pseudoinverse filters will
be used. If string (among [
'encoder'
,'decoder'
]), decides which ofEncoder
orDecoder
will be the pseudo inverse of the other one.  padding (int) – Zeropadding added to both sides of the input. Passed to Encoder and Decoder.
 output_padding (int) – Additional size added to one side of the output shape. Passed to Decoder.
 **kwargs – Arguments which will be passed to the filterbank class additionally to the usual n_filters, kernel_size and stride. Depends on the filterbank family.
Returns:  fb_name (str, className) – Filterbank family from which to make encoder
and decoder. To choose among [

class
asteroid.filterbanks.
get
[source]¶ Returns a filterbank class from a string. Returns its input if it is callable (already a
Filterbank
for example).Parameters: identifier (str or Callable or None) – the filterbank identifier. Returns: Filterbank
or None
Learnable filterbanks¶
Free¶

class
asteroid.filterbanks.free_fb.
FreeFB
(n_filters, kernel_size, stride=None, **kwargs)[source]¶ Bases:
asteroid.filterbanks.enc_dec.Filterbank
Free filterbank without any constraints. Equivalent to
nn.Conv1d
.Parameters: Variables: n_feats_out (int) – Number of output filters.
References
[1] : “Filterbank design for endtoend speech separation”. Submitted to ICASSP 2020. Manuel Pariente, Samuele Cornell, Antoine Deleforge, Emmanuel Vincent.

filters
¶ Abstract method for filters.

Analytic Free¶

class
asteroid.filterbanks.analytic_free_fb.
AnalyticFreeFB
(n_filters, kernel_size, stride=None, **kwargs)[source]¶ Bases:
asteroid.filterbanks.enc_dec.Filterbank
Free analytic (fully learned with analycity constraints) filterbank. For more details, see [1].
Parameters: Variables: n_feats_out (int) – Number of output filters.
References
[1] : “Filterbank design for endtoend speech separation”. Submitted to ICASSP 2020. Manuel Pariente, Samuele Cornell, Antoine Deleforge, Emmanuel Vincent.

filters
¶ Abstract method for filters.

Parameterized Sinc¶

class
asteroid.filterbanks.param_sinc_fb.
ParamSincFB
(n_filters, kernel_size, stride=None, sample_rate=16000, min_low_hz=50, min_band_hz=50)[source]¶ Bases:
asteroid.filterbanks.enc_dec.Filterbank
Extension of the parameterized filterbank from [1] proposed in [2]. Modified and extended from from https://github.com/mravanelli/SincNet
Parameters:  n_filters (int) – Number of filters. Half of n_filters (the real parts) will have parameters, the other half will correspond to the imaginary parts. n_filters should be even.
 kernel_size (int) – Length of the filters.
 stride (int, optional) – Stride of the convolution. If None (default),
set to
kernel_size // 2
.  sample_rate (int, optional) – The sample rate (used for initialization).
 min_low_hz (int, optional) – Lowest low frequency allowed (Hz).
 min_band_hz (int, optional) – Lowest band frequency allowed (Hz).
Variables: n_feats_out (int) – Number of output filters.
References
[1] : “Speaker Recognition from raw waveform with SincNet”. SLT 2018. Mirco Ravanelli, Yoshua Bengio. https://arxiv.org/abs/1808.00158
[2] : “Filterbank design for endtoend speech separation”. Submitted to ICASSP 2020. Manuel Pariente, Samuele Cornell, Antoine Deleforge, Emmanuel Vincent. https://arxiv.org/abs/1910.10400

filters
¶ Compute filters from parameters
Fixed filterbanks¶
STFT¶

class
asteroid.filterbanks.stft_fb.
STFTFB
(n_filters, kernel_size, stride=None, window=None, **kwargs)[source]¶ Bases:
asteroid.filterbanks.enc_dec.Filterbank
STFT filterbank.
Parameters:  n_filters (int) – Number of filters. Determines the length of the STFT filters before windowing.
 kernel_size (int) – Length of the filters (i.e the window).
 stride (int, optional) – Stride of the convolution (hop size). If None
(default), set to
kernel_size // 2
.  window (
numpy.ndarray
, optional) – If None, defaults tonp.sqrt(np.hanning())
.
Variables: n_feats_out (int) – Number of output filters.

filters
¶ Abstract method for filters.

asteroid.filterbanks.stft_fb.
perfect_synthesis_window
(analysis_window, hop_size)[source]¶  Computes a window for perfect synthesis given an analysis window and
 a hop size.
Parameters:  analysis_window (np.array) – Analysis window of the transform.
 hop_size (int) – Hop size in number of samples.
Returns: np.array – the synthesis window to use for perfectly inverting the STFT.
MPGTFB¶

class
asteroid.filterbanks.multiphase_gammatone_fb.
MultiphaseGammatoneFB
(n_filters=128, kernel_size=16, sample_rate=8000, stride=None, **kwargs)[source]¶ Bases:
asteroid.filterbanks.enc_dec.Filterbank
MultiPhase Gammatone Filterbank as described in [1]. Please cite [1] whenever using this. Original code repository: <https://github.com/spuhh/mpgtf>
Parameters: References: [1] David Ditter, Timo Gerkmann, “A MultiPhase Gammatone Filterbank for
Speech Separation via TasNet”, ICASSP 2020 Available: <https://ieeexplore.ieee.org/document/9053602/>
filters
¶ Abstract method for filters.


asteroid.filterbanks.multiphase_gammatone_fb.
erb_scale_2_freq_hz
(freq_erb)[source]¶ Convert frequency on ERB scale to frequency in Hertz

asteroid.filterbanks.multiphase_gammatone_fb.
freq_hz_2_erb_scale
(freq_hz)[source]¶ Convert frequency in Hertz to frequency on ERB scale
Transforms¶
GriffinLim and MISI¶

asteroid.filterbanks.griffin_lim.
griffin_lim
(mag_specgram, stft_enc, angles=None, istft_dec=None, n_iter=6, momentum=0.9)[source]¶ Estimates matching phase from magnitude spectogram using the ‘fast’ Griffin Lim algorithm [1].
Parameters:  mag_specgram (torch.Tensor) – (any, dim, ension, freq, frames) as returned by Encoder(STFTFB), the magnitude spectrogram to be inverted.
 stft_enc (Encoder[STFTFB]) – The Encoder(STFTFB()) object that was used to compute the input mag_spec.
 angles (None or Tensor) – Angles to use to initialize the algorithm. If None (default), angles are init with uniform ditribution.
 istft_dec (None or Decoder[STFTFB]) – Optional Decoder to use to get back to the time domain. If None (default), a perfect reconstruction Decoder is built from stft_enc.
 n_iter (int) – Number of griffinlim iterations to run.
 momentum (float) – The momentum of fast GriffinLim. Original GriffinLim is obtained for momentum=0.
Returns: torch.Tensor – estimated waveforms of shape (any, dim, ension, time).
Examples
>>> stft = Encoder(STFTFB(n_filters=256, kernel_size=256, stride=128)) >>> wav = torch.randn(2, 1, 8000) >>> spec = stft(wav) >>> masked_spec = spec * torch.sigmoid(torch.randn_like(spec)) >>> mag = transforms.take_mag(masked_spec, 2) >>> est_wav = griffin_lim(mag, stft, n_iter=32)
References
[1] Perraudin et al. “A fast GriffinLim algorithm,” WASPAA 2013. [2] D. W. Griffin and J. S. Lim: “Signal estimation from modified shorttime Fourier transform,” ASSP 1984.

asteroid.filterbanks.griffin_lim.
misi
(mixture_wav, mag_specgrams, stft_enc, angles=None, istft_dec=None, n_iter=6, momentum=0.0, src_weights=None, dim=1)[source]¶ Jointly estimates matching phase from magnitude spectograms using the Multiple Input Spectrogram Inversion (MISI) algorithm [1].
Parameters:  mixture_wav (torch.Tensor) – (batch, time)
 mag_specgrams (torch.Tensor) – (batch, n_src, freq, frames) as returned by Encoder(STFTFB), the magnitude spectrograms to be jointly inverted using MISI (modified or not).
 stft_enc (Encoder[STFTFB]) – The Encoder(STFTFB()) object that was used to compute the input mag_spec.
 angles (None or Tensor) – Angles to use to initialize the algorithm. If None (default), angles are init with uniform ditribution.
 istft_dec (None or Decoder[STFTFB]) – Optional Decoder to use to get back to the time domain. If None (default), a perfect reconstruction Decoder is built from stft_enc.
 n_iter (int) – Number of MISI iterations to run.
 momentum (float) – Momentum on updates (this argument comes from GriffinLim). Defaults to 0 as it was never proposed anywhere.
 src_weights (None or torch.Tensor) – Consistency weight for each source. Shape needs to be broadcastable to istft_dec(mag_specgrams). We make sure that the weights sum up to 1 along dim dim. If src_weights is None, compute them based on relative power.
 dim (int) – Axis which contains the sources in mag_specgrams. Used for consistency constraint.
Returns: torch.Tensor – estimated waveforms of shape (batch, n_src, time).
Examples
>>> stft = Encoder(STFTFB(n_filters=256, kernel_size=256, stride=128)) >>> wav = torch.randn(2, 3, 8000) >>> specs = stft(wav) >>> masked_specs = specs * torch.sigmoid(torch.randn_like(specs)) >>> mag = transforms.take_mag(masked_specs, 2) >>> est_wav = misi(wav.sum(1), mag, stft, n_iter=32)
References
[1] Gunawan and Sen, “Iterative Phase Estimation for the Synthesis of Separated Sources From SingleChannel Mixtures,” in IEEE Signal Processing Letters, 2010. [2] Wang, LeRoux et al. “EndtoEnd Speech Separation with Unfolded Iterative Phase Reconstruction.” Interspeech 2018 (2018)
Complex transforms¶

asteroid.filterbanks.transforms.
angle
(tensor, dim=2)[source]¶ Return the angle of the complexlike torch tensor.
Parameters:  tensor (torch.Tensor) – the complex tensor from which to extract the phase.
 dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns: torch.Tensor
– The counterclockwise angle from the positive real axis on the complex plane in radians.

asteroid.filterbanks.transforms.
apply_complex_mask
(tf_rep, mask, dim=2)[source]¶ Applies a complexvalued mask to a complexvalued representation.
Operands are assumed to have the real parts of each entry followed by the imaginary parts of each entry along dimension dim, e.g. for,
dim = 1
, the matrixis interpreted as
where j is such that j * j = 1.
Parameters:  tf_rep (
torch.Tensor
) – The time frequency representation to apply the mask to.  (class (mask) – torch.Tensor): The complexvalued mask to be applied.
 dim (int) – The frequency (or equivalent) dimension of both tf_rep an mask along which real and imaginary values are concatenated.
Returns: torch.Tensor
– tf_rep multiplied by the mask in the complex sense. tf_rep (

asteroid.filterbanks.transforms.
apply_mag_mask
(tf_rep, mask, dim=2)[source]¶ Applies a realvalued mask to a complexvalued representation.
If tf_rep has 2N elements along dim, mask has N elements, mask is duplicated along dim to apply the same mask to both the Re and Im.
tf_rep is assumed to have the real parts of each entry followed by the imaginary parts of each entry along dimension dim, e.g. for,
dim = 1
, the matrixis interpreted as
where j is such that j * j = 1.
Parameters:  tf_rep (
torch.Tensor
) – The time frequency representation to apply the mask to. Re and Im are concatenated along dim.  mask (
torch.Tensor
) – The realvalued mask to be applied.  dim (int) – The frequency (or equivalent) dimension of both tf_rep and mask along which real and imaginary values are concatenated.
Returns: torch.Tensor
– tf_rep multiplied by the mask. tf_rep (

asteroid.filterbanks.transforms.
apply_real_mask
(tf_rep, mask, dim=2)[source]¶ Applies a realvalued mask to a realvalued representation.
It corresponds to ReIm mask in [1].
Parameters:  tf_rep (
torch.Tensor
) – The time frequency representation to apply the mask to.  mask (
torch.Tensor
) – The realvalued mask to be applied.  dim (int) – Kept to have the same interface with the other ones.
Returns: torch.Tensor
– tf_rep multiplied by the mask. tf_rep (

asteroid.filterbanks.transforms.
check_complex
(tensor, dim=2)[source]¶ Assert tensor in complexlike in a given dimension.
Parameters:  tensor (torch.Tensor) – tensor to be checked.
 dim (int) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Raises: AssertionError if dimension is not even in the specified dimension

asteroid.filterbanks.transforms.
ebased_vad
(mag_spec, th_db=40)[source]¶ Compute energybased VAD from a magnitude spectrogram (or equivalent).
Parameters:  mag_spec (torch.Tensor) – the spectrogram to perform VAD on. Expected shape (batch, *, freq, time). The VAD mask will be computed independently for all the leading dimensions until the last two. Independent of the ordering of the last two dimensions.
 th_db (int) – The threshold in dB from which a TFbin is considered silent.
Returns: torch.BoolTensor, the VAD mask.
Examples
>>> import torch >>> mag_spec = torch.abs(torch.randn(10, 2, 65, 16)) >>> batch_src_mask = ebased_vad(mag_spec)

asteroid.filterbanks.transforms.
from_mag_and_phase
(mag, phase, dim=2)[source]¶ Return a complexlike torch tensor from magnitude and phase components.
Parameters:  mag (torch.tensor) – magnitude of the tensor.
 phase (torch.tensor) – angle of the tensor
 dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns: torch.Tensor
– The corresponding complexlike torch tensor.

asteroid.filterbanks.transforms.
from_numpy
(array, dim=2)[source]¶ Convert complex numpy array to complexlike torch tensor.
Parameters:  array (np.array) – array to be converted.
 dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns: torch.Tensor
– Corresponding torch.Tensor (complex axis in dim `dim`=

asteroid.filterbanks.transforms.
from_torchaudio
(tensor, dim=2)[source]¶ Converts torchaudio style complex tensor to complexlike torch tensor.
Parameters:  tensor (torch.tensor) – torchaudiostyle complexlike torch tensor.
 dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns: torch.Tensor
– asteroidstyle complexlike torch tensor.

asteroid.filterbanks.transforms.
mul_c
(inp, other, dim=2)[source]¶ Entrywise product for complex valued tensors.
Operands are assumed to have the real parts of each entry followed by the imaginary parts of each entry along dimension dim, e.g. for,
dim = 1
, the matrixis interpreted as
where j is such that j * j = 1.
Parameters:  inp (
torch.Tensor
) – The first operand with real and imaginary parts concatenated on the dim axis.  other (
torch.Tensor
) – The second operand.  dim (int, optional) – frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns: torch.Tensor
– The complex multiplication between inp and otherFor now, it assumes that other has the same shape as inp along dim.
 inp (

asteroid.filterbanks.transforms.
take_mag
(x, dim=2)[source]¶ Takes the magnitude of a complex tensor.
The operands is assumed to have the real parts of each entry followed by the imaginary parts of each entry along dimension dim, e.g. for,
dim = 1
, the matrixis interpreted as
where j is such that j * j = 1.
Parameters:  x (
torch.Tensor
) – Complex valued tensor.  dim (int) – frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns: torch.Tensor
– The magnitude of x. x (

asteroid.filterbanks.transforms.
to_numpy
(tensor, dim=2)[source]¶ Convert complexlike torch tensor to numpy complex array
Parameters:  tensor (torch.Tensor) – Complex tensor to convert to numpy.
 dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns: numpy.array
– Corresponding complex array.

asteroid.filterbanks.transforms.
to_torchaudio
(tensor, dim=2)[source]¶ Converts complexlike torch tensor to torchaudio style complex tensor.
Parameters:  tensor (torch.tensor) – asteroidstyle complexlike torch tensor.
 dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns: torch.Tensor
– torchaudiostyle complexlike torch tensor.