Filterbank API

Filterbank, Encoder and Decoder

class asteroid.filterbanks.Filterbank(n_filters, kernel_size, stride=None)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Base Filterbank class. Each subclass has to implement a filters property.

Parameters:
  • n_filters (int) – Number of filters.
  • kernel_size (int) – Length of the filters.
  • stride (int, optional) – Stride of the conv or transposed conv. (Hop size). If None (default), set to kernel_size // 2.
Variables:

n_feats_out (int) – Number of output filters.

get_config()[source]

Returns dictionary of arguments to re-instantiate the class.

filters

Abstract method for filters.

class asteroid.filterbanks.Encoder(filterbank, is_pinv=False, as_conv1d=True, padding=0)[source]

Bases: asteroid.filterbanks.enc_dec._EncDec

Encoder class.

Add encoding methods to Filterbank classes. Not intended to be subclassed.

Parameters:
  • filterbank (Filterbank) – The filterbank to use as an encoder.
  • is_pinv (bool) – Whether to be the pseudo inverse of filterbank.
  • as_conv1d (bool) – Whether to behave like nn.Conv1d. If True (default), forwarding input with shape (batch, 1, time) will output a tensor of shape (batch, freq, conv_time). If False, will output a tensor of shape (batch, 1, freq, conv_time).
  • padding (int) – Zero-padding added to both sides of the input.
forward(waveform)[source]

Convolve input waveform with the filters from a filterbank. :param waveform: any tensor with samples along the

last dimension. The waveform representation with and batch/channel etc.. dimension.
Returns:torch.Tensor – The corresponding TF domain signal.
Shapes:
>>> (time, ) --> (freq, conv_time)
>>> (batch, time) --> (batch, freq, conv_time)  # Avoid
>>> if as_conv1d:
>>>     (batch, 1, time) --> (batch, freq, conv_time)
>>>     (batch, chan, time) --> (batch, chan, freq, conv_time)
>>> else:
>>>     (batch, chan, time) --> (batch, chan, freq, conv_time)
>>> (batch, any, dim, time) --> (batch, any, dim, freq, conv_time)
classmethod pinv_of(filterbank, **kwargs)[source]

Returns an Encoder, pseudo inverse of a Filterbank or Decoder.

class asteroid.filterbanks.Decoder(filterbank, is_pinv=False, padding=0, output_padding=0)[source]

Bases: asteroid.filterbanks.enc_dec._EncDec

Decoder class.

Add decoding methods to Filterbank classes. Not intended to be subclassed.

Parameters:
  • filterbank (Filterbank) – The filterbank to use as an decoder.
  • is_pinv (bool) – Whether to be the pseudo inverse of filterbank.
  • padding (int) – Zero-padding added to both sides of the input.
  • output_padding (int) – Additional size added to one side of the output shape.
Notes
padding and output_padding arguments are directly passed to F.conv_transpose1d.
forward(spec)[source]

Applies transposed convolution to a TF representation.

This is equivalent to overlap-add.

Parameters:spec (torch.Tensor) – 3D or 4D Tensor. The TF representation. (Output of Encoder.forward()).
Returns:torch.Tensor – The corresponding time domain signal.
classmethod pinv_of(filterbank)[source]

Returns an Decoder, pseudo inverse of a filterbank or Encoder.

class asteroid.filterbanks.make_enc_dec[source]

Creates congruent encoder and decoder from the same filterbank family.

Parameters:
  • fb_name (str, className) – Filterbank family from which to make encoder and decoder. To choose among ['free', 'analytic_free', 'param_sinc', 'stft']. Can also be a class defined in a submodule in this subpackade (e.g. FreeFB).
  • n_filters (int) – Number of filters.
  • kernel_size (int) – Length of the filters.
  • stride (int, optional) – Stride of the convolution. If None (default), set to kernel_size // 2.
  • who_is_pinv (str, optional) – If None, no pseudo-inverse filters will be used. If string (among ['encoder', 'decoder']), decides which of Encoder or Decoder will be the pseudo inverse of the other one.
  • padding (int) – Zero-padding added to both sides of the input. Passed to Encoder and Decoder.
  • output_padding (int) – Additional size added to one side of the output shape. Passed to Decoder.
  • **kwargs – Arguments which will be passed to the filterbank class additionally to the usual n_filters, kernel_size and stride. Depends on the filterbank family.
Returns:

Encoder, Decoder

class asteroid.filterbanks.get[source]

Returns a filterbank class from a string. Returns its input if it is callable (already a Filterbank for example).

Parameters:identifier (str or Callable or None) – the filterbank identifier.
Returns:Filterbank or None

Learnable filterbanks

Free

class asteroid.filterbanks.free_fb.FreeFB(n_filters, kernel_size, stride=None, **kwargs)[source]

Bases: asteroid.filterbanks.enc_dec.Filterbank

Free filterbank without any constraints. Equivalent to nn.Conv1d.

Parameters:
  • n_filters (int) – Number of filters.
  • kernel_size (int) – Length of the filters.
  • stride (int, optional) – Stride of the convolution. If None (default), set to kernel_size // 2.
Variables:

n_feats_out (int) – Number of output filters.

References

[1] : “Filterbank design for end-to-end speech separation”. Submitted to ICASSP 2020. Manuel Pariente, Samuele Cornell, Antoine Deleforge, Emmanuel Vincent.

filters

Abstract method for filters.

Analytic Free

class asteroid.filterbanks.analytic_free_fb.AnalyticFreeFB(n_filters, kernel_size, stride=None, **kwargs)[source]

Bases: asteroid.filterbanks.enc_dec.Filterbank

Free analytic (fully learned with analycity constraints) filterbank. For more details, see [1].

Parameters:
  • n_filters (int) – Number of filters. Half of n_filters will have parameters, the other half will be the hilbert transforms. n_filters should be even.
  • kernel_size (int) – Length of the filters.
  • stride (int, optional) – Stride of the convolution. If None (default), set to kernel_size // 2.
Variables:

n_feats_out (int) – Number of output filters.

References

[1] : “Filterbank design for end-to-end speech separation”. Submitted to ICASSP 2020. Manuel Pariente, Samuele Cornell, Antoine Deleforge, Emmanuel Vincent.

filters

Abstract method for filters.

Parameterized Sinc

class asteroid.filterbanks.param_sinc_fb.ParamSincFB(n_filters, kernel_size, stride=None, sample_rate=16000, min_low_hz=50, min_band_hz=50)[source]

Bases: asteroid.filterbanks.enc_dec.Filterbank

Extension of the parameterized filterbank from [1] proposed in [2]. Modified and extended from from https://github.com/mravanelli/SincNet

Parameters:
  • n_filters (int) – Number of filters. Half of n_filters (the real parts) will have parameters, the other half will correspond to the imaginary parts. n_filters should be even.
  • kernel_size (int) – Length of the filters.
  • stride (int, optional) – Stride of the convolution. If None (default), set to kernel_size // 2.
  • sample_rate (int, optional) – The sample rate (used for initialization).
  • min_low_hz (int, optional) – Lowest low frequency allowed (Hz).
  • min_band_hz (int, optional) – Lowest band frequency allowed (Hz).
Variables:

n_feats_out (int) – Number of output filters.

References

[1] : “Speaker Recognition from raw waveform with SincNet”. SLT 2018. Mirco Ravanelli, Yoshua Bengio. https://arxiv.org/abs/1808.00158

[2] : “Filterbank design for end-to-end speech separation”. Submitted to ICASSP 2020. Manuel Pariente, Samuele Cornell, Antoine Deleforge, Emmanuel Vincent. https://arxiv.org/abs/1910.10400

get_config()[source]

Returns dictionary of arguments to re-instantiate the class.

filters

Compute filters from parameters

Fixed filterbanks

STFT

class asteroid.filterbanks.stft_fb.STFTFB(n_filters, kernel_size, stride=None, window=None, **kwargs)[source]

Bases: asteroid.filterbanks.enc_dec.Filterbank

STFT filterbank.

Parameters:
  • n_filters (int) – Number of filters. Determines the length of the STFT filters before windowing.
  • kernel_size (int) – Length of the filters (i.e the window).
  • stride (int, optional) – Stride of the convolution (hop size). If None (default), set to kernel_size // 2.
  • window (numpy.ndarray, optional) – If None, defaults to np.sqrt(np.hanning()).
Variables:

n_feats_out (int) – Number of output filters.

filters

Abstract method for filters.

asteroid.filterbanks.stft_fb.perfect_synthesis_window(analysis_window, hop_size)[source]
Computes a window for perfect synthesis given an analysis window and
a hop size.
Parameters:
  • analysis_window (np.array) – Analysis window of the transform.
  • hop_size (int) – Hop size in number of samples.
Returns:

np.array – the synthesis window to use for perfectly inverting the STFT.

MP-GTFB

class asteroid.filterbanks.multiphase_gammatone_fb.MultiphaseGammatoneFB(n_filters=128, kernel_size=16, sample_rate=8000, stride=None, **kwargs)[source]

Bases: asteroid.filterbanks.enc_dec.Filterbank

Multi-Phase Gammatone Filterbank as described in [1]. Please cite [1] whenever using this. Original code repository: <https://github.com/sp-uhh/mp-gtf>

Parameters:
  • n_filters (int) – Number of filters.
  • kernel_size (int) – Length of the filters.
  • sample_rate (int, optional) – The sample rate (used for initialization).
  • stride (int, optional) – Stride of the convolution. If None (default), set to kernel_size // 2.

References: [1] David Ditter, Timo Gerkmann, “A Multi-Phase Gammatone Filterbank for

Speech Separation via TasNet”, ICASSP 2020 Available: <https://ieeexplore.ieee.org/document/9053602/>
filters

Abstract method for filters.

asteroid.filterbanks.multiphase_gammatone_fb.erb_scale_2_freq_hz(freq_erb)[source]

Convert frequency on ERB scale to frequency in Hertz

asteroid.filterbanks.multiphase_gammatone_fb.freq_hz_2_erb_scale(freq_hz)[source]

Convert frequency in Hertz to frequency on ERB scale

asteroid.filterbanks.multiphase_gammatone_fb.gammatone_impulse_response(samplerate_hz, len_sec, center_freq_hz, phase_shift)[source]

Generate single parametrized gammatone filter

asteroid.filterbanks.multiphase_gammatone_fb.normalize_filters(filterbank)[source]

Normalizes a filterbank such that all filters have the same root mean square (RMS).

Transforms

Griffin-Lim and MISI

asteroid.filterbanks.griffin_lim.griffin_lim(mag_specgram, stft_enc, angles=None, istft_dec=None, n_iter=6, momentum=0.9)[source]

Estimates matching phase from magnitude spectogram using the ‘fast’ Griffin Lim algorithm [1].

Parameters:
  • mag_specgram (torch.Tensor) – (any, dim, ension, freq, frames) as returned by Encoder(STFTFB), the magnitude spectrogram to be inverted.
  • stft_enc (Encoder[STFTFB]) – The Encoder(STFTFB()) object that was used to compute the input mag_spec.
  • angles (None or Tensor) – Angles to use to initialize the algorithm. If None (default), angles are init with uniform ditribution.
  • istft_dec (None or Decoder[STFTFB]) – Optional Decoder to use to get back to the time domain. If None (default), a perfect reconstruction Decoder is built from stft_enc.
  • n_iter (int) – Number of griffin-lim iterations to run.
  • momentum (float) – The momentum of fast Griffin-Lim. Original Griffin-Lim is obtained for momentum=0.
Returns:

torch.Tensor – estimated waveforms of shape (any, dim, ension, time).

Examples

>>> stft = Encoder(STFTFB(n_filters=256, kernel_size=256, stride=128))
>>> wav = torch.randn(2, 1, 8000)
>>> spec = stft(wav)
>>> masked_spec = spec * torch.sigmoid(torch.randn_like(spec))
>>> mag = transforms.take_mag(masked_spec, -2)
>>> est_wav = griffin_lim(mag, stft, n_iter=32)

References

[1] Perraudin et al. “A fast Griffin-Lim algorithm,” WASPAA 2013. [2] D. W. Griffin and J. S. Lim: “Signal estimation from modified short-time Fourier transform,” ASSP 1984.

asteroid.filterbanks.griffin_lim.misi(mixture_wav, mag_specgrams, stft_enc, angles=None, istft_dec=None, n_iter=6, momentum=0.0, src_weights=None, dim=1)[source]

Jointly estimates matching phase from magnitude spectograms using the Multiple Input Spectrogram Inversion (MISI) algorithm [1].

Parameters:
  • mixture_wav (torch.Tensor) – (batch, time)
  • mag_specgrams (torch.Tensor) – (batch, n_src, freq, frames) as returned by Encoder(STFTFB), the magnitude spectrograms to be jointly inverted using MISI (modified or not).
  • stft_enc (Encoder[STFTFB]) – The Encoder(STFTFB()) object that was used to compute the input mag_spec.
  • angles (None or Tensor) – Angles to use to initialize the algorithm. If None (default), angles are init with uniform ditribution.
  • istft_dec (None or Decoder[STFTFB]) – Optional Decoder to use to get back to the time domain. If None (default), a perfect reconstruction Decoder is built from stft_enc.
  • n_iter (int) – Number of MISI iterations to run.
  • momentum (float) – Momentum on updates (this argument comes from GriffinLim). Defaults to 0 as it was never proposed anywhere.
  • src_weights (None or torch.Tensor) – Consistency weight for each source. Shape needs to be broadcastable to istft_dec(mag_specgrams). We make sure that the weights sum up to 1 along dim dim. If src_weights is None, compute them based on relative power.
  • dim (int) – Axis which contains the sources in mag_specgrams. Used for consistency constraint.
Returns:

torch.Tensor – estimated waveforms of shape (batch, n_src, time).

Examples

>>> stft = Encoder(STFTFB(n_filters=256, kernel_size=256, stride=128))
>>> wav = torch.randn(2, 3, 8000)
>>> specs = stft(wav)
>>> masked_specs = specs * torch.sigmoid(torch.randn_like(specs))
>>> mag = transforms.take_mag(masked_specs, -2)
>>> est_wav = misi(wav.sum(1), mag, stft, n_iter=32)

References

[1] Gunawan and Sen, “Iterative Phase Estimation for the Synthesis of Separated Sources From Single-Channel Mixtures,” in IEEE Signal Processing Letters, 2010. [2] Wang, LeRoux et al. “End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction.” Interspeech 2018 (2018)

Complex transforms

asteroid.filterbanks.transforms.angle(tensor, dim=-2)[source]

Return the angle of the complex-like torch tensor.

Parameters:
  • tensor (torch.Tensor) – the complex tensor from which to extract the phase.
  • dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns:

torch.Tensor – The counterclockwise angle from the positive real axis on the complex plane in radians.

asteroid.filterbanks.transforms.apply_complex_mask(tf_rep, mask, dim=-2)[source]

Applies a complex-valued mask to a complex-valued representation.

Operands are assumed to have the real parts of each entry followed by the imaginary parts of each entry along dimension dim, e.g. for, dim = 1, the matrix

is interpreted as

where j is such that j * j = -1.

Parameters:
  • tf_rep (torch.Tensor) – The time frequency representation to apply the mask to.
  • (class (mask) – torch.Tensor): The complex-valued mask to be applied.
  • dim (int) – The frequency (or equivalent) dimension of both tf_rep an mask along which real and imaginary values are concatenated.
Returns:

torch.Tensortf_rep multiplied by the mask in the complex sense.

asteroid.filterbanks.transforms.apply_mag_mask(tf_rep, mask, dim=-2)[source]

Applies a real-valued mask to a complex-valued representation.

If tf_rep has 2N elements along dim, mask has N elements, mask is duplicated along dim to apply the same mask to both the Re and Im.

tf_rep is assumed to have the real parts of each entry followed by the imaginary parts of each entry along dimension dim, e.g. for, dim = 1, the matrix

is interpreted as

where j is such that j * j = -1.

Parameters:
  • tf_rep (torch.Tensor) – The time frequency representation to apply the mask to. Re and Im are concatenated along dim.
  • mask (torch.Tensor) – The real-valued mask to be applied.
  • dim (int) – The frequency (or equivalent) dimension of both tf_rep and mask along which real and imaginary values are concatenated.
Returns:

torch.Tensortf_rep multiplied by the mask.

asteroid.filterbanks.transforms.apply_real_mask(tf_rep, mask, dim=-2)[source]

Applies a real-valued mask to a real-valued representation.

It corresponds to ReIm mask in [1].

Parameters:
  • tf_rep (torch.Tensor) – The time frequency representation to apply the mask to.
  • mask (torch.Tensor) – The real-valued mask to be applied.
  • dim (int) – Kept to have the same interface with the other ones.
Returns:

torch.Tensortf_rep multiplied by the mask.

asteroid.filterbanks.transforms.check_complex(tensor, dim=-2)[source]

Assert tensor in complex-like in a given dimension.

Parameters:
  • tensor (torch.Tensor) – tensor to be checked.
  • dim (int) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Raises:

AssertionError if dimension is not even in the specified dimension

asteroid.filterbanks.transforms.ebased_vad(mag_spec, th_db=40)[source]

Compute energy-based VAD from a magnitude spectrogram (or equivalent).

Parameters:
  • mag_spec (torch.Tensor) – the spectrogram to perform VAD on. Expected shape (batch, *, freq, time). The VAD mask will be computed independently for all the leading dimensions until the last two. Independent of the ordering of the last two dimensions.
  • th_db (int) – The threshold in dB from which a TF-bin is considered silent.
Returns:

torch.BoolTensor, the VAD mask.

Examples

>>> import torch
>>> mag_spec = torch.abs(torch.randn(10, 2, 65, 16))
>>> batch_src_mask = ebased_vad(mag_spec)
asteroid.filterbanks.transforms.from_mag_and_phase(mag, phase, dim=-2)[source]

Return a complex-like torch tensor from magnitude and phase components.

Parameters:
  • mag (torch.tensor) – magnitude of the tensor.
  • phase (torch.tensor) – angle of the tensor
  • dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns:

torch.Tensor – The corresponding complex-like torch tensor.

asteroid.filterbanks.transforms.from_numpy(array, dim=-2)[source]

Convert complex numpy array to complex-like torch tensor.

Parameters:
  • array (np.array) – array to be converted.
  • dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns:

torch.Tensor – Corresponding torch.Tensor (complex axis in dim `dim`=

asteroid.filterbanks.transforms.from_torchaudio(tensor, dim=-2)[source]

Converts torchaudio style complex tensor to complex-like torch tensor.

Parameters:
  • tensor (torch.tensor) – torchaudio-style complex-like torch tensor.
  • dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns:

torch.Tensor – asteroid-style complex-like torch tensor.

asteroid.filterbanks.transforms.mul_c(inp, other, dim=-2)[source]

Entrywise product for complex valued tensors.

Operands are assumed to have the real parts of each entry followed by the imaginary parts of each entry along dimension dim, e.g. for, dim = 1, the matrix

is interpreted as

where j is such that j * j = -1.

Parameters:
  • inp (torch.Tensor) – The first operand with real and imaginary parts concatenated on the dim axis.
  • other (torch.Tensor) – The second operand.
  • dim (int, optional) – frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns:

torch.Tensor – The complex multiplication between inp and other

For now, it assumes that other has the same shape as inp along dim.

asteroid.filterbanks.transforms.take_mag(x, dim=-2)[source]

Takes the magnitude of a complex tensor.

The operands is assumed to have the real parts of each entry followed by the imaginary parts of each entry along dimension dim, e.g. for, dim = 1, the matrix

is interpreted as

where j is such that j * j = -1.

Parameters:
  • x (torch.Tensor) – Complex valued tensor.
  • dim (int) – frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns:

torch.Tensor – The magnitude of x.

asteroid.filterbanks.transforms.to_numpy(tensor, dim=-2)[source]

Convert complex-like torch tensor to numpy complex array

Parameters:
  • tensor (torch.Tensor) – Complex tensor to convert to numpy.
  • dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns:

numpy.array – Corresponding complex array.

asteroid.filterbanks.transforms.to_torchaudio(tensor, dim=-2)[source]

Converts complex-like torch tensor to torchaudio style complex tensor.

Parameters:
  • tensor (torch.tensor) – asteroid-style complex-like torch tensor.
  • dim (int, optional) – the frequency (or equivalent) dimension along which real and imaginary values are concatenated.
Returns:

torch.Tensor – torchaudio-style complex-like torch tensor.