3 ms·
My understanding of convolutions is that it's a way of extracting patterns from images. To convert audio into an image and then create convolutions from that se
by personjerry 9y ago
My understanding of convolutions is that it's a way of extracting patterns from images. To convert audio into an image and then create convolutions from that seems... convoluted, if you will. I imagine a better way would be to think of what the equivalent of a convolution would be in the audio space? I.e. noise detection, treble/bass filters, etc.?
- Matumio 9y agoConvolution is generic signal-processing. It's quite common to use a one-dimensional convolution for audio filters, it would work perfectly fine as a bass filter for example. However, 2D conv+maxpool is an image processing technique that gets you translation invariance. Fine for the time dimension of the spectrogram, but rather dubious for the frequency axis. Surely you'd want to distinguish if some feature happens at a high or low frequency?
- ssalazar 9y ago> Fine for the time dimension of the spectrogram, but rather dubious for the frequency axis. MFCCs[1] are exactly that, a type of convolution along the frequency axis of a Fourier transform, and are highly apt features for music classification tasks. It makes sense if you think of timbre as a time-varying relationship between the harmonics of a single pitch; translation invariance along the frequency axis can tell you that you there are partials typical e.g. of a guitar or of a flute, without caring what particular pitch those instruments are playing. And timbre is a bigger source of variety in popular music than e.g. the particular notes used. [1] https://en.wikipedia.org/wiki/Mel-frequency_cepstrum https://en.wikipedia.org/wiki/Mel-frequency_cepstrum