5 ms·
> there is a fundamental tradeoff between frequency resolution and time resolution I've always found it interesting that while that's fundamentally true in ter
by crazygringo 6y ago
> there is a fundamental tradeoff between frequency resolution and time resolution
I've always found it interesting that while that's fundamentally true in terms of information, my understanding is that we perceive things with far more resolution than the uncertainty principle would allow. Specifically, we're able to judge frequencies with far more accuracy than a fuzzy spectrogram would suggest.
From what I understand, our brain essentially performs a kind of "deconvolution" on the fuzzy frequency data to identify a far "sharper" and defined frequency, which is relatively straightforward since the frequency "spread" is a known quantity.
This works well most of the time because we correctly assume we're dealing with relatively isolated sound sources emanating a distinct fundamental with a distinct series of overtones.
Our perception can become innacurate when that assumption fails to hold, and so sounds merge or become indistinguishable, we hear beat tones that don't technically exist, our brain gives up trying to hear frequencies and classifies it all as noise, etc.
I've never come across audio spectrogram software that attempted to perform a frequency deconvolution in a way that roughly simulates what our own ears do, but I'd love to know if anyone else has and could point me to it.
- carlosf 6y agoSearch for voice coding. That's how many voice coding algorithms work, you try to find a digital filter that generates a sound that is as close as the original according to a perception based metric, then transmit the filter coefficients. I don't remember the exact details, but if I'm not mistaken generating this sort of metric is really time consuming.
- crazygringo 6y agoThanks! That's definitely along the same lines, although that's for the special case of only a single fundamental frequency (with overtones). And I'm not sure it uses deconvolution -- I've heard of maximum likelihood estimation being used, in order to get a single frequency, though the ideas are closely related. Using deconvolution would be more for the purposes of cleaning up a general-purpose spectrogram for human eyes -- for analysis and for sound editing, whether a single voice or a band of several instruments.
- Scene_Cast2 6y agoAnother part is that the human hearing is probably closer to Empirical Mode Decomposition (EMD) than a Fourier variant. With EMD, a phantom "beat frequency" would actually show up in the transform space.
- LeegleechN 6y agoThe purely algorithmic way to do it is the Wigner-Ville distribution, but it isn't practical for complex sounds due to the quadratic explosion of interactions between all time-frequency components. For a small number of well-separated 'chirp' signals it can give you exact localization. I think the software you are looking for would have to be based on a machine learning rather than purely theory-based approach if its intended for use with natural sound signals.
- deleted 6y ago[deleted]
- bradrn 6y agoI’m pretty sure I heard somewhere that the ear does autocorrelation rather than a Fourier transform, but I’m not sure how correct that is. EDIT: Scene_Cast2’s comment below says it’s Empirical Mode Decomposition, not autocorrelation. Anyway, I see no reason why spectrograms have to be fuzzy… a wide window size can locate frequencies very precisely while smoothing out fast variations in amplitude, which sounds pretty similar to how we hear things. (Interestingly, when analysing the voice, linguists tend to use the opposite: a narrow window size, which smears out frequencies making the resonance bands more obvious, while allowing visualisation of fast glottal vibrations.)
- zihotki 6y agoOur brain and ears perceive sound not as separate frequencies but as band of frequencies. Humans are not able to differentiate frequencies that are very close to each other because of that. We can identify separate musical notes because they fit into separate bands. Amir from AudioScienceReview did a good introductory video about the psychoacoustica as well as frequency response in general https://www.youtube.com/watch?v=TwGd0aMn1wE https://www.youtube.com/watch?v=TwGd0aMn1wE