7 ms·

This is amazing. The GitHub repo has more information too, about how the common explanation of how our ears work (isolating sounds in an FFT-like-process) may b

This is amazing. The GitHub repo has more information too, about how the common explanation of how our ears work (isolating sounds in an FFT-like-process) may be incorrect: > Contrary to what you might think, our ears don't seem to rely on an FFT-like process to extract isolated frequencies. Instead, our ears detect periodic parts in the signal, although in most cases those periodic parts closely match the FFT frequencies. There is a simple experiment that proves this point: https://github.com/soundshader/soundshader.github.io#why-acf-matches-our-perception-of-sound https://github.com/soundshader/soundshader.github.io#why-acf...

jcims 5y ago
This is my laymans test for the FFT theory. Put in earbuds or a headset and (quietly) play sounds that are slightly different in frequency (1-10hz). You will still hear a sort of ethereal beat tone between them that's different than the beat you would hear if you were listening to the same through speakers. I don't see how this would occur frequency-domain perception and it's too present (at least in my experiment) to be attributed to bone conduction across the skull.
- morelisp 5y ago
  https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beats https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beat...
  - jcims 5y ago
    Whoa! At least that gives me some comfort I wasn't imagining it. I wish I understood some of the neurological explanation of how the beats are perceived because it doesn't seem to match with my understanding of the explanation of how our ears work. If everything is pushed into the frequency domain based on the stimulation of different parts of the cochlea, where does the time domain beat emerge?
    - filoeleven 5y ago
      Part of why we evolved two ears is to be able to locate sounds within our perceptive field. I think the best example is to listen to some recordings made with two head-mounted microphones. I like these here; one was the closest I’ve come to believing I was standing in a pond while in my own house. It requires headphones to get the effect: https://quietamerican.org/field_vietnam.html https://quietamerican.org/field_vietnam.html There’s another component at play too: beat frequencies. This happens anytime you have different frequencies playing simultaneously. This is a result of simple waveform interference. Lots of examples, but I’ll never pass up a chance to link to Julius Sumner Miller[0]: https://youtu.be/7dxkW5bsUgs https://youtu.be/7dxkW5bsUgs So the brain is doing lots of work to integrate the stereo “image,” in much the same way we can wear 3D glasses and perceive depth[1]. Binaural beats reduce things down to a more fundamental level: you’re playing with how your mind integrates the stereo field in a weird way, and it produces a beat frequency that does not exist in the pulsed air. This may be learned behavior. [0] I’m eagerly waiting for some music producer to sample this video: “all the music fell out”, “we should have this mechanism called beats”, “beats: wonderful!”, etc. [1] I wonder what the effect of rapidly switching the left/right components of a stereo image would be. Probably nausea.
      - jcims 5y ago
        The part that doesn't make sense to me is that the binaural beat frequency corresponds with the physical beat frequency of the sound. So if you got 432Hz in one ear and 428Hz in the other, you're going to hear a 4Hz beat frequency between the two. If the cochlea is effectively taste buds for sound, the only thing the brain is going to get is which part of the cochlea is being tickled. There's no time domain information there, just some ambiguous 'pitch'. If that's the case though, how does the brain know to synthesize the 4Hz differential between these two frequencies. The 432Hz and the 428Hz aren't making it to the brain, just the fact both ears are getting tickled in very close but different places. (Also my dad absolutely LOVED watching JSM and would always call us into the room any time he was doing one of his crazy experiments on TV. I agree his stuff is very 'sampleable' Edit: Just watched the video, it's actually a gold mine for hip hop lol. Just play this in the background and scrub around his videos - https://www.youtube.com/watch?v=JVISRjhXzzM https://www.youtube.com/watch?v=JVISRjhXzzM good thing it's friday (may need to fix volume) - http://www.youtubemultiplier.com/6095af3ba32b6-jsm-on-beats-on-dre.php http://www.youtubemultiplier.com/6095af3ba32b6-jsm-on-beats-...)
- crazygringo 5y ago
  As far as I know, we don't know the exact mechanism by which binaural beats are produced, but it does seem to be from FFT information. The hair cells in our cochlea function essentially as a great big FFT -- this is well-established -- and so our source neural input begins with that. The brain doesn't have access to the underlying waveform at all, as far as I'm aware. It is incredibly sensitive to the timing from each ear though (just as FFT includes phase information). You can read all about what we do know here: https://en.wikipedia.org/wiki/Neuronal_encoding_of_sound https://en.wikipedia.org/wiki/Neuronal_encoding_of_sound Our brain does perform advanced signal processing to condense sets of overtones into a single fundamental frequency, which even works in the case of a "missing fundamental", and binaural beats are conceivably explained by this mechanism. Though it could be a different mechanism at work as well, related to how we process audio spatially.
crazygringo 5y ago
I don't believe that's correct. The hair cells in our inner ear are activated in response to specific frequencies, virtually identical to how an FFT operates. There is then further processing attached to that, in the same way that we perceive colors rather than cones directly in our vision. With audio, we need to collapse entire series of overtones into a fundamental frequency with a timbre (color). And specifically in the experiment linked, it's showing how the ear intelligently restores the missing fundamental. But I don't believe there's any biological/neurological evidence to support that the ear detects periodic repetitions of an overall waveform, and that this explains the missing fundamental. To the contrary, how our hearing works biologically suggests that we are indeed performing something like FFT, but then applying sophisticated pattern recognition on it -- on the "FFT", not on the waveform.

jcims 5y ago

This is my laymans test for the FFT theory. Put in earbuds or a headset and (quietly) play sounds that are slightly different in frequency (1-10hz). You will still hear a sort of ethereal beat tone between them that's different than the beat you would hear if you were listening to the same through speakers. I don't see how this would occur frequency-domain perception and it's too present (at least in my experiment) to be attributed to bone conduction across the skull.

morelisp 5y ago
https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beats https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beat...
- jcims 5y ago
  Whoa! At least that gives me some comfort I wasn't imagining it. I wish I understood some of the neurological explanation of how the beats are perceived because it doesn't seem to match with my understanding of the explanation of how our ears work. If everything is pushed into the frequency domain based on the stimulation of different parts of the cochlea, where does the time domain beat emerge?
  - filoeleven 5y ago
    Part of why we evolved two ears is to be able to locate sounds within our perceptive field. I think the best example is to listen to some recordings made with two head-mounted microphones. I like these here; one was the closest I’ve come to believing I was standing in a pond while in my own house. It requires headphones to get the effect: https://quietamerican.org/field_vietnam.html https://quietamerican.org/field_vietnam.html There’s another component at play too: beat frequencies. This happens anytime you have different frequencies playing simultaneously. This is a result of simple waveform interference. Lots of examples, but I’ll never pass up a chance to link to Julius Sumner Miller[0]: https://youtu.be/7dxkW5bsUgs https://youtu.be/7dxkW5bsUgs So the brain is doing lots of work to integrate the stereo “image,” in much the same way we can wear 3D glasses and perceive depth[1]. Binaural beats reduce things down to a more fundamental level: you’re playing with how your mind integrates the stereo field in a weird way, and it produces a beat frequency that does not exist in the pulsed air. This may be learned behavior. [0] I’m eagerly waiting for some music producer to sample this video: “all the music fell out”, “we should have this mechanism called beats”, “beats: wonderful!”, etc. [1] I wonder what the effect of rapidly switching the left/right components of a stereo image would be. Probably nausea.
    - jcims 5y ago
      The part that doesn't make sense to me is that the binaural beat frequency corresponds with the physical beat frequency of the sound. So if you got 432Hz in one ear and 428Hz in the other, you're going to hear a 4Hz beat frequency between the two. If the cochlea is effectively taste buds for sound, the only thing the brain is going to get is which part of the cochlea is being tickled. There's no time domain information there, just some ambiguous 'pitch'. If that's the case though, how does the brain know to synthesize the 4Hz differential between these two frequencies. The 432Hz and the 428Hz aren't making it to the brain, just the fact both ears are getting tickled in very close but different places. (Also my dad absolutely LOVED watching JSM and would always call us into the room any time he was doing one of his crazy experiments on TV. I agree his stuff is very 'sampleable' Edit: Just watched the video, it's actually a gold mine for hip hop lol. Just play this in the background and scrub around his videos - https://www.youtube.com/watch?v=JVISRjhXzzM https://www.youtube.com/watch?v=JVISRjhXzzM good thing it's friday (may need to fix volume) - http://www.youtubemultiplier.com/6095af3ba32b6-jsm-on-beats-on-dre.php http://www.youtubemultiplier.com/6095af3ba32b6-jsm-on-beats-...)
crazygringo 5y ago
As far as I know, we don't know the exact mechanism by which binaural beats are produced, but it does seem to be from FFT information. The hair cells in our cochlea function essentially as a great big FFT -- this is well-established -- and so our source neural input begins with that. The brain doesn't have access to the underlying waveform at all, as far as I'm aware. It is incredibly sensitive to the timing from each ear though (just as FFT includes phase information). You can read all about what we do know here: https://en.wikipedia.org/wiki/Neuronal_encoding_of_sound https://en.wikipedia.org/wiki/Neuronal_encoding_of_sound Our brain does perform advanced signal processing to condense sets of overtones into a single fundamental frequency, which even works in the case of a "missing fundamental", and binaural beats are conceivably explained by this mechanism. Though it could be a different mechanism at work as well, related to how we process audio spatially.

crazygringo 5y ago

I don't believe that's correct. The hair cells in our inner ear are activated in response to specific frequencies, virtually identical to how an FFT operates. There is then further processing attached to that, in the same way that we perceive colors rather than cones directly in our vision. With audio, we need to collapse entire series of overtones into a fundamental frequency with a timbre (color). And specifically in the experiment linked, it's showing how the ear intelligently restores the missing fundamental. But I don't believe there's any biological/neurological evidence to support that the ear detects periodic repetitions of an overall waveform, and that this explains the missing fundamental. To the contrary, how our hearing works biologically suggests that we are indeed performing something like FFT, but then applying sophisticated pattern recognition on it -- on the "FFT", not on the waveform.