4 ms·

This kind of problem has been extensively studied for years and it turns out that it is not that easy. The fundamental is not the strongest harmonic, the harmon

This kind of problem has been extensively studied for years and it turns out that it is not that easy. The fundamental is not the strongest harmonic, the harmonics don't line up, what happens when you play multiple notes at the same time, etc. It is a big stinking mess.

iainctduncan 6y ago
And that's before even separating out by instrument! It's really quite incredible what goes on between the ear and the brain for this kind of thing.
- pishpash 6y ago
  Correlative methods are the wrong model by themselves but could be one part of the right model. Cognition happens by correlation, but also by denoising, by adaptive filtering, by interference cancellation, even by predictive generation. Listening is not a passive process.
creato 6y ago
There's a game called Rocksmith that has been on the market for many years that basically does this in real time on live guitar playing with far more than enough accuracy to make the game fun. And that is doing it basically per-note! This task (aligning sheet music with MIDI) would allow for global optimization approaches which should be far easier and more likely to work.
- kevingadd 6y ago
  Electric guitar going through a clean (ish) input is a pretty narrowly scoped problem that won't necessarily generalize
  - Benjamin_Dobell 6y ago
    MIDI music is far cleaner than analogue input from a guitar.
dragontamer 6y ago
I'm no expert, but isn't that just autocorrelation? (https://en.wikipedia.org/wiki/Autocorrelation https://en.wikipedia.org/wiki/Autocorrelation) Autocorrelation is the algorithm used for Radar (at least, that's what my professors told me). When radar bounces off of another object, there are different "echos" of that radar (ex: an object was 10 miles away and 20 miles away: the 20-mile object will take 2x longer to come back). Its messy and everything. The radar input is very messy, full of reflections, echos, and more. But autocorrelation takes all of that information, and tells you where the objects were.
- klodolph 6y ago
  Autocorrelation is a good start, IF there is only one note playing at a time, and then it SOMETIMES works. You also get multiple peaks in the autocorrelation and it is not obvious which one to choose as the fundamental frequency.
- pishpash 6y ago
  There is an equivalent problem in audio, which is source localization. That's much simpler than note cognition.

iainctduncan 6y ago

And that's before even separating out by instrument! It's really quite incredible what goes on between the ear and the brain for this kind of thing.

pishpash 6y ago
Correlative methods are the wrong model by themselves but could be one part of the right model. Cognition happens by correlation, but also by denoising, by adaptive filtering, by interference cancellation, even by predictive generation. Listening is not a passive process.

creato 6y ago

There's a game called Rocksmith that has been on the market for many years that basically does this in real time on live guitar playing with far more than enough accuracy to make the game fun. And that is doing it basically per-note! This task (aligning sheet music with MIDI) would allow for global optimization approaches which should be far easier and more likely to work.

kevingadd 6y ago
Electric guitar going through a clean (ish) input is a pretty narrowly scoped problem that won't necessarily generalize
- Benjamin_Dobell 6y ago
  MIDI music is far cleaner than analogue input from a guitar.

dragontamer 6y ago

I'm no expert, but isn't that just autocorrelation? (https://en.wikipedia.org/wiki/Autocorrelation https://en.wikipedia.org/wiki/Autocorrelation) Autocorrelation is the algorithm used for Radar (at least, that's what my professors told me). When radar bounces off of another object, there are different "echos" of that radar (ex: an object was 10 miles away and 20 miles away: the 20-mile object will take 2x longer to come back). Its messy and everything. The radar input is very messy, full of reflections, echos, and more. But autocorrelation takes all of that information, and tells you where the objects were.

klodolph 6y ago
Autocorrelation is a good start, IF there is only one note playing at a time, and then it SOMETIMES works. You also get multiple peaks in the autocorrelation and it is not obvious which one to choose as the fundamental frequency.
pishpash 6y ago
There is an equivalent problem in audio, which is source localization. That's much simpler than note cognition.