3 ms·

For the technically knowledgeable, music fingerprinting is a concrete problem which is understandable but pretty difficult if you get into the details without l

by TillE 3y ago

For the technically knowledgeable, music fingerprinting is a concrete problem which is understandable but pretty difficult if you get into the details without looking at how other people have already solved it. It fits into that space of an unusual but comprehensible problem, unlike superficially similar features like recognizing animals or objects in images, which is mostly weird ML magic.

hunter2_ 3y ago
I would say animals in images is more akin to matching two different musical performances of the same song (where one of the two can even be the user humming), which Shazam doesn't offer but some systems like Google Assistant do! Rather, matching two recordings of the exact same performance (one ingested by Shazam at training time and one ingested by Shazam at run time) is more akin to identifying individuals (facial recognition) than identifying species.
loxias 3y ago
It was really hard to do the first time. :) I'm honored to have been part of the first team to do any viable acoustic music recognition, in 2001 (much earlier than Shazam, a point of pride of course[0]). You're dead on that it's pretty difficult if you don't benefit from others, we did a ton of work that in retrospect wasn't necessary. I liked the advanced psychoacoustic model, faithfully implemented in high performant C direct from Zwicker. (Psychoacoustics). To a first approximation, about 10/s model -> pca -> top 16 dim -> VQ and the resulting bytes contain more than 50% of the entropy (!!) Shove all of those in a home grown what-you-now-call-a vector DB, do dozens of range queries, and search for any song common to multiple results. Boom, music recognition. Understandable in retrospect but things like that aren't Everest they're like... multiple unclimbed mountains. 0. And far too early to have any applications. Company existed 2000-2001 \o/
- garrettgrimsley 3y ago
  Looks like others, including Shazam, beat you to the punch in 2000: https://patents.google.com/patent/US7853664B1/ https://patents.google.com/patent/US7853664B1/ https://patents.google.com/patent/US6941275 https://patents.google.com/patent/US6941275 Very interesting hearing about all of the differing approaches people have taken to solving this problem! Do you have further writings on this topic?

by TillE 3y ago

hunter2_ 3y ago

I would say animals in images is more akin to matching two different musical performances of the same song (where one of the two can even be the user humming), which Shazam doesn't offer but some systems like Google Assistant do! Rather, matching two recordings of the exact same performance (one ingested by Shazam at training time and one ingested by Shazam at run time) is more akin to identifying individuals (facial recognition) than identifying species.

loxias 3y ago

It was really hard to do the first time. :) I'm honored to have been part of the first team to do any viable acoustic music recognition, in 2001 (much earlier than Shazam, a point of pride of course[0]). You're dead on that it's pretty difficult if you don't benefit from others, we did a ton of work that in retrospect wasn't necessary. I liked the advanced psychoacoustic model, faithfully implemented in high performant C direct from Zwicker. (Psychoacoustics). To a first approximation, about 10/s model -> pca -> top 16 dim -> VQ and the resulting bytes contain more than 50% of the entropy (!!) Shove all of those in a home grown what-you-now-call-a vector DB, do dozens of range queries, and search for any song common to multiple results. Boom, music recognition. Understandable in retrospect but things like that aren't Everest they're like... multiple unclimbed mountains. 0. And far too early to have any applications. Company existed 2000-2001 \o/

garrettgrimsley 3y ago
Looks like others, including Shazam, beat you to the punch in 2000: https://patents.google.com/patent/US7853664B1/ https://patents.google.com/patent/US7853664B1/ https://patents.google.com/patent/US6941275 https://patents.google.com/patent/US6941275 Very interesting hearing about all of the differing approaches people have taken to solving this problem! Do you have further writings on this topic?