3 ms·
> to turn every song into a hash Just to be clear, it's not turning each song into a hash. It's turning each song into many hundreds (thousands?) of hashes.
by crazygringo 3y ago
> to turn every song into a hash
Just to be clear, it's not turning each song into a hash.
It's turning each song into many hundreds (thousands?) of hashes.
And then you're looking for the greatest number of mostly-consecutive matches of tens (or low hundreds) of hashes from your shorter sample.
Also, I don't think this would be done with training a model today, because you're adding many, many new songs each day, that would necessitate constant retraining. Hashes are still going to be the superior approach, not just for efficiency but for robustness generally.
- brainfog 3y ago> that would necessitate constant retraining You wouldn't necessarily need to retrain that frequently. If your model outputs hashes / vectors that can be used for searching, you just need to run inference on your new data as it comes in.
- Legend2440 3y agoDefinitely this, embeddings would be the modern approach.
- raylad 3y agoThe "modern" approach sounds like it would be a lot worse than this approach both in terms of input and runtime performance. Trendy ("modern") is not necessarily better.
- casualscience 3y agoI'm an MLE, I would probably chop the songs into short segments, add noise (particularly trying to layer in people talking, room noise, and apply frequency-based filtering), and create a dataset like that. Then I would create contrastive embeddings with a hinge loss with a convnet on the spectrogram. Ultimately this looks the same, but the "hashes" come from a convnet now. But you still are doing some nearest neighbor thing to actually choose the best match. I imagine this is what 90% of MLEs would do, not sure if it would work better or worse than what Shazaam did. Prior to knowing Shazaam works, I might think this is a pretty hard problem, knowing Shazaam works, I am very confident the approach above would be competitive.
- osrec 3y agoWhy add noise to the training set, rather than attempt to denoise the input?
- wiz21c 3y agoTo start from an original song and move it towards something that resebme a real life recording ? IOW : make the NN learn to distinguish between the song sound and its environment ?
- omnster 3y agoI'd assume adding noise is done once per song and is thus a bit computationally cheaper than trying to denoise each input.
- bottled_poe 3y agoBecause then you’re training it on data that is more similar to the operating environment for the application. It’s a better fit for purpose. If the target environment was a clean audio signal, you’d optimise for that instead.
- sdenton4 3y agoSo you want a location-sensitive hash, or embedding, and you want it to be noise resistant. The ml approach is to define a family of data augmentations A, and a network N, such that for some augmentation f, we have N(f(x)) ~= N(x). Then we learn the weights of N, and on real data have N(x')~=N(x). The denoising approach is to define a set of denoising algorithms D and hash function H, so that H(D(x'))~=H(x). This largely relies on D(x')~=x, which may have real problems. So the neutral network learns the function we actually need, with the properties we want, where the denoiser is designed for a proxy problem. But that's not all... Eventually our noise model needs extending (eg, reverb is a problem): the ML approach adds a new set of augmentations to A. This is fine: it's easy to add new augmentations. But the denoiser might need some real algorithm work, and hope that there's no bad interaction with other parts of the pipeline, or too much additional compute overhead. (And de-reverb is notoriously hard.)
- mk67 3y ago
- iamflimflam1 3y agoYou would probably take the approach that is used for face recognition. You train a model that can tell if two faces are the “same”. Then you match an unknown face against your database of faces. You can get clever and pull out the “encoding” of a face from the trained model.
- teeray 3y agoDepends what the model targets. If I gave this problem to a bunch of musicians, they’d be pulling out features like the key, tempo, meter, chord progressions, any distinctive riffs or basslines, etc. Those are the things hashes could be built from and would be more information-dense than samples of the particular recording. Using a model to deconstruct a song like that might enable the ability to recognize someone playing the opening bars of Mr. Brightside on a piano in a loud bar as well as its drunkest patrons.
- lysecret 3y agoYou would just rename hashes to embeddings.