2 ms·
> No checksum is unique. Your problem is to do with the probability of collision. Thanks :-) Yeah the question should be more accurate :-) In the description I
by sandreas 4y ago
> No checksum is unique. Your problem is to do with the probability of collision.
Thanks :-) Yeah the question should be more accurate :-) In the description I already use "avoid collisions".
> My guess is that hashing a very small part of the file would suffice as long as it contains actual audio (not silence)
Yeah, I had that in mind. The problem is, that this would go deeply into audio analysis (silence detection would be slower than hashing the whole file), so this is all into probability. It also is important, how many files the libary contains and my question is exactly about that: How to avoid collsions while still being fast enough to index a libray with > 10000 audio files (audio books and music).
> If you are concerned that you may have different edits of the same audio, you could try skipping through taking a small sample from every n seconds.
That is more likely to happen on music than on audio books. Therefore I asked about hashing one big chunk of e.g. 5MB. If the file is smaller, it hashes the whole file, if it is bigger, it hashes only a part in the middle of the file...
I also thought of a combination of properties: PartialHash and FullHash only calculating the full hash, if the PartialHash has an acutual collision. That would make it kind of robust for edge cases but also fast in the average case.
But I'm trying to keep it simple...