> > To our surprise, we found a lack of published, well-optimized, large-data hash functions. Most hash work seems to focus on small input sizes (for things lik

> > To our surprise, we found a lack of published, well-optimized, large-data hash functions. Most hash work seems to focus on small input sizes (for things like dictionary lookup) or on cryptographic quality. I do still find this to be the case. I recently had to come up with a hash I could use for quickly IDing medium-sized data chunks (hundreds of MBs to small numbers of GB), with no need for cryptographic-level security. Best I could find after a surprisingly uninformative search was murmur3. I'm still not confident in my selection.