3 ms·
We've been using phash for an image board for a while now and are quite happy with it. We only use it to detect reposts when someone uploads an image. It gives
by phoboslab 12y ago
We've been using phash for an image board for a while now and are quite happy with it. We only use it to detect reposts when someone uploads an image. It gives some false positives quite often, but that's totally okay for our use case. We specifically set it up to err on the safe side. Users are only presented with a "Are you sure your upload is not a duplicate?" message.
Currently we're just doing a `WHERE BIT_COUNT(images.phash ^ inputHash) < 12` in MySQL over 400k rows, which still works reasonably well (~200ms) given that it can't use an index for the XOR/BIT_COUNT operation. To my knowledge there's no way to speed up this query in MySQL, so if we continue to grow we probably have to write a small daemon that is able to search hashes more efficiently.
- jo_ 12y agoWhat is your data type for inputHash and images.phash? ByteArray? Character array? Blob?
- phoboslab 12y agoJust a 64 bit integer (BIGINT).
- strebler 12y agoHmmm...400K rows seems a bit small for this, but you might be able to build a (FLANN) based vocabulary tree that turns phash into a more stable signature (that makes use of database indexes). Then your SQL query would be more like: "where (images.phashsig=inputsig)". Phash might need a tweak, to output 64 floats instead of bits. But it would be more robust to the "random bits that flip" problem. It's similar to what an engine like TinEye does, but instead of using a bunch of SIFT/SURF/etc features to do numerous visual word lookups, you'd just put in (a modified?) phash and get one word out.