3 ms·
Vinny Glennon, One of the founders here. Thanks very much for the up votes. The Chrome extension does not work in private browsing. I have a set of porn sites(1
by vinnyglennon 14y ago
Vinny Glennon, One of the founders here. Thanks very much for the up votes. The Chrome extension does not work in private browsing. I have a set of porn sites(1.7 million stored in redis) that I check if incoming links are a member of. You can selectively block sites ( https://www.seenbefore.com/blacklist_items https://www.seenbefore.com/blacklist_items).
- andrewthornton 14y agoWhere did you get your list from, and can you share it?
- gingerjoos 14y agoDo you store the entire set of 1.7 million entries in redis? Or is redis an index to data stored elsewhere, in a relational DB perhaps? I was under the impression that redis wouldn't be all that useful to store a lot of data. Would be great if something as quick as redis could work with large data sets.
- vinnyglennon 14y agoStoring a list of 1.7 million strings for us takes 70mb stored in memory. Testing for membership is an O(1) op. Very happy with it. We use mongo as a dumb data store as well as a bunch of other infrastructure tools, like http://circleci.com http://circleci.com we could have only dreamt of years ago.
- StavrosK 14y agoWhy not use a bloom filter?
- genwin 14y agoMaybe because of this fact (according to Wikipedia)?: "The more elements that are added to the set, the larger the probability of false positives."
- StavrosK 14y agoThat depends on its size, though. You can make it larger and get fewer false positives.
- vidarh 14y agoThe main benefit of Bloom filters is that they can be made small. Given that his database takes only 70MB or so and he's not trying to ship this to devices that might have much in terms of space limitations, there would appear to be little point.
- StavrosK 14y agoEh, true, I guess redis is sufficiently awesome.
- genwin 14y ago> Testing for membership is an O(1) op Curious how. O(1) an array index lookup, not a string lookup, I thought.
- milkshakes 14y agohttp://redis.io/commands/sismember http://redis.io/commands/sismember
- genwin 14y ago