4 ms·
Skimmed through it, but it seems that there is still an improvement with the bloom filter with a factor of 2~3. Is that correct? (estimated 6(?)ms once correcte
by jaekwon 14y ago
Skimmed through it, but it seems that there is still an improvement with the bloom filter with a factor of 2~3. Is that correct? (estimated 6(?)ms once corrected with indices, vs 2ms with the bloom)
- wiredfool 14y agoThis sample set is way too small to be arguing about a millisecond here or there. The variation from what's cached, or if something else happens on the machine, or whatever is going to affect the benchmarks. For a 'real' benchmark, I'd like to see a few gigs of data, or something that's not going to fit in cache. The real gain from the bloom filter is going to be in a 4-10x reduction in the working set for the same amount of data. But when the working set is trivial, then the fluctuating and fixed overheads are going to dominate. I'd also like to see how good the bloom filter's indexing is at scale, or if it winds up being some sort of table scan.
- tmoertel 14y agoFor a "real" benchmark, I'd like to see the query repeated until the response-time distribution stabilizes. Then take the response time at the 90th, 95th, and 99th percentiles, and also the mean. Actually, I'd like to see the CDF plot for the whole distribution.
- tmoertel 14y agoMy hunch is that, in the original environment, PostgreSQL with the right indexes would be as fast as the bloom filter. I emailed the OP about adding an index on movie_person(person_id), and he (besides being very cool and gracious about everything) said that he was going to add it and re-run his timings. I suspect will see new results soon.