5 ms·
Hmm that’s only 190Hz on average, but we don’t know what kind of search engine it is. For example if he’s doing ML inference for every query, it would make perf
by NavinF 5y ago
Hmm that’s only 190Hz on average, but we don’t know what kind of search engine it is. For example if he’s doing ML inference for every query, it would make perfect sense to get a few cabinets at a data center. I’ve done so for a much smaller project that only needs 4 GPUs and saved a ton of money.
- fxtentacle 5y agoNah, it's text-only requests returning JSON arrays of which newspaper article URLs mention which influencer or brand name keyword. The biggest hardware price point is that you need insane amounts of RAM so that you can mmap the bloom hash for the mapping from word_id to document_ids.
- winrid 5y agoYou could have used a sharded database like Mongo. Just throw up 10 shards, use "source" (influencer or brand name) as shard key?
- fxtentacle 5y agoYes, I could have used Mongo, but it would have been 100x to 1000x slower than an mmap-ed look up table.
- joshuamorton 5y agoBut you don't actually need that level of performance? You've made this system more complex and expensive to achieve a requirement that doesn't matter?
- shoo 5y agoyou seem to have a deeper knowledge of the business & organisational context that dictate the true requirements than someone working there. please share these details so we can all learn!
- joshuamorton 5y agoSure: the network request time of a person making a request over the open internet is going to be an order of magnitude longer than a DB lookup (in the right style, with a reverse-index) on the scale of data this person is describing. So making the lookup 10x faster saves you...1% of the request latency. And at the qps they've described, it's not a throughput issue either. So I'm pretty confident in saying that this is a case of premature optimization. And at some point the increase in parallelization of scans dominates mmap speed, unless you're redundantly sharding your mmaped hash table across multiple machines. And there are cases where network bandwidth is the bottleneck before disk bandwidth, though probably not this case. But yeah basically, the answer is something like "if this is the optimal choice, it probably didnt matter that much".
- fxtentacle 5y agoThis reads to me as if you have never really used mmap in a dedicated C/C++ application. Just to give you a data point, looking up one word_id in the LUT and reading 20 document_ids from it takes on average 0.0000015 ms. So if that alternative database takes on average 0.1ms per index read, then it's starting out roughly 65000x slower. "than a DB lookup (in the right style, with a reverse-index)" Unless, of course, you're managing petabytes of data ;) "at the qps they've described, it's not a throughput issue either" It's mostly a cost thing. If a single request takes 2x the time, that's also a 2x on the hosting bill. "parallelization of scans dominates mmap speed" Yes, eventually that might happen. Roughly when you have 100000 servers. But before that your 10gbit/s node-to-node link will saturate. Oops.
- winrid 5y agoAre you managing petabytes of data though? What kind of servers are you running? What's your max QPS? The fact is with your mmap impl. you probably use ram + virtual memory, and have more ram than needed to compensate for the fact that you don't keep the most used keys in memory, which a DB will do for you. Point is if you have petabytes of data and access patterns only mean you access a subset of it, even Mongo might be cheaper to run.
- winrid 5y agoYeah the question is what level of performance you need I guess... was hoping you could clarify :)
- nostrebored 5y agoWhy ever use mmap instead of sharded inverted indices of word-doc here, a la elasticsearch?