4 ms·
well let's say you want to create a minute by minute report on URL's visited in an ecommerce site that has a large amount of usage. Every minute I want to proce
by nstart 10y ago
well let's say you want to create a minute by minute report on URL's visited in an ecommerce site that has a large amount of usage. Every minute I want to process all those URLs and decide if I want to create a flash sale on the site. Since this was an example I just thought of, let's imagine that there are 400k URLs visited every minute.
That would be a decent definition of what I meant by large amounts of data. Sorry for not making that clearer before.
Your friend's example is pretty good. Curious how they work on the data in memory.
- ahoka 10y agoIt depends on how many unique URLs you have. If you have like a few hundreds or even thousand I would just use a simple hash table with counters to see what's a hot sale right now (I guess that's what you want in your example). If you have several millions than maybe I would put it into Redis or something similar where I can utilize ready-to-use HyperLogLog cardinality estimation.