4 ms·
I think the main concern is that in many systems each key doesn't have an equal amount of work associated with it (this sort of thing is usually referred to as
by drifkin 12y ago
I think the main concern is that in many systems each key doesn't have an equal amount of work associated with it (this sort of thing is usually referred to as a "hotspot").
An example: suppose you have some distributed system storing article metadata and all of the sudden one of your articles becomes very widely shared. The machine that the popular key hashes to gets slammed. Perhaps we'd want to adjust it so that that particular machine is just dedicated to that one article, or some other way to distribute that one article across multiple machines. But we're just using a hash function, so without doing something fancier, we can run into problems when the load suddenly becomes wildly uneven.
- reubenbond 12y agoSolving for read hot-spots is not difficult if you're willing to accept a small read penalty: Your typical Kademlia DHT has k-replicas of each piece of data, so you read near the target node (node closest to the target key) rather than directly from it. This way, nodes at different points in the network read from many different replicas. Of course, this depends on your consistency requirements.