3 ms·
Do I understand correctly, that they spent memory storing largish N hash values per server, so that request hash determines which server to send request to usi
by nopurpose 15d ago
Do I understand correctly, that they spent memory storing largish N hash values per server, so that request hash determines which server to send request to using closest higher value of all server hashes?
That in effect boils down to consistently selecting server S with probability P, where P is function of weight and total number of servers?
Surely there must be better way to select server with a given probability without storing a massive lookup table of hashes? Randevouz hashing of some sorts
- varispeed 15d agoWho cares if you can buy all the RAM available. To hell with small business and working class who now cannot afford it.
- QuaternionsBhop 15d agoPlus it's limited to 65k entries. Perhaps a btree where parent nodes sum the weights of child nodes would work well. Using the input hash scaled by total weight, a binary search lookup would compute the partial sums for comparison on the fly. Adding/removing a node would only update the ~8 parents when the btree order is 4. Eytzinger layout and struct-of-arrays could be used to improve cache locality during lookup. This does mean an add/remove could drastically change the overall mapping, perhaps that's why consistent hashing is used instead.