4 ms·
Our solution maintains the data in a simpler scheme because keeping the data in a maintainable format is important aspect when the data is ever changing - every
by bilalhusain 11y ago
Our solution maintains the data in a simpler scheme because keeping the data in a maintainable format is important aspect when the data is ever changing - every couple of days millions of people update their preferences and we need to be able to apply these changes. When choosing a single data structure for all the numbers, we need to optimize for space or for faster inserts/updates. Our solution is a trade-off between these options.
- deleted 11y ago[deleted]
- Veedrac 11y agoA hash table is more than compact enough. It should work fine even at high load factors (80%/90%) given a decent hash scheme (eg. Robin Hood hashing) which means you need only 4GB memory. It's random access and on average should have one cache miss per lookup. Updating is rather trivial.
- bilalhusain 11y agoSounds promising. Little bit concerned on the computational cost of hashing function. Gotta explore this option!
- Veedrac 11y agoI wouldn't be worried about hashing, either, given you effectively want to hash a 34 bit integer to a ⌈log 4e8⌉ = 29 bit integer. An easy way is to take a 32 to 32 bit hash (eg. http://stackoverflow.com/a/12996028/1763356 http://stackoverflow.com/a/12996028/1763356), calculate hash(top 32 bits) ^ hash(bottom 32 bits) and take the bottom 29 bits of that. I wouldn't be surprised if you can do this in less time than an integer division.