4 ms·
Ask HN: Best way to store integer array/list per key
What is the best data-store or technique if we were to store millions of integer lists identified by a unique key. The following functions are intended on such lists:
1. Read for a given key
2. Add items to the end of the list against a key
3. Read portions of this list, from-to index
4. Rarely, remove items from the list
The data size is estimated to be roughly 100-million keys, with average of 2-3 thousand integer-list per key.
The key itself is an integer.
- deleted 13y ago[deleted]
- sangupta 13y agoThe lists are expected to contain around 2-3 thousand integers. The maximum may lie in the range of 10-11 thousand. As the lists are expected to mostly grow, the numbers would increase. But, the older elements can be purged to another storage mechanism as they might not be very heavily accessed.
- xd 13y agoSorry I meant to say what size of integer you require, 8,16,32,64bit?
- sangupta 13y agoSorry - my bad. 32-bit for now. I don't anticipate them going to 64-bit so soon.
- xd 13y agoJust working on something, will be right with you.
- sangupta 13y agoNo problem - I will wait for your response. Thanks in advance.
- xd 13y agohttps://gist.github.com/AndrewRose/8805629 https://gist.github.com/AndrewRose/8805629 Considering you are storing only millions of rows I figured a mysql implementation using blobs should do the trick .. but it all depends on the load you expect .. worth playing around with the idea I think.
- sangupta 13y agoThanks for the code and helping with this. I have posted my queries to the gist itself.