3 ms·
Hot partitions do always seem to be an issue, for an 'infinitely scalable' NoSQL database that is a bit on the pricey side. But what's a better option for a di
by leggomylibro 9y ago
Hot partitions do always seem to be an issue, for an 'infinitely scalable' NoSQL database that is a bit on the pricey side.
But what's a better option for a distributed, managed database-as-a-service? Rolling your own does mean significant operational burden.
I wonder if you could do something like add a random number to each of your keys before hashing. It would increase your storage size by Nx, but it seems like that would spread your load as well.
- vosper 9y agoTFA suggests Google's BigTable
- xendo 9y agoKeys are the main way to access data in Dynamo. They need to be deterministic to provide fast lookup.
- nw-fcx 9y agoI've been able to improve my partitioning by adding a random suffix from a small set to the key for the writes, and then just trying all suffixes for the reads. Because my work load was relatively write-heavy (used at least ~10x more write capacity than read capacity), and because most of the read capacity was consumed by the size of the entries and not the lookups (so I didn't even have a linear increase in the need for read capacity), I think I came out ahead. Ultimately, we've decided that dynamodb was not really suited to our use case anyway, but I'd say that's mostly because our total data set was fairly tiny and we our writes were not evenly distributed over time, which is just at odds with the capacity model.