5 ms·
I've seen two large usages of DynamoDB at two different companies, and for what it's worth, in both cases we've had similar trouble as the author. In one case w
by brandur 9y ago
I've seen two large usages of DynamoDB at two different companies, and for what it's worth, in both cases we've had similar trouble as the author. In one case we ended up ripping it out and moving to a sharded Postgres scheme, and in the other we've left in place for now because a migration will be such a monumental effort, but it's pretty much universally maligned.
Fundamentally, the problem seems to be that choosing a partitioning key that's appropriate for DynamoDB's operational properties is ... unlikely. In their own docs on choosing a partition key [1] they use "user ID" as an example of one with good uniformity, but in reality if you choose something like that, you're probably about to be in for a world of pain: in many systems big users can be 7+ orders of magnitude bigger than small users, so what initially looked like a respectable partitioning key turns out to be very lopsided.
As mentioned in the article, you can then try to increase throughput, but you won't have enough control over the newly provisioned capacity to really address the problem. You can massively overprovision, but then you're paying for a lot of capacity that's sitting idle, and even then sometimes it's not enough.
Your best bet is probably to choose a partition key that's perfectly uniformly distributed (like a random ID), but at that point you're designing your product around DynamoDB rather than vice versa, and you should probably wonder why you're not looking at alternatives.
---
[1] http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GuidelinesForTables.html#GuidelinesForTables.UniformWorkload http://docs.aws.amazon.com/amazondynamodb/latest/developergu...
- hox 9y agoConpletely agreed; using DynamoDB for primary storage suffers from needing to design data models around the underlying technology. This is true for secondary keys as well, and is even constrained by the odd pricing model that DynamoDB follows. I've been happy using DynamoDB as a large-scale caching layer, but even that only fits very specific use case criteria.
- fragsworth 9y ago> using DynamoDB for primary storage suffers from needing to design data models around the underlying technology. Every database system, at large scale, suffers from this problem.
- grogenaut 9y agoI laugh every time I see this because everyone seems to forget the rough time they had the first time they encountered a relational database and how to map their problem into that space... Sure it's pretty straight forward now but that's the point... You're still doing it... You are still using your domain knowledge of relational to map your problem ilto the underlying tech. You just don't notice it. "Oh companies have people and people have multiple addresses with one marked as primary and addresses have 1-3 address lines and several optional fields based on local and and and and..." That's several years of learning if you had to discover it yourself there, not just on the address domain but also on how to decompose it.
- throwaway91111 9y agoWhy on earth don't they hash the key to improve the key distribution?
- choward 9y agoHow would that help? The same user id will still hash to the same thing.
- Jach 9y agoI think I had the same confusion after reading the GP's last statement. The idea of hashing is that the resulting number will be uniformly distributed across the hash space, as opposed to monotonically increasing bit by bit like an auto-increment or timestamp, but it's not the partition key value itself that needs to be uniformly distributed but your access patterns of the partitions. As the docs note, a device id (and also a user id) could be good or bad to use as a partition key, it depends on the read/write patterns given the key.
- xendo 9y agoWrite-through cache solves many of the issues with uneven access patterns.
- idunno246 9y agothey do, thats why the partition key used to be called the hash key
- imauld 9y agoThey do.From their getting started docs: > DynamoDB uses the partition key value as input to an internal hash function; the output from the hash function determines the partition where the item is stored.
- leggomylibro 9y agoHot partitions do always seem to be an issue, for an 'infinitely scalable' NoSQL database that is a bit on the pricey side. But what's a better option for a distributed, managed database-as-a-service? Rolling your own does mean significant operational burden. I wonder if you could do something like add a random number to each of your keys before hashing. It would increase your storage size by Nx, but it seems like that would spread your load as well.
- vosper 9y agoTFA suggests Google's BigTable
- xendo 9y agoKeys are the main way to access data in Dynamo. They need to be deterministic to provide fast lookup.
- nw-fcx 9y agoI've been able to improve my partitioning by adding a random suffix from a small set to the key for the writes, and then just trying all suffixes for the reads. Because my work load was relatively write-heavy (used at least ~10x more write capacity than read capacity), and because most of the read capacity was consumed by the size of the entries and not the lookups (so I didn't even have a linear increase in the need for read capacity), I think I came out ahead. Ultimately, we've decided that dynamodb was not really suited to our use case anyway, but I'd say that's mostly because our total data set was fairly tiny and we our writes were not evenly distributed over time, which is just at odds with the capacity model.
- foota 9y agoIf you have significantly more buckets than users though shouldn't the variations in access pattern even out?
- solidsnack9000 9y agoIt depends on what the extremes look like. Say things are power law distributed -- you have three users that require 1000000x capacity -- then as you shard more you basically guarantee that some shards are much, much more heavily loaded than others.