2 ms·

The biggest problem I'm aware of with DynamoDB is the hot key / partition issue[1]. Throughout is distributed evenly across nodes, you can't control how many no

by eloff 6y ago

The biggest problem I'm aware of with DynamoDB is the hot key / partition issue[1]. Throughout is distributed evenly across nodes, you can't control how many nodes you have, so you always have a node that's hot either temporarily or permanently and so you end up having to over provision all your nodes to be able to handle that hot case, which ends up costing far more than alternatives. What's your take on this? This is the chief reason I avoid DynamoDB, which in theory would be a good fit for some of my problems. [1] https://syslog.ravelin.com/you-probably-shouldnt-use-dynamodb-89143c1287ca https://syslog.ravelin.com/you-probably-shouldnt-use-dynamod...

luhn 6y ago
As of a couple years ago, DynamoDB will redistribute throughput between shards based on usage [1], so in theory this should eliminate the hot shard problem. I haven't had a chance to test this in practice, if anybody has hands-on experience I'd love to hear it. You also finally have a way of identifying hot keys with the terribly named CloudWatch Contributor Insights for DynamoDB. [2] For exceptional use cases, you also have the option of On-Demand Capacity to pay for what you use and not worry about capacity at all. [3] [1] https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html#bp-partition-key-partitions-adaptive https://docs.aws.amazon.com/amazondynamodb/latest/developerg... [2] https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/contributorinsights.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... [3] https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadWriteCapacityMode.html#HowItWorks.OnDemand https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
- eloff 6y ago
  That sounds like the problem had been solved and my information is just out of date now. Maybe I should give DynamoDB another look now.
Judson 6y ago
With instant adaptive capacity, I think quite a few hot key issues are mitigated. https://aws.amazon.com/blogs/database/how-amazon-dynamodb-adaptive-capacity-accommodates-uneven-data-access-patterns-or-why-what-you-know-about-dynamodb-might-be-outdated/ https://aws.amazon.com/blogs/database/how-amazon-dynamodb-ad...
abd12 6y ago
luhn responded to this one pretty well :) Basically, most of these issues are gone. As long as you don't have extreme skew in your partition keys, you don't need to worry about throughput limits.

by eloff 6y ago

luhn 6y ago

As of a couple years ago, DynamoDB will redistribute throughput between shards based on usage [1], so in theory this should eliminate the hot shard problem. I haven't had a chance to test this in practice, if anybody has hands-on experience I'd love to hear it. You also finally have a way of identifying hot keys with the terribly named CloudWatch Contributor Insights for DynamoDB. [2] For exceptional use cases, you also have the option of On-Demand Capacity to pay for what you use and not worry about capacity at all. [3] [1] https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html#bp-partition-key-partitions-adaptive https://docs.aws.amazon.com/amazondynamodb/latest/developerg... [2] https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/contributorinsights.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... [3] https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadWriteCapacityMode.html#HowItWorks.OnDemand https://docs.aws.amazon.com/amazondynamodb/latest/developerg...

eloff 6y ago
That sounds like the problem had been solved and my information is just out of date now. Maybe I should give DynamoDB another look now.

Judson 6y ago

With instant adaptive capacity, I think quite a few hot key issues are mitigated. https://aws.amazon.com/blogs/database/how-amazon-dynamodb-adaptive-capacity-accommodates-uneven-data-access-patterns-or-why-what-you-know-about-dynamodb-might-be-outdated/ https://aws.amazon.com/blogs/database/how-amazon-dynamodb-ad...

abd12 6y ago

luhn responded to this one pretty well :) Basically, most of these issues are gone. As long as you don't have extreme skew in your partition keys, you don't need to worry about throughput limits.