4 ms·

Amazon DynamoDB On Demand

105 points by abd12 8y ago

jedberg 8y ago
Finally! One of the worst things as a startup was trying to capacity plan my DynamoDB when I had no idea what the useage pattern would be. Now I can not worry about it while I get data for future capacity planning.
- jopsen 8y ago
  Why I've been attracted to azure table storage and google firestore. Capacity planning is hard.
- Dunedan 8y ago
  There was already had DynamoDB Auto Scaling to achieve that. Main benefit of DynamoDB On-Demand from my point of view is: - easier setup (a single setting instead of configuring auto scaling rules and policies for the table and all its indicies) - possibly cheaper (depends on the not yet published details of the per-request pricing)
  - jedberg 8y ago
    Auto scaling lagged though. And you still had to guess at the base level. They mentioned in the keynote that this is a big improvement over auto scaling.
    - supernovagu 8y ago
      I am puzzled by this statement: Tables using on-demand mode support all DynamoDB features (such as encryption at rest, point-in-time recovery, global tables, and so on) with the exception of auto scaling, which is not applicable with this mode. Source: https://aws.amazon.com/blogs/aws/amazon-dynamodb-on-demand-no-capacity-planning-and-pay-per-request-pricing/ https://aws.amazon.com/blogs/aws/amazon-dynamodb-on-demand-n... Does it mean auto scaling is no longer needed, or it won't happen in this mode (some hidden fixed capacity?)
      - Dunedan 8y ago
        With DynamoDB On-Demand AWS handles scaling transparently for the customer. So they'll definitely scale, but as a customer you shouldn't notice it. I'm quite curious how that works out in practice with rapidly changing request rates.
        supernovagu 8y ago
        Thanks!
- tracker1 8y ago
  I think it is about time... Azure Storage Tables have been that way pretty much from the start, comparatively.
javery 8y ago
It looks like you will be charged more for on-demand but I don't see a list of the rates yet, anyone find them?
- jopsen 8y ago
  If you only pay for what you use then that seems reasonable.
- giaour 8y ago
  The pricing page is available at https://aws.amazon.com/dynamodb/pricing/on-demand/ https://aws.amazon.com/dynamodb/pricing/on-demand/ , but it looks like it's not yet updated with the per-request rates. The blog post mentions rates in us-east-1: "in the US East (N. Virginia) region, you are charged $1.25 per million write requests units and $0.25 per million read request units, plus the usual data storage costs."
  - phamilton 8y ago
    My math: Provisioned: $0.00065 per WCU. One WCU is 1 write per second for an hour = 3600 writes. So 277 WCU is 1 million writes in an hour. Cost is $0.18 per million write requests. (If you reserve this for a year it's $0.08 per million write requests.) We generally run our tables with autoscaling set to 70% of capacity, so at $0.18 per million write requests our effective cost is $0.25 per million write requests. A good deal cheaper than on demand. We have a few tables where we run closer to 30% of target capacity because of hot shards. There our effective cost is $0.60 per million write requests, which is still cheaper than on demand (assuming on demand doesn't have the hot shard problem). I'm having trouble understanding when this would be beneficial.
    - kenhwang 8y ago
      I'm considering using this for our staging tables since 90% of the time, we'd want to scale to 0 instead of 1. Doesn't do much for cost savings, but it does make it easier to manage.
    - ksmail99 8y ago
dschuetz 8y ago
Another Amazon Ad?
- rstupek 8y ago
  AWS Re:invent is going on. People in tech are interested in these things. They're not ads per se
deleted 8y ago
[deleted]
maximilianroos 8y ago
Am I correct in thinking Google Cloud Datastore has this already? What are the biggest remaining differences between DynamoDB vs Datastore?
- manigandham 8y ago
  DynamoDB has a simpler key/value data model like BigTable. Cloud Datastore is a document-store with a richer data model (built on top of BigTable, or at least it was until the latest version with Firestore).
jakozaur 8y ago
Doesn't seems to be attractive price-wise. Quick math (us-east-1 North Virginia, reads, strong consistency): 1. Provisioned: $0.00013 per RCU 1 RCU provides 3,600 reads per hour 0.00013 * 1000000 / 3600 ~= $0.036 per mln reads https://aws.amazon.com/dynamodb/pricing/provisioned/ https://aws.amazon.com/dynamodb/pricing/provisioned/ 2. On-demand: $0.25 per million read request units https://aws.amazon.com/dynamodb/pricing/on-demand/ https://aws.amazon.com/dynamodb/pricing/on-demand/ So AWS on-demand is more expensive if your read utilisation is higher than 14.4%. Not that exciting for any serious use-case. Plus you can reserve DynamoDB capacity for a year and save ~56%: (30 + 0.0025 * 24 * 365) / (100* 0.00013 * 365 * 24 )
- bboreham 8y ago
  That calculation only works if you can spread your workload over ten minutes in the hour. If you want to get it done in ten seconds, then you don’t have any more to do for the next hour, then on-demand is much better.
stuart_marshall 8y ago
Love that this feature finally released. I was a dev manager on DynamoDB years ago when we first started working on this. I left three years ago and had almost given up hope that this feature would see the light of day. The great thing about it is that it helps with spiky and sporadic workloads. In my current work I have a bunch of unit tests that hit a set of dynamodb tables. I have those tables provisioned at a 5 RCU and 5 WCU each. Multiply this by ten tables and I'm paying $25 per month. In the grand scheme of costs, this isn't much, but it's annoying and adds up. (One can argue that unit tests should be hitting a mock or a dynamodb local. That's true, but you also want some tests that actually hit the service.) On demand can be great for sporadic tests, for prototypes, for small personal projects, and for just messing around. There are also spiky non-customer-facing production scenarios that can benefit from On demand.
wanghq 8y ago
It's nice to see that DynamoDB becomes truly serverless (although it lived under the serverless umbrella for a while) with this enhancement. But I feel the on-demand mode should be best accompanied with the provisioned mode, which is the hybrid mode. So instead of using the laggy autoscaling to handle the burst traffic, using the on-demand capacity to serve that. Disclosure: I work for Alibaba Cloud and use Alibaba Cloud Table Store which provides the hybrid mode ( https://www.alibabacloud.com/help/doc-detail/27291.htm https://www.alibabacloud.com/help/doc-detail/27291.htm )

105 points by abd12 8y ago

Finally! One of the worst things as a startup was trying to capacity plan my DynamoDB when I had no idea what the useage pattern would be. Now I can not worry about it while I get data for future capacity planning.

jopsen 8y ago
Why I've been attracted to azure table storage and google firestore. Capacity planning is hard.
Dunedan 8y ago
There was already had DynamoDB Auto Scaling to achieve that. Main benefit of DynamoDB On-Demand from my point of view is: - easier setup (a single setting instead of configuring auto scaling rules and policies for the table and all its indicies) - possibly cheaper (depends on the not yet published details of the per-request pricing)
- jedberg 8y ago
  Auto scaling lagged though. And you still had to guess at the base level. They mentioned in the keynote that this is a big improvement over auto scaling.
  - supernovagu 8y ago
    I am puzzled by this statement: Tables using on-demand mode support all DynamoDB features (such as encryption at rest, point-in-time recovery, global tables, and so on) with the exception of auto scaling, which is not applicable with this mode. Source: https://aws.amazon.com/blogs/aws/amazon-dynamodb-on-demand-no-capacity-planning-and-pay-per-request-pricing/ https://aws.amazon.com/blogs/aws/amazon-dynamodb-on-demand-n... Does it mean auto scaling is no longer needed, or it won't happen in this mode (some hidden fixed capacity?)
    - Dunedan 8y ago
      With DynamoDB On-Demand AWS handles scaling transparently for the customer. So they'll definitely scale, but as a customer you shouldn't notice it. I'm quite curious how that works out in practice with rapidly changing request rates.
      - supernovagu 8y ago
        Thanks!
tracker1 8y ago
I think it is about time... Azure Storage Tables have been that way pretty much from the start, comparatively.

javery 8y ago

It looks like you will be charged more for on-demand but I don't see a list of the rates yet, anyone find them?

jopsen 8y ago
If you only pay for what you use then that seems reasonable.
giaour 8y ago
The pricing page is available at https://aws.amazon.com/dynamodb/pricing/on-demand/ https://aws.amazon.com/dynamodb/pricing/on-demand/ , but it looks like it's not yet updated with the per-request rates. The blog post mentions rates in us-east-1: "in the US East (N. Virginia) region, you are charged $1.25 per million write requests units and $0.25 per million read request units, plus the usual data storage costs."
- phamilton 8y ago
  My math: Provisioned: $0.00065 per WCU. One WCU is 1 write per second for an hour = 3600 writes. So 277 WCU is 1 million writes in an hour. Cost is $0.18 per million write requests. (If you reserve this for a year it's $0.08 per million write requests.) We generally run our tables with autoscaling set to 70% of capacity, so at $0.18 per million write requests our effective cost is $0.25 per million write requests. A good deal cheaper than on demand. We have a few tables where we run closer to 30% of target capacity because of hot shards. There our effective cost is $0.60 per million write requests, which is still cheaper than on demand (assuming on demand doesn't have the hot shard problem). I'm having trouble understanding when this would be beneficial.
  - kenhwang 8y ago
    I'm considering using this for our staging tables since 90% of the time, we'd want to scale to 0 instead of 1. Doesn't do much for cost savings, but it does make it easier to manage.
  - ksmail99 8y ago

dschuetz 8y ago

Another Amazon Ad?

rstupek 8y ago
AWS Re:invent is going on. People in tech are interested in these things. They're not ads per se

deleted 8y ago

[deleted]

maximilianroos 8y ago

Am I correct in thinking Google Cloud Datastore has this already? What are the biggest remaining differences between DynamoDB vs Datastore?

manigandham 8y ago
DynamoDB has a simpler key/value data model like BigTable. Cloud Datastore is a document-store with a richer data model (built on top of BigTable, or at least it was until the latest version with Firestore).

jakozaur 8y ago

Doesn't seems to be attractive price-wise. Quick math (us-east-1 North Virginia, reads, strong consistency): 1. Provisioned: $0.00013 per RCU 1 RCU provides 3,600 reads per hour 0.00013 * 1000000 / 3600 ~= $0.036 per mln reads https://aws.amazon.com/dynamodb/pricing/provisioned/ https://aws.amazon.com/dynamodb/pricing/provisioned/ 2. On-demand: $0.25 per million read request units https://aws.amazon.com/dynamodb/pricing/on-demand/ https://aws.amazon.com/dynamodb/pricing/on-demand/ So AWS on-demand is more expensive if your read utilisation is higher than 14.4%. Not that exciting for any serious use-case. Plus you can reserve DynamoDB capacity for a year and save ~56%: (30 + 0.0025 * 24 * 365) / (100* 0.00013 * 365 * 24 )

bboreham 8y ago
That calculation only works if you can spread your workload over ten minutes in the hour. If you want to get it done in ten seconds, then you don’t have any more to do for the next hour, then on-demand is much better.

stuart_marshall 8y ago

Love that this feature finally released. I was a dev manager on DynamoDB years ago when we first started working on this. I left three years ago and had almost given up hope that this feature would see the light of day. The great thing about it is that it helps with spiky and sporadic workloads. In my current work I have a bunch of unit tests that hit a set of dynamodb tables. I have those tables provisioned at a 5 RCU and 5 WCU each. Multiply this by ten tables and I'm paying $25 per month. In the grand scheme of costs, this isn't much, but it's annoying and adds up. (One can argue that unit tests should be hitting a mock or a dynamodb local. That's true, but you also want some tests that actually hit the service.) On demand can be great for sporadic tests, for prototypes, for small personal projects, and for just messing around. There are also spiky non-customer-facing production scenarios that can benefit from On demand.

wanghq 8y ago

It's nice to see that DynamoDB becomes truly serverless (although it lived under the serverless umbrella for a while) with this enhancement. But I feel the on-demand mode should be best accompanied with the provisioned mode, which is the hybrid mode. So instead of using the laggy autoscaling to handle the burst traffic, using the on-demand capacity to serve that. Disclosure: I work for Alibaba Cloud and use Alibaba Cloud Table Store which provides the hybrid mode ( https://www.alibabacloud.com/help/doc-detail/27291.htm https://www.alibabacloud.com/help/doc-detail/27291.htm )