5 ms·

Agreed! Dynamodb has some really great features and high performance when used correctly but it also has a steep learning curve especially for anyone coming fr

Agreed! Dynamodb has some really great features and high performance when used correctly but it also has a steep learning curve especially for anyone coming from sql and other relational databases. AWS tries to push the concept of single table design which allows you to store an entire application's data in a single dynamodb table and using a bunch of complicated indexing and overloading columns with multiple types of data to sort of get relational data without any joins. This is really easy to get wrong and you need to have all your access patterns figured out ahead of time so that you can create the indices you'll need when you create the table. You can't create new local indices later... only global secondary indices which duplicate the entire table (and double your storage costs). Dynamodb streams are really cool - great way to setup webhooks for data changes. Dynamodb pay per request is super cheap and fast for a serverless db. Aurora serverless isn't pay per request and has slow cold starts. My recommendations if you are using dynamodb because you want something serverless and NOT because nosql is actually the best storage option: * don't do single table design - just make more requests to different tables and join your data in code * if joining in code is too slow then use a sql database... * abstract all your data access calls so you can eventually migrate to a sql database

tylerchurch 5y ago
Do you have a good recommendation for learning about what use cases DynamoDB makes sense for? I've used MongoDB and various SQL databases extensively, and when I tried DynamoDB it felt like a solution in search of a problem compared to my usual DBs of choice.
- jugg1es 5y ago
  I built a pseudo-graph database for relational data for my company on DynamoDB that has been really amazing in both performance and utility. The data model is defined entirely by configuration, so adding use cases is just modifying the configuration. I've only had to touch the core system once or twice in 3 years and we've added about 10 use cases to it. The only thing it can't do well are things that a RDBMS has no trouble doing - namely getting all of one entity for analytical purposes, but we solved that problem using redshift and dynamoDB streams.
  - j6m8 5y ago
    Hey cool! I'm super curious to hear more about this. I _also_ wrote a pseudo-graph-database on DynamoDB (https://github.com/aplbrain/grand https://github.com/aplbrain/grand) :) It pretends it's a networkx.Graph, generally, but we also have a Cypher implementation on top of it. Would love to chat more about this sometime if you were interested!
    - jugg1es 5y ago
      Why do you call yours a pseudo-graph database? I use that term because my system only allows 1 true level of depth before you have to query again. You also have to predefine the data model, but that's so that it can automatically choose the right entity type (to format the PK/SK correctly) and so that it supports automatic Global Secondary Index selection as it dynamically generates the query. It also means that each entity can only have up to 5 extra indexable key pairs. Well, I was wrong. I just looked it up and you can now have 20 GSI. It used to be 5 haha. Anyway, it would be cool to talk about your project. I'll check it out this week.
- harrisonjackson 5y ago
  https://aws.amazon.com/blogs/database/how-to-determine-if-amazon-dynamodb-is-appropriate-for-your-needs-and-then-plan-your-migration/ https://aws.amazon.com/blogs/database/how-to-determine-if-am... It is best in extreme cases. Some types of high scale use and super super cheap serverless/bootstrapping startup smaller scale. Low-scale - it is easy to create a table and just start writing to it. The pay per request pricing is simple and plays well with IAM, other AWS services and the serverless framework. High-scale - scales easily up and down, global replication is easy, backups and high availability are easy - it is all self-managed so a lot of this is baked in or very easy to toggle on. Depending on your ops resources, the easy tooling may be worth the extra costs or maybe your data lends it self particularly well to how dynamodb works. Some of our use cases .... - user profiles to supplement cognito user attributes - chat/chat groups/chat group memberships/chat websocket watchers/chat messages (this one was not good. I would do it again with just messages and the websocket watchers) - tenant data with lots of custom attributes - forms and form submissions - event webhooks from 3rd party services - catch/store/process using dynamodb streams - step function states and logs - it has some nice versioning write patterns - web scraping data from ebay - web scraping data from golf tee time sites When you need search or flexible querying you can put elasticsearch in front of it and use dynamodb streams to push items into the index. If you need analytics you can use another layer like redshift.
- victor106 5y ago
  https://www.dynamodbbook.com/ https://www.dynamodbbook.com/ This book is quite good. Used it to get a deeper understanding of Dynamodb.
zepolen 5y ago
> Dynamodb pay per request is super cheap and fast for a serverless db. Thanks for the laugh. Dynamodb is stupid expensive under any decent workload.
- harrisonjackson 5y ago
  Ha, no doubt. But so is everything serverless, right? The free tier isn't limited to one year and low-usage is inexpensive.
- jugg1es 5y ago
  I think this depends entirely on the value you're getting from it - as with everything in IT.
  - qaq 5y ago
    yes and no at decent load the only value it provides over Aurora is if you can eclipse performance Aurora scales to or you have extremely spiky workload.

tylerchurch 5y ago

Do you have a good recommendation for learning about what use cases DynamoDB makes sense for? I've used MongoDB and various SQL databases extensively, and when I tried DynamoDB it felt like a solution in search of a problem compared to my usual DBs of choice.

jugg1es 5y ago
I built a pseudo-graph database for relational data for my company on DynamoDB that has been really amazing in both performance and utility. The data model is defined entirely by configuration, so adding use cases is just modifying the configuration. I've only had to touch the core system once or twice in 3 years and we've added about 10 use cases to it. The only thing it can't do well are things that a RDBMS has no trouble doing - namely getting all of one entity for analytical purposes, but we solved that problem using redshift and dynamoDB streams.
- j6m8 5y ago
  Hey cool! I'm super curious to hear more about this. I _also_ wrote a pseudo-graph-database on DynamoDB (https://github.com/aplbrain/grand https://github.com/aplbrain/grand) :) It pretends it's a networkx.Graph, generally, but we also have a Cypher implementation on top of it. Would love to chat more about this sometime if you were interested!
  - jugg1es 5y ago
    Why do you call yours a pseudo-graph database? I use that term because my system only allows 1 true level of depth before you have to query again. You also have to predefine the data model, but that's so that it can automatically choose the right entity type (to format the PK/SK correctly) and so that it supports automatic Global Secondary Index selection as it dynamically generates the query. It also means that each entity can only have up to 5 extra indexable key pairs. Well, I was wrong. I just looked it up and you can now have 20 GSI. It used to be 5 haha. Anyway, it would be cool to talk about your project. I'll check it out this week.
harrisonjackson 5y ago
https://aws.amazon.com/blogs/database/how-to-determine-if-amazon-dynamodb-is-appropriate-for-your-needs-and-then-plan-your-migration/ https://aws.amazon.com/blogs/database/how-to-determine-if-am... It is best in extreme cases. Some types of high scale use and super super cheap serverless/bootstrapping startup smaller scale. Low-scale - it is easy to create a table and just start writing to it. The pay per request pricing is simple and plays well with IAM, other AWS services and the serverless framework. High-scale - scales easily up and down, global replication is easy, backups and high availability are easy - it is all self-managed so a lot of this is baked in or very easy to toggle on. Depending on your ops resources, the easy tooling may be worth the extra costs or maybe your data lends it self particularly well to how dynamodb works. Some of our use cases .... - user profiles to supplement cognito user attributes - chat/chat groups/chat group memberships/chat websocket watchers/chat messages (this one was not good. I would do it again with just messages and the websocket watchers) - tenant data with lots of custom attributes - forms and form submissions - event webhooks from 3rd party services - catch/store/process using dynamodb streams - step function states and logs - it has some nice versioning write patterns - web scraping data from ebay - web scraping data from golf tee time sites When you need search or flexible querying you can put elasticsearch in front of it and use dynamodb streams to push items into the index. If you need analytics you can use another layer like redshift.
victor106 5y ago
https://www.dynamodbbook.com/ https://www.dynamodbbook.com/ This book is quite good. Used it to get a deeper understanding of Dynamodb.

zepolen 5y ago

> Dynamodb pay per request is super cheap and fast for a serverless db. Thanks for the laugh. Dynamodb is stupid expensive under any decent workload.

harrisonjackson 5y ago
Ha, no doubt. But so is everything serverless, right? The free tier isn't limited to one year and low-usage is inexpensive.
jugg1es 5y ago
I think this depends entirely on the value you're getting from it - as with everything in IT.
- qaq 5y ago
  yes and no at decent load the only value it provides over Aurora is if you can eclipse performance Aurora scales to or you have extremely spiky workload.