5 ms·

Do you have a good recommendation for learning about what use cases DynamoDB makes sense for? I've used MongoDB and various SQL databases extensively, and when

Do you have a good recommendation for learning about what use cases DynamoDB makes sense for? I've used MongoDB and various SQL databases extensively, and when I tried DynamoDB it felt like a solution in search of a problem compared to my usual DBs of choice.

jugg1es 5y ago
I built a pseudo-graph database for relational data for my company on DynamoDB that has been really amazing in both performance and utility. The data model is defined entirely by configuration, so adding use cases is just modifying the configuration. I've only had to touch the core system once or twice in 3 years and we've added about 10 use cases to it. The only thing it can't do well are things that a RDBMS has no trouble doing - namely getting all of one entity for analytical purposes, but we solved that problem using redshift and dynamoDB streams.
- j6m8 5y ago
  Hey cool! I'm super curious to hear more about this. I _also_ wrote a pseudo-graph-database on DynamoDB (https://github.com/aplbrain/grand https://github.com/aplbrain/grand) :) It pretends it's a networkx.Graph, generally, but we also have a Cypher implementation on top of it. Would love to chat more about this sometime if you were interested!
  - jugg1es 5y ago
    Why do you call yours a pseudo-graph database? I use that term because my system only allows 1 true level of depth before you have to query again. You also have to predefine the data model, but that's so that it can automatically choose the right entity type (to format the PK/SK correctly) and so that it supports automatic Global Secondary Index selection as it dynamically generates the query. It also means that each entity can only have up to 5 extra indexable key pairs. Well, I was wrong. I just looked it up and you can now have 20 GSI. It used to be 5 haha. Anyway, it would be cool to talk about your project. I'll check it out this week.
harrisonjackson 5y ago
https://aws.amazon.com/blogs/database/how-to-determine-if-amazon-dynamodb-is-appropriate-for-your-needs-and-then-plan-your-migration/ https://aws.amazon.com/blogs/database/how-to-determine-if-am... It is best in extreme cases. Some types of high scale use and super super cheap serverless/bootstrapping startup smaller scale. Low-scale - it is easy to create a table and just start writing to it. The pay per request pricing is simple and plays well with IAM, other AWS services and the serverless framework. High-scale - scales easily up and down, global replication is easy, backups and high availability are easy - it is all self-managed so a lot of this is baked in or very easy to toggle on. Depending on your ops resources, the easy tooling may be worth the extra costs or maybe your data lends it self particularly well to how dynamodb works. Some of our use cases .... - user profiles to supplement cognito user attributes - chat/chat groups/chat group memberships/chat websocket watchers/chat messages (this one was not good. I would do it again with just messages and the websocket watchers) - tenant data with lots of custom attributes - forms and form submissions - event webhooks from 3rd party services - catch/store/process using dynamodb streams - step function states and logs - it has some nice versioning write patterns - web scraping data from ebay - web scraping data from golf tee time sites When you need search or flexible querying you can put elasticsearch in front of it and use dynamodb streams to push items into the index. If you need analytics you can use another layer like redshift.
victor106 5y ago
https://www.dynamodbbook.com/ https://www.dynamodbbook.com/ This book is quite good. Used it to get a deeper understanding of Dynamodb.

jugg1es 5y ago

I built a pseudo-graph database for relational data for my company on DynamoDB that has been really amazing in both performance and utility. The data model is defined entirely by configuration, so adding use cases is just modifying the configuration. I've only had to touch the core system once or twice in 3 years and we've added about 10 use cases to it. The only thing it can't do well are things that a RDBMS has no trouble doing - namely getting all of one entity for analytical purposes, but we solved that problem using redshift and dynamoDB streams.

j6m8 5y ago
Hey cool! I'm super curious to hear more about this. I _also_ wrote a pseudo-graph-database on DynamoDB (https://github.com/aplbrain/grand https://github.com/aplbrain/grand) :) It pretends it's a networkx.Graph, generally, but we also have a Cypher implementation on top of it. Would love to chat more about this sometime if you were interested!
- jugg1es 5y ago
  Why do you call yours a pseudo-graph database? I use that term because my system only allows 1 true level of depth before you have to query again. You also have to predefine the data model, but that's so that it can automatically choose the right entity type (to format the PK/SK correctly) and so that it supports automatic Global Secondary Index selection as it dynamically generates the query. It also means that each entity can only have up to 5 extra indexable key pairs. Well, I was wrong. I just looked it up and you can now have 20 GSI. It used to be 5 haha. Anyway, it would be cool to talk about your project. I'll check it out this week.

harrisonjackson 5y ago

https://aws.amazon.com/blogs/database/how-to-determine-if-amazon-dynamodb-is-appropriate-for-your-needs-and-then-plan-your-migration/ https://aws.amazon.com/blogs/database/how-to-determine-if-am... It is best in extreme cases. Some types of high scale use and super super cheap serverless/bootstrapping startup smaller scale. Low-scale - it is easy to create a table and just start writing to it. The pay per request pricing is simple and plays well with IAM, other AWS services and the serverless framework. High-scale - scales easily up and down, global replication is easy, backups and high availability are easy - it is all self-managed so a lot of this is baked in or very easy to toggle on. Depending on your ops resources, the easy tooling may be worth the extra costs or maybe your data lends it self particularly well to how dynamodb works. Some of our use cases .... - user profiles to supplement cognito user attributes - chat/chat groups/chat group memberships/chat websocket watchers/chat messages (this one was not good. I would do it again with just messages and the websocket watchers) - tenant data with lots of custom attributes - forms and form submissions - event webhooks from 3rd party services - catch/store/process using dynamodb streams - step function states and logs - it has some nice versioning write patterns - web scraping data from ebay - web scraping data from golf tee time sites When you need search or flexible querying you can put elasticsearch in front of it and use dynamodb streams to push items into the index. If you need analytics you can use another layer like redshift.

victor106 5y ago

https://www.dynamodbbook.com/ https://www.dynamodbbook.com/ This book is quite good. Used it to get a deeper understanding of Dynamodb.