5 ms·

It’s interesting that lambda still requires a nosql db for good perf. The example they link to from this article uses dynamodb. Not a great choice for a gener

It’s interesting that lambda still requires a nosql db for good perf. The example they link to from this article uses dynamodb. Not a great choice for a general purpose web app, where sql would be a better choice for many use cases. https://github.com/serverless/components/tree/master/templates/8-fullstack-application https://github.com/serverless/components/tree/master/templat... It’s cool that aws has the data api for lambda to connect to aurora serverless. But then you’re stuck with MySQL instead of postgres, and also stuck with ~100ms db access times. https://www.jeremydaly.com/aurora-serverless-data-api-a-first-look/ https://www.jeremydaly.com/aurora-serverless-data-api-a-firs...

jwilliams 7y ago
What makes DynamoDB a bad choice for a web app? Seems to be where it excels.
- davidjnelson 7y ago
  It’s difficult to model relationships. You have to design a hashing strategy manually to avoid hotspots. Most web frameworks are based on sql. The api is extremely verbose and there isn’t the same level of maturity for abstractions which simplify it. Indexes are much more complicated https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/SQLtoNoSQL.Indexes.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... In general, the tooling is less mature. You can run a fake dynamodb locally, but not the exact same engine. There are certainly use cases where nosql is very helpful - heavy writes, as a denormalized cache, simpler non relational data models etc http://highscalability.com/blog/2010/12/6/what-the-heck-are-you-actually-using-nosql-for.html http://highscalability.com/blog/2010/12/6/what-the-heck-are-...
  - jwilliams 7y ago
    I feel like we might be diverging on what "a general purpose web app" represents. I've never really seen those issues unless there is a more extreme circumstance. In my read of "web app", in most cases you want to push/data in and out of the database. And modelling 1:1 and 1:n relationships is pretty natural in my experience. Deeply nested relationships are a pain, but they're usually a pain in SQL too. Document-orientation helps here a bit, but with some drawbacks. Where I think it does fall over completely is analytics, and OLAP-like work. I guess hotspots can be an issue. However, that implies considerable related transaction volume, which I'd argue DynamoDB can excel at. If you're using a framework like Rails - then sure, DynamoDB is out. But so is Lambda. The pattern of SPA (React/etc), Lambda, and DynamoDB is what's emerging instead of those frameworks. Definitely agree it can't cover many cases, but covers a lot of ground for me
    - davidjnelson 7y ago
      Right, I'm talking about many to many relationships. Those are a pain to deal with in non relational databases. It's possible, just not the simplest tool for the task. This is a great strength of sql, especially when coupled with a good orm used wisely. Also, don't underestimate the power of frameworks like rails, django et al for productivity. Most apps I write for my own business, or see in production at companies I work for use a mix of sql and nosql depending on what they are doing. Mongo seems quite popular for heavy write loads, while still supporting some indexing and querying. It's great for fan out also.
  - stunt 7y ago
    You can't just use the same data models and normalization strategies of a relational databases in the DynamoDB. Plus there are many differences with other Nosql databases too that you must be aware of. (e.g. scaling behaviour) I often see teams and devs failing to use DynamoDB effectively only because they don't know/use best practices. It just needs more time. What I don't like about DynamoDB is the vendor lock-in. Otherwise it's a great product and works really well for many use cases.
    - davidjnelson 7y ago
      I agree DynamoDB is a great product that works well for many use cases. I use it for certain things and it works very well. Relational db's are a better fit for most web apps though.
- valgaze 7y ago
  +1 on querying model challenges: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... DynamoDB, however, has: 1) 1st-class integration with aws-sdk 2) A cool feature "streams" which can trigger other services when your data changes: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... For what it's worth, this tool kit (components) could be useful useful for the toy case of someone setting up something like DocumentDB. DocumentDB (not DynamoDB) is the mongo-emulating nosql document-based DB option-- it has automated vertical/horizontal scaling (up to 64tb https://docs.aws.amazon.com/documentdb/latest/developerguide/limits.html https://docs.aws.amazon.com/documentdb/latest/developerguide...) A DocumentDB cluster has no public endpoints so any resources that interact with it (ie a lambda function) need to be in the same. You can't modify a lambda's VPC unless that lambda function has a special permission set from an IAM blah blah Automating that bit would be cool
stunt 7y ago
AWS has also introduced Query API for AWS RDS, and Data API for Aurora Serverless. Both are designed to send query requests over HTTPS (same as what you would do with DynamoDB). And that means the Lambda function doesn't have to deal with a database connection, thus there shouldn't be performance issues when using Lambda+RDS "anymore".
- davidjnelson 7y ago
  > It’s cool that aws has the data api for lambda to connect to aurora serverless. But then you’re stuck with MySQL instead of postgres, and also stuck with ~100ms db access times. 100ms is pretty damn slow still unfortunately. Also, I much prefer postgres to mysql.

jwilliams 7y ago

What makes DynamoDB a bad choice for a web app? Seems to be where it excels.

davidjnelson 7y ago
It’s difficult to model relationships. You have to design a hashing strategy manually to avoid hotspots. Most web frameworks are based on sql. The api is extremely verbose and there isn’t the same level of maturity for abstractions which simplify it. Indexes are much more complicated https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/SQLtoNoSQL.Indexes.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... In general, the tooling is less mature. You can run a fake dynamodb locally, but not the exact same engine. There are certainly use cases where nosql is very helpful - heavy writes, as a denormalized cache, simpler non relational data models etc http://highscalability.com/blog/2010/12/6/what-the-heck-are-you-actually-using-nosql-for.html http://highscalability.com/blog/2010/12/6/what-the-heck-are-...
- jwilliams 7y ago
  I feel like we might be diverging on what "a general purpose web app" represents. I've never really seen those issues unless there is a more extreme circumstance. In my read of "web app", in most cases you want to push/data in and out of the database. And modelling 1:1 and 1:n relationships is pretty natural in my experience. Deeply nested relationships are a pain, but they're usually a pain in SQL too. Document-orientation helps here a bit, but with some drawbacks. Where I think it does fall over completely is analytics, and OLAP-like work. I guess hotspots can be an issue. However, that implies considerable related transaction volume, which I'd argue DynamoDB can excel at. If you're using a framework like Rails - then sure, DynamoDB is out. But so is Lambda. The pattern of SPA (React/etc), Lambda, and DynamoDB is what's emerging instead of those frameworks. Definitely agree it can't cover many cases, but covers a lot of ground for me
  - davidjnelson 7y ago
    Right, I'm talking about many to many relationships. Those are a pain to deal with in non relational databases. It's possible, just not the simplest tool for the task. This is a great strength of sql, especially when coupled with a good orm used wisely. Also, don't underestimate the power of frameworks like rails, django et al for productivity. Most apps I write for my own business, or see in production at companies I work for use a mix of sql and nosql depending on what they are doing. Mongo seems quite popular for heavy write loads, while still supporting some indexing and querying. It's great for fan out also.
- stunt 7y ago
  You can't just use the same data models and normalization strategies of a relational databases in the DynamoDB. Plus there are many differences with other Nosql databases too that you must be aware of. (e.g. scaling behaviour) I often see teams and devs failing to use DynamoDB effectively only because they don't know/use best practices. It just needs more time. What I don't like about DynamoDB is the vendor lock-in. Otherwise it's a great product and works really well for many use cases.
  - davidjnelson 7y ago
    I agree DynamoDB is a great product that works well for many use cases. I use it for certain things and it works very well. Relational db's are a better fit for most web apps though.
valgaze 7y ago
+1 on querying model challenges: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... DynamoDB, however, has: 1) 1st-class integration with aws-sdk 2) A cool feature "streams" which can trigger other services when your data changes: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... For what it's worth, this tool kit (components) could be useful useful for the toy case of someone setting up something like DocumentDB. DocumentDB (not DynamoDB) is the mongo-emulating nosql document-based DB option-- it has automated vertical/horizontal scaling (up to 64tb https://docs.aws.amazon.com/documentdb/latest/developerguide/limits.html https://docs.aws.amazon.com/documentdb/latest/developerguide...) A DocumentDB cluster has no public endpoints so any resources that interact with it (ie a lambda function) need to be in the same. You can't modify a lambda's VPC unless that lambda function has a special permission set from an IAM blah blah Automating that bit would be cool

stunt 7y ago

AWS has also introduced Query API for AWS RDS, and Data API for Aurora Serverless. Both are designed to send query requests over HTTPS (same as what you would do with DynamoDB). And that means the Lambda function doesn't have to deal with a database connection, thus there shouldn't be performance issues when using Lambda+RDS "anymore".

davidjnelson 7y ago
> It’s cool that aws has the data api for lambda to connect to aurora serverless. But then you’re stuck with MySQL instead of postgres, and also stuck with ~100ms db access times. 100ms is pretty damn slow still unfortunately. Also, I much prefer postgres to mysql.