3 ms·

Not sure I understand this. Why did they choose Dynamo? And what does serverless has to do with this? This seems like a simple case of people choosing the wron

Not sure I understand this. Why did they choose Dynamo? And what does serverless has to do with this? This seems like a simple case of people choosing the wrong tool for the job, without really understanding its limitations or capabilities. > say, switching between Oracle, MS SQL and Postgress. That's because those are all relational DBs that use SQL. If they chose some other NoSQL DB they would likely have encountered similar "peculiar" strengths and weaknesses. And I'm still not sure what this has to do with serverless - they could've have easily implemented a serverless architecture with MySQL or Postgress on RDS instead of Dynamo. But they chose not to.

davidjnelson 8y ago
Aws might have fixed this by now with severless aurora, but as of earlier this year the issue was connection pools didn’t work on lambda functions due to their ephemeral nature not being able to keep the tcp sockets open for more than a few minutes before the functions froze or were terminated.
- pavelrub 8y ago
  Connection pools always worked quite well with Lambdas, as long as the connection pool was initiated in the global scope of the running code, and not within the handler function. See [1] and [2] [1] - https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-code.html https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-... [2] - https://blog.spotinst.com/2017/11/19/best-practices-serverless-connection-pooling-database/ https://blog.spotinst.com/2017/11/19/best-practices-serverle...
- dlhavema 8y ago
  We use pools in lambdas to Amazon Aurora and it works out fine. Only limit we hit was when the db server was configured way too small.
  - zedpm 8y ago
    Are you running your Lambdas inside the VPC (and incurring huge cold-start latency), or are you running your database exposed to the public internet? I've avoided using Lambda for anything that requires relational database connectivity because those appear to be the two options, and neither is viable for me.
- scarface74 8y ago
  The Data API was announced less than a month ago. https://aws.amazon.com/about-aws/whats-new/2018/11/aurora-serverless-data-api-beta/ https://aws.amazon.com/about-aws/whats-new/2018/11/aurora-se...
viraptor 8y ago
> Why did they choose Dynamo? And what does serverless has to do with this? Possibly because lambda is integrated with dynamodb. You can hook a function to an event like a row being inserted or modified: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... That lets you skip a lot of explicit queue management code and the change+notify is atomic by default.
- scarface74 8y ago
  You can use lambda within an Aurora (AWS version of MySQL) trigger also. The advantage of DynamoDB when using lambda is that it is API based instead of based on database connections and pooling and it doesn’t run inside of a VPC - meaning a network interface doesn’t need to be created inside your VPC every time a new lambda process comes online. But now Aurora Serverless has the Data API that doesn’t use traditional database connections so there is even less of a need for DynamoDB for lambda performance reasons.
- pavelrub 8y ago
  Sure, but that's simply a single point in favour of Dynamo in what should have been a comprehensive analysis of strengths and weaknesses with respect to the required use case. If they chose DynamoDB simply due to this strength (ignoring for the moment that the same thing can be achieved with native functions in Aurora), while ignoring the other things that ultimately turned out to be more important, then this again points to a flawed analysis. It's hard to see how serverless is to blame for this.
matwood 8y ago
> That's because those are all relational DBs that use SQL I have dealt with all of those, including MySQL. Switching is never easy. Every technology choice, even ones that should be similar, like an RDBMS, have strengths and weaknesses.

davidjnelson 8y ago

Aws might have fixed this by now with severless aurora, but as of earlier this year the issue was connection pools didn’t work on lambda functions due to their ephemeral nature not being able to keep the tcp sockets open for more than a few minutes before the functions froze or were terminated.

pavelrub 8y ago
Connection pools always worked quite well with Lambdas, as long as the connection pool was initiated in the global scope of the running code, and not within the handler function. See [1] and [2] [1] - https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-code.html https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-... [2] - https://blog.spotinst.com/2017/11/19/best-practices-serverless-connection-pooling-database/ https://blog.spotinst.com/2017/11/19/best-practices-serverle...
dlhavema 8y ago
We use pools in lambdas to Amazon Aurora and it works out fine. Only limit we hit was when the db server was configured way too small.
- zedpm 8y ago
  Are you running your Lambdas inside the VPC (and incurring huge cold-start latency), or are you running your database exposed to the public internet? I've avoided using Lambda for anything that requires relational database connectivity because those appear to be the two options, and neither is viable for me.
scarface74 8y ago
The Data API was announced less than a month ago. https://aws.amazon.com/about-aws/whats-new/2018/11/aurora-serverless-data-api-beta/ https://aws.amazon.com/about-aws/whats-new/2018/11/aurora-se...

viraptor 8y ago

> Why did they choose Dynamo? And what does serverless has to do with this? Possibly because lambda is integrated with dynamodb. You can hook a function to an event like a row being inserted or modified: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... That lets you skip a lot of explicit queue management code and the change+notify is atomic by default.

scarface74 8y ago
You can use lambda within an Aurora (AWS version of MySQL) trigger also. The advantage of DynamoDB when using lambda is that it is API based instead of based on database connections and pooling and it doesn’t run inside of a VPC - meaning a network interface doesn’t need to be created inside your VPC every time a new lambda process comes online. But now Aurora Serverless has the Data API that doesn’t use traditional database connections so there is even less of a need for DynamoDB for lambda performance reasons.
pavelrub 8y ago
Sure, but that's simply a single point in favour of Dynamo in what should have been a comprehensive analysis of strengths and weaknesses with respect to the required use case. If they chose DynamoDB simply due to this strength (ignoring for the moment that the same thing can be achieved with native functions in Aurora), while ignoring the other things that ultimately turned out to be more important, then this again points to a flawed analysis. It's hard to see how serverless is to blame for this.

matwood 8y ago

> That's because those are all relational DBs that use SQL I have dealt with all of those, including MySQL. Switching is never easy. Every technology choice, even ones that should be similar, like an RDBMS, have strengths and weaknesses.