4 ms·

I'm always surprised at the lack of discussion around database connection reuse with AWS Lambda. It's a pretty big deal that every single API call requires a n

I'm always surprised at the lack of discussion around database connection reuse with AWS Lambda. It's a pretty big deal that every single API call requires a new database connection. The only solutions I've seen so far are to run a separate app to interface with the database, or moving the connection outside of the handler (which still has issues). Am I missing something or is everyone just really happy to use DynamoDB?

Mizza 10y ago
If you design your correctly (at least in Python), you can cache and reuse your open database connection.
- ranman 10y ago
  How exactly have you been doing that? It would be a cool blog post if you had the time. I've been thinking about really high volume applications that connect to haproxy instances that do DB load balancing but even that has it's own set of initial latency issues. I haven't fully figured out that architecture. In python in particular I'd really love to know how to setup the connection reuse.
  - Mizza 10y ago
    Check out the handler source code in Zappa for an example of a pattern like that. Similarly, if you use Zappa to deploy your application, if you create your database connection when the application loads, it'll just work. Stop by the Zappa slack if you want to explore this in more detail! https://slack.zappa.io https://slack.zappa.io It could always use more investigation, but there are quite a few Zappa users at extremely high loads now without any issues.
    - avyfain 10y ago
      This project looks super interesting. Thanks for sharing it! Now I want to come up with a use case to test it out over the next few days, maybe something for the Echo? I joined your Slack, we'll see where this takes us :)
      - Mizza 10y ago
        That's a great use case, many people are using flask-ask and Zappa together to great success: https://github.com/johnwheeler/flask-ask https://github.com/johnwheeler/flask-ask
    - joeyspn 10y ago
      You've got a problem with the naked domain redirects http://zappa.io http://zappa.io https://zappa.io/ https://zappa.io/
ende 10y ago
One way to do it is to stand an http rest API middle layer in between with something like http://postgrest.com http://postgrest.com.
andrew_k 10y ago
If you are creating connections outside of handler, it gets cached by Lambda between invocations. However you'll need to tune your database settings or you'll run out of connections on traffic spikes. Depends on your pattern of usage. Also cold start of lambda in VPC could take around 15 seconds https://www.reddit.com/r/aws/comments/49l91l/lambda_functions_in_vpc_cold_boot_times_of_10/ https://www.reddit.com/r/aws/comments/49l91l/lambda_function... https://forums.aws.amazon.com/thread.jspa?messageID=735318&tstart=0 https://forums.aws.amazon.com/thread.jspa?messageID=735318&t... My experience with DynamoDb was much smoother than with Postgres RDS in VPC
- deleted 10y ago
  [deleted]
collyw 10y ago
I am curious what sort of stuff is being built that need microservices (presumably so they scale) yet still has a database back end (usually the IO is the bottleneck in most applications).
pmontra 10y ago
DynamoDB is not immune from that problem. There is no way out: if the container is terminated the connection goes down. The key is not making Lambda stop and rm the container (in docker terms). If there are enough requests the container is reused and the connection stays up, but you must initialize it outside the function. An example with DynamoDB const AWS = require("aws-sdk"); const docClient = new AWS.DynamoDB.DocumentClient(); module.exports.theFunction = (event, context, callback) => { var params = { TableName: "table", Item: { "attribute": "value" } }; docClient.put(params, function (err, data) { ... }); callback(null, {...}); }; I didn't try but it should work with any other db. However, is Lambda cost effective for services with the amount of requests required to have their containers almost never terminated?
- brianwawok 10y ago
  If there are enough requests to keep your container up all the time, why are you using lambda??
  - pmontra 10y ago
    Exactly what I was asking but I didn't do the math. Maybe it's OK for burst shaped traffic. The first request pays a toll, the others are quick.
jondubois 10y ago
I think Lambda was never designed for building entire apps/APIs - It was designed for handling mid-sized processing tasks.
bni 10y ago
Is application side connection pooling always necessary to have good response time? MySQL has a really fast server side connection pool.
- brianwawok 10y ago
  Usually 15 or 20 ms hit to make a new connection even in MySQL. Also adds a fairly big server load. I suspect if a given MySQL instance supports 1000 long lived connections.. it would only support 100-200 connections that are closing every request. (Have not benchmarked this side, would be curious to see )
breandr 10y ago
Hi Everhusk, I don't specifically call it out in the article, but the code is actually reusing database connections. As mentioned in other comments, the trick is to establish the connection outside of the handler so that the connection only happens once per container.

Mizza 10y ago

If you design your correctly (at least in Python), you can cache and reuse your open database connection.

ranman 10y ago
How exactly have you been doing that? It would be a cool blog post if you had the time. I've been thinking about really high volume applications that connect to haproxy instances that do DB load balancing but even that has it's own set of initial latency issues. I haven't fully figured out that architecture. In python in particular I'd really love to know how to setup the connection reuse.
- Mizza 10y ago
  Check out the handler source code in Zappa for an example of a pattern like that. Similarly, if you use Zappa to deploy your application, if you create your database connection when the application loads, it'll just work. Stop by the Zappa slack if you want to explore this in more detail! https://slack.zappa.io https://slack.zappa.io It could always use more investigation, but there are quite a few Zappa users at extremely high loads now without any issues.
  - avyfain 10y ago
    This project looks super interesting. Thanks for sharing it! Now I want to come up with a use case to test it out over the next few days, maybe something for the Echo? I joined your Slack, we'll see where this takes us :)
    - Mizza 10y ago
      That's a great use case, many people are using flask-ask and Zappa together to great success: https://github.com/johnwheeler/flask-ask https://github.com/johnwheeler/flask-ask
  - joeyspn 10y ago
    You've got a problem with the naked domain redirects http://zappa.io http://zappa.io https://zappa.io/ https://zappa.io/

ende 10y ago

One way to do it is to stand an http rest API middle layer in between with something like http://postgrest.com http://postgrest.com.

andrew_k 10y ago

If you are creating connections outside of handler, it gets cached by Lambda between invocations. However you'll need to tune your database settings or you'll run out of connections on traffic spikes. Depends on your pattern of usage. Also cold start of lambda in VPC could take around 15 seconds https://www.reddit.com/r/aws/comments/49l91l/lambda_functions_in_vpc_cold_boot_times_of_10/ https://www.reddit.com/r/aws/comments/49l91l/lambda_function... https://forums.aws.amazon.com/thread.jspa?messageID=735318&tstart=0 https://forums.aws.amazon.com/thread.jspa?messageID=735318&t... My experience with DynamoDb was much smoother than with Postgres RDS in VPC

deleted 10y ago
[deleted]

collyw 10y ago

I am curious what sort of stuff is being built that need microservices (presumably so they scale) yet still has a database back end (usually the IO is the bottleneck in most applications).

pmontra 10y ago

DynamoDB is not immune from that problem. There is no way out: if the container is terminated the connection goes down. The key is not making Lambda stop and rm the container (in docker terms). If there are enough requests the container is reused and the connection stays up, but you must initialize it outside the function. An example with DynamoDB const AWS = require("aws-sdk"); const docClient = new AWS.DynamoDB.DocumentClient(); module.exports.theFunction = (event, context, callback) => { var params = { TableName: "table", Item: { "attribute": "value" } }; docClient.put(params, function (err, data) { ... }); callback(null, {...}); }; I didn't try but it should work with any other db. However, is Lambda cost effective for services with the amount of requests required to have their containers almost never terminated?

brianwawok 10y ago
If there are enough requests to keep your container up all the time, why are you using lambda??
- pmontra 10y ago
  Exactly what I was asking but I didn't do the math. Maybe it's OK for burst shaped traffic. The first request pays a toll, the others are quick.

jondubois 10y ago

I think Lambda was never designed for building entire apps/APIs - It was designed for handling mid-sized processing tasks.

bni 10y ago

Is application side connection pooling always necessary to have good response time? MySQL has a really fast server side connection pool.

brianwawok 10y ago
Usually 15 or 20 ms hit to make a new connection even in MySQL. Also adds a fairly big server load. I suspect if a given MySQL instance supports 1000 long lived connections.. it would only support 100-200 connections that are closing every request. (Have not benchmarked this side, would be curious to see )

breandr 10y ago

Hi Everhusk, I don't specifically call it out in the article, but the code is actually reusing database connections. As mentioned in other comments, the trick is to establish the connection outside of the handler so that the connection only happens once per container.