6 ms·

Amazon RDS Proxy for AWS Lambda

78 points by Tehnix 7y ago

tybit 7y ago
This is really nice, we used lambda for event based ingestion into RDS and MySQL connection limits were such a pain. However I do wish they’d include how this will evolve alongside last years announced data API.
vpontis 7y ago
This is really cool! I've had to do the same thing with PGBouncer for Postgres + serverless and it's a pain. Would love to see this come out for Postgres and come to Google Cloud :)
beering 7y ago
TL;DR: AWS now has a managed connection pooling service for MySQL on RDS only (public preview right now). The article talks about using it in conjunction with Lambda.
- jakemal 7y ago
  It took me over half the article before I could parse "Amazon RDS Proxy for AWS Lambda" as a coherent sentence rather than a series of semi-related words strung together.
dmahanta 7y ago
hahaha JDBC as a service. We really needed this, what has the world come too.
- JonMR 7y ago
  You may already know all this, but I thought I’d add to what you said. JDBC doesn’t provide connection pooling out of the box. There are connection pooling libraries that layer into JDBC. Most frameworks and application servers do bundle a connection pool out of the box. Lambda is just a different paradigm than a single monolithic app with a connection pool. I assume that’s what you’re getting at. :) There are other use cases though. Tools like pgBouncer are fairly useful in a large enterprise where a single database has multiple clients. They provide a single point to coordinate failovers and manage resource limits. I generally prefer a 1:1 relationship between an application and a database, but sometimes you have to play the cards you’re dealt.
topicseed 7y ago
Long overdue, and now waiting for the PostgreSQL's RDS Proxy as it's the main database we use along with Lambdas... and gosh it's tedious!
dylanz 7y ago
This is awesome. We were just looking at throwing up PgBouncer in front of RDS in the coming year. Looking forward to PostgreSQL support!
- ocdnix 7y ago
  I really hope it can replace PgBouncer. We've spent way too much time getting it running in ECS, mostly due to us needing a separate PgBouncer service for the primary and replica RDS instances. And then a third ECS service running HAProxy in TCP mode to load balance between the PgBouncer services, to provide a read-only endpoint that balances between the primary and replica. Add issues like: having to patch in HAProxy TCP keep-alive parameter support (it only allows enabling/disabling it, leaving params up to the OS, which you can't control on ECS Fargate, _and_ the AWS NLB having a 350-second TCP flow idle timeout - good luck with long-running queries); AWS NLBs having horrible timing issues when handling target group changes... it was not a great experience.
  - takeda 7y ago
    We also set up pgbouncer, but after testing we realized that running it in a container added extra latency so went with ec2 instances behind ASG. For load blanacing we used ELBv1, we also tried to use NLB, but 350 second timeout, and broken behavior when after timeout it just silently closes the connections was a no-no. ELBv1 doesn't silently close connections, and idle timeout can be set to 1h. After that we configured pgbouncer to close idle connections after 55 minutes. ELBv1 also doesn't have the timing issues with health checks. IMO NLB is maybe performant, but that's the only good thing about it, in every other category it is crap and for this use case ELBv1 isn't a bottleneck. If you will plan to revise your setup, be aware that pgbouncer is single threaded so throwing multicore machines at it won't do much good. If CPU ends up being a bottleneck it is better to use a large single core instance and then adding more instances. The only problem with that is that as you're adding more instances you're increasing number of connections to the database, but you could have additional process that could monitor ASG and set the number of instances and dynamically adjust number of server-side connections (doesn't require restarting pgbouncer). BTW: Since version 1.12.0 you can also run multiple pgbouncer instances listening on the same port so that can help make a better use of extra cores if you chose multicore instances.
astuyvenberg 7y ago
Finally, this has been a long time coming. Connecting to existing RDS instances from lambda functions has been a pain for users looking to adopt serverless tech into their existing applications.
crgwbr 7y ago
Does anyone know why RDS Postgres always seems to lag behind their MySQL offering? I can imagine Postgres is that much more difficult to develop a connection pooling service for, especially since PGBouncer and PGPool-II both already exist and are widely used.
- Aeolun 7y ago
  I think both for the reasons you named, and because more people use MySQL maybe?
knodi 7y ago
If you're doing this you're using lambda's wrong.
- turdnagel 7y ago
  Let me guess, you're a DynamoDB fan?
- revicon 7y ago
  What's the right way?
  - PetahNZ 7y ago
    Not sure the op's case, but typically it's better for queue processing.
nzoschke 7y ago
I wonder how this compares to Aurora Serverless Data API. This is an HTTP API that you POST a query to and get results data back. The result format is a bit clunky but there's all the column information you'd expect. I found this golang database interface which ended up dropping in as expected with a few tweaks. https://github.com/Clever/rds/tree/birthday https://github.com/Clever/rds/tree/birthday My bill for Aurora Serverless Postgres last month was less than a dollar.
- PetahNZ 7y ago
  We use Aurora Serverless and this is a sorely needed feature for us. We get large spikes of traffic that also cause large increases in connections to our DB, but Aurora Serverless cannot scale fast enough to cater for this. Unfortunately it doesn't seem we can enable this proxy for serverless clusters.
  - sudhirj 7y ago
    Aurora serverless already has a proxy by default - if it can’t scale fast enough it’s likely because your queries are too long and interleaved to find a logical switchover point. This proxy is probably an offshoot of the serverless proxy.
    - PetahNZ 7y ago
      It has a hard limit on max_connections based on the "Aurora Capacity Unit", e.g. for 16 ACU's max_connections it is 2,000. We see spikes of 10x to 100x concurrent users in a matter of minutes (think hug of death style). And although our queries run fast and we see limited increase of CPU on the cluster we almost instantly hit the connection limit. https://www.jeremydaly.com/aurora-serverless-the-good-the-bad-and-the-scalable/ https://www.jeremydaly.com/aurora-serverless-the-good-the-ba...
joecot 7y ago
For anyone else who spent 10 minutes trying to find a link to pricing from this page and failing, here it is: https://aws.amazon.com/rds/proxy/pricing/ https://aws.amazon.com/rds/proxy/pricing/ Free til the end of 2019, 0.015 per vCPU (of the underlying RDS instance) per hour the proxy is enabled. So it is not scaled down to 0 when not in use. For comparison, a t3.micro (2 vCPU, 1gb ram) is 0.0104 per hour. If you're a big project it might be worth it, but just keep in mind rolling your own proxy or using the RDS serverless APIs are cheaper options.
revicon 7y ago
I don't understand why this isn't launching with data api support? AWS specifically built data apis into the Aurora Serverless product for this exact usecase... https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/data-api.html https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide... If I'm using Aurora Serverless right now and want to switch over to RDS Proxy I'm going to have to rewrite my calls.
foo4u 7y ago
The same functionality could have been accomplished by just allowing us to specify the max number of concurrent instances of a function that can be active at a time. With proper request queuing the problem would be solved. As usual, AWS solves a problem they failed to design for by charging for an overly complex solution. Flame me if you want but there are K8s serverless frameworks that address this out of the box.
- pavelrub 7y ago
  Specifying the max number of concurrent lambda execution has been possible for years - it's called reserved concurrency. This does nothing to solve the issue however. There are many use cases where you can't delay handling requests.

78 points by Tehnix 7y ago

This is really nice, we used lambda for event based ingestion into RDS and MySQL connection limits were such a pain. However I do wish they’d include how this will evolve alongside last years announced data API.

vpontis 7y ago

This is really cool! I've had to do the same thing with PGBouncer for Postgres + serverless and it's a pain. Would love to see this come out for Postgres and come to Google Cloud :)

beering 7y ago

TL;DR: AWS now has a managed connection pooling service for MySQL on RDS only (public preview right now). The article talks about using it in conjunction with Lambda.

jakemal 7y ago
It took me over half the article before I could parse "Amazon RDS Proxy for AWS Lambda" as a coherent sentence rather than a series of semi-related words strung together.

dmahanta 7y ago

hahaha JDBC as a service. We really needed this, what has the world come too.

JonMR 7y ago
You may already know all this, but I thought I’d add to what you said. JDBC doesn’t provide connection pooling out of the box. There are connection pooling libraries that layer into JDBC. Most frameworks and application servers do bundle a connection pool out of the box. Lambda is just a different paradigm than a single monolithic app with a connection pool. I assume that’s what you’re getting at. :) There are other use cases though. Tools like pgBouncer are fairly useful in a large enterprise where a single database has multiple clients. They provide a single point to coordinate failovers and manage resource limits. I generally prefer a 1:1 relationship between an application and a database, but sometimes you have to play the cards you’re dealt.

topicseed 7y ago

Long overdue, and now waiting for the PostgreSQL's RDS Proxy as it's the main database we use along with Lambdas... and gosh it's tedious!

dylanz 7y ago

This is awesome. We were just looking at throwing up PgBouncer in front of RDS in the coming year. Looking forward to PostgreSQL support!

ocdnix 7y ago
I really hope it can replace PgBouncer. We've spent way too much time getting it running in ECS, mostly due to us needing a separate PgBouncer service for the primary and replica RDS instances. And then a third ECS service running HAProxy in TCP mode to load balance between the PgBouncer services, to provide a read-only endpoint that balances between the primary and replica. Add issues like: having to patch in HAProxy TCP keep-alive parameter support (it only allows enabling/disabling it, leaving params up to the OS, which you can't control on ECS Fargate, _and_ the AWS NLB having a 350-second TCP flow idle timeout - good luck with long-running queries); AWS NLBs having horrible timing issues when handling target group changes... it was not a great experience.
- takeda 7y ago
  We also set up pgbouncer, but after testing we realized that running it in a container added extra latency so went with ec2 instances behind ASG. For load blanacing we used ELBv1, we also tried to use NLB, but 350 second timeout, and broken behavior when after timeout it just silently closes the connections was a no-no. ELBv1 doesn't silently close connections, and idle timeout can be set to 1h. After that we configured pgbouncer to close idle connections after 55 minutes. ELBv1 also doesn't have the timing issues with health checks. IMO NLB is maybe performant, but that's the only good thing about it, in every other category it is crap and for this use case ELBv1 isn't a bottleneck. If you will plan to revise your setup, be aware that pgbouncer is single threaded so throwing multicore machines at it won't do much good. If CPU ends up being a bottleneck it is better to use a large single core instance and then adding more instances. The only problem with that is that as you're adding more instances you're increasing number of connections to the database, but you could have additional process that could monitor ASG and set the number of instances and dynamically adjust number of server-side connections (doesn't require restarting pgbouncer). BTW: Since version 1.12.0 you can also run multiple pgbouncer instances listening on the same port so that can help make a better use of extra cores if you chose multicore instances.

astuyvenberg 7y ago

Finally, this has been a long time coming. Connecting to existing RDS instances from lambda functions has been a pain for users looking to adopt serverless tech into their existing applications.

crgwbr 7y ago

Does anyone know why RDS Postgres always seems to lag behind their MySQL offering? I can imagine Postgres is that much more difficult to develop a connection pooling service for, especially since PGBouncer and PGPool-II both already exist and are widely used.

Aeolun 7y ago
I think both for the reasons you named, and because more people use MySQL maybe?

knodi 7y ago

If you're doing this you're using lambda's wrong.

turdnagel 7y ago
Let me guess, you're a DynamoDB fan?
revicon 7y ago
What's the right way?
- PetahNZ 7y ago
  Not sure the op's case, but typically it's better for queue processing.

nzoschke 7y ago

I wonder how this compares to Aurora Serverless Data API. This is an HTTP API that you POST a query to and get results data back. The result format is a bit clunky but there's all the column information you'd expect. I found this golang database interface which ended up dropping in as expected with a few tweaks. https://github.com/Clever/rds/tree/birthday https://github.com/Clever/rds/tree/birthday My bill for Aurora Serverless Postgres last month was less than a dollar.

PetahNZ 7y ago
We use Aurora Serverless and this is a sorely needed feature for us. We get large spikes of traffic that also cause large increases in connections to our DB, but Aurora Serverless cannot scale fast enough to cater for this. Unfortunately it doesn't seem we can enable this proxy for serverless clusters.
- sudhirj 7y ago
  Aurora serverless already has a proxy by default - if it can’t scale fast enough it’s likely because your queries are too long and interleaved to find a logical switchover point. This proxy is probably an offshoot of the serverless proxy.
  - PetahNZ 7y ago
    It has a hard limit on max_connections based on the "Aurora Capacity Unit", e.g. for 16 ACU's max_connections it is 2,000. We see spikes of 10x to 100x concurrent users in a matter of minutes (think hug of death style). And although our queries run fast and we see limited increase of CPU on the cluster we almost instantly hit the connection limit. https://www.jeremydaly.com/aurora-serverless-the-good-the-bad-and-the-scalable/ https://www.jeremydaly.com/aurora-serverless-the-good-the-ba...

joecot 7y ago

For anyone else who spent 10 minutes trying to find a link to pricing from this page and failing, here it is: https://aws.amazon.com/rds/proxy/pricing/ https://aws.amazon.com/rds/proxy/pricing/ Free til the end of 2019, 0.015 per vCPU (of the underlying RDS instance) per hour the proxy is enabled. So it is not scaled down to 0 when not in use. For comparison, a t3.micro (2 vCPU, 1gb ram) is 0.0104 per hour. If you're a big project it might be worth it, but just keep in mind rolling your own proxy or using the RDS serverless APIs are cheaper options.

revicon 7y ago

I don't understand why this isn't launching with data api support? AWS specifically built data apis into the Aurora Serverless product for this exact usecase... https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/data-api.html https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide... If I'm using Aurora Serverless right now and want to switch over to RDS Proxy I'm going to have to rewrite my calls.

foo4u 7y ago

The same functionality could have been accomplished by just allowing us to specify the max number of concurrent instances of a function that can be active at a time. With proper request queuing the problem would be solved. As usual, AWS solves a problem they failed to design for by charging for an overly complex solution. Flame me if you want but there are K8s serverless frameworks that address this out of the box.

pavelrub 7y ago
Specifying the max number of concurrent lambda execution has been possible for years - it's called reserved concurrency. This does nothing to solve the issue however. There are many use cases where you can't delay handling requests.