8 ms·

Amazon Aurora Serverless

111 points by tiagobraw 8y ago

talawahdotnet 8y ago
I feel like the choice of the name "serverless" is unnecessarily confusing/controversial, but I like the concept. The ability to scale down to zero could certainly be useful for automated testing use cases. As with anything on AWS it will take some testing to discover all the quirks and caveats, but I like the general direction this is headed.
- daveFNbuck 8y ago
  > The ability to scale down to zero could certainly be useful for automated testing use cases. Isn't that exactly Pricing Example 2?
  - talawahdotnet 8y ago
    Yup. I was just saying it is a useful feature.
- kondro 8y ago
  AWS tends to put the name "serverless" to anything that is designed to work with the rest of their serverless platform. In this case, Aurora Serverless gets the name because it will scale to 0 which is useful for things like Lambda or Batch functions that only get called sporadically.
  - candiodari 8y ago
    I don't get why using a multitenant setup for MySQL (like in the PHP days) isn't a much better solution for this, as it doesn't require many instances of MySQL (or many VMs each running a single instance), nor does it require constant resources. This does require constant resource use, and you incur the significant startup delay for what seems to me very little reason. But I guess for Amazon, inefficiency translates to $$$, so wouldn't really make sense to do this. I wonder that more generally about cloud as well. For many things making the backends multitenant would be far cheaper for the customers and just a bit more expensive for the providers ... there's the security aspect (but is it really that different ? Even with VMs the kernel and hypervisors still need to not leak information, and even with Physical servers the "reboot and recovery" and remote control mechanisms are often still avenues for security exploits) This low level separation is easy and cheap to implement for the providers, but (it seems to me) much more expensive for the customers. But I guess that's pretty much true for all cloud stuff. It provides automation, but at an operational cost, which is really not the trade off I'd like to make.
- mrep 8y ago
  Initially, I was responding to agree with you. However, my first renaming thought was for autoscaling aurora which I realized was incorrect because you still have to deal with things like server security updates. Serverless sounds like the better term to me still because it sounds exactly like what it describes: you never have to deal with a server. Although, I would actually add auto scaling to the name (serverless autoscaling aurora) so users know they don't have to deal with servers nor autoscaling. What do you guys think?
  - mrep 8y ago
    Can any marketers comment because I feel like this has gotta be purely due to marketers not wanting their product to become a generic term since litery no company names any product in a way that directly describes their product.
    - evfanknitram 8y ago
      So Azure SQL Database, Azure Virtual Machines, Azure Bot Service does not describe the products? AWS is hilariousy bad at naming. Not all companies are.
      - some_account 8y ago
        I very much prefer AWS names to the Azure naming, but I don't like Windows culture or behavior either so I think it's just general dislike of Microsoft software.
        evfanknitram 8y ago
        It sounds like you describe something which is irrational.
      - guitarbill 8y ago
        Azure does have a benefit of hindsight, and often their stuff is just named Azure + Obvious Name. On the other hand, they came up with Bing (talk about hilariously bad), and Azure. Which they've chosen to pronounce AZH-ər (Ajer) and not AZ-yoor (Ajur, like in Côte d'Azur). This is non-obvious, and makes them sound like uncultured Americans (like when you pronounce niche as nitch; instead of like cache, which I haven't heard pronounced catch (yet)).
- some_account 8y ago
  The general trend is serverless because not thinking about scaling is a very nice idea for developers and companies. Once you see your environment adapt to the load on the fly, I don't think you ever want to go back to manually handling it.
- hasa 8y ago
  Second that. Serverless is a buzzword like AI. But the concept is brilliant from many perspectives. It may save architects and developers from db scaling headaches.
red_dinosaur 8y ago
Assuming this hasn't changed since the preview started: > The cool down period for scale-down is 15 minutes since the last scaling operation. The cool down period for scale-up is 5 minutes since the last scaling operation.
- skunkworker 8y ago
  My biggest concern is how long will it take to warm up from cold? But it's good knowing that it'll remain warm for 15 min.
  - kondro 8y ago
    It says 25 seconds on average. The cold-to-warm state change sounds like it's something you would only ever want to do in batch jobs. But I can see many situations where you have a DB that only needs to actually be awake for short periods of time throughout the day for batch jobs. And even more that only need the barest minimum capacity available 24/7 with large spikes for batch-based processing.
  - ranman 8y ago
    > But it's good knowing that it'll remain warm for 15 min. Not sure what you're talking about. It's warm for as long as you want it to be. That's configurable. Read the post.
- talawahdotnet 8y ago
  It has according to this blog post: https://aws.amazon.com/blogs/aws/aurora-serverless-ga/ https://aws.amazon.com/blogs/aws/aurora-serverless-ga/ "The service currently has autoscaling cooldown periods of 1.5 minutes for scaling up and 5 minutes for scaling down"
- mrep 8y ago
  From what I have read here / experienced personally, google scaling > amazon scaling > microsoft scaling and each one is an order of magnitude greater in performance (there is an article on microsoft functions vs Amazon lambda which gives you the idea and don't even get me started on google tech versus microsoft/amazon). However, it is the opposite for sales/marketing/support apparently. It'll be interesting to see where the market goes towards in the future.
mooreds 8y ago
For testing and development databases, would prefer to scale to zero. Weird that they don't offer that as an option.
- daveFNbuck 8y ago
  Isn't that Pricing Example 2?
- kondro 8y ago
  It does scale to 0 when there are no connections to the DB. Cold-to-warm takes 25 seconds, but for dev/test this shouldn't be a problem.
  - mooreds 8y ago
    Ah, I totally misread this line: "Aurora Serverless can scale from a minimum of 2 ACUs to a maximum of 256 ACUs. You can specify the minimum and maximum ACUs your database can consume." I thought it meant you had to always run 2 ACUs. My bad.
- ranman 8y ago
  literally in the first two paragraphs of the article.
guru4consulting 8y ago
I had assumed that this would be a HTTPS service with a public end-point, available outside a VPC. But, looks like it is still deployed inside the VPC and accessible over JDBC. Did anyone find any configuration option to deploy it outside VPC? Overall, it looks like good old Aurora RDS with an auto start/stop option, and costing based on that start/stop duration.
- nahname 8y ago
  Aurora requires a VPC, but you can create public subnets to allow for external access to the database. https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-public-private-vpc.html https://docs.aws.amazon.com/AmazonECS/latest/developerguide/...
  - guru4consulting 8y ago
    but then, placing the instance in a public subnet is also not a good idea. And moreover, JDBC connections are not optimized for serverless scenarios.. the cost of connecting/dis-connecting are too high when compared to a light-weight HTTP. And any vulnerability can be potentially exploited. I was hoping to see a true HTTP service, highly optimized for Lambda clients.
    - sarvind 8y ago
      The blog post [1] provides additional insights that imply it can work well clients despite its ephemerality, and is optimized for client connection pools. "Scaling operations are transparent to the connected clients and applications since existing connections and session state are transferred to the new nodes." As for the public subnet concern, you can apply security groups to your database instance to ensure only authorized CIDR ranges can call it. Additionally, you can use IAM authentication [2] for DB callers (only recommended for light-weight applications with low concurrency), [1] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Using... [2] https://aws.amazon.com/blogs/aws/aurora-serverless-ga https://aws.amazon.com/blogs/aws/aurora-serverless-ga
paulddraper 8y ago
AWS, why not PostgreSQL? The bare minimum cost for Aurora PostgreSQL is $200/month. Aurora MySQL is an order of magnitude less. Why not PostgreSQL?
- ranman 8y ago
  It's on the way.
  - paulddraper 8y ago
    That's good to hear. Because Aurora PostgreSQL has already behind MySQL for years. Glad to hear this isn't a perpetual condition.
tgp22 8y ago
This definitely works from a cost pov. We use a small instance of AWS Aurora which is mostly idle. If we go this route we will need to see how quick the cold-start is and how quick the ACU scaling is. Serverless SQL DB - definitely a nice way to to go.
dev_dull 8y ago
> Pay only for the database resources you consume, on a per-second basis. > You pay a flat rate per second of ACU usage, with a minimum of 5 minutes of usage each time the database is activated. Cool idea, but an IO operation once every 5 minutes is considered (billing-wise) a full time service. This could unlock a lot of cool potential if it had finer granularity. I can think of some IoT applications that have have infrequent operations which would be cost effective with less minimum time.
smokedmeat 8y ago
Is serverless Aurora available in a multi-az high availability form as well?
- ranman 8y ago
  It is multi-az yes.
pritambarhate 8y ago
This really is a new paradigm shift for relational databases. In general, on the DB side, you are not able to scale horizontally like application tier. So for the fear of degradation of service, most of us end up overprovisioning on the DB side. With Serverless, auto-scaling Aurora, I think things will change for the DB tier too. The Serverless section in Aurora FAQ [1] is also worth reading. The main gotcha I think is: >> Q: Why isn't my Aurora Serverless DB Cluster automatically scaling? >> Once a scaling operation is initiated, Aurora Serverless attempts to find a scaling point, which is is a point in time at which the database can safely complete scaling. Aurora Serverless might not be able to find a scaling point if you have long-running queries or transactions in progress, or temporary tables or table locks in use. So I think one will need to focus on quick OLTP type queries. Still, I am quite excited to see how this shapes up. [1]: https://aws.amazon.com/rds/aurora/faqs/ https://aws.amazon.com/rds/aurora/faqs/
- stephenr 8y ago
  The 'main gotcha' seems like the common thing people always gloss over when using IaaS. It solves already solvable problems (albeit with more specialised tools/people) with a 'magic' solution that works until it doesn't, and then requires even more specialised people to maybe solve it. But that's the whole profit model for IaaS: offer 'magic' solutions that are worse than what a qualified ops team will build, for the same money, to small companies who dont actually need the bells and whistles, and hope that by the time they do need the bells and whistles they're locked into your platform too much to change when they inevitably realise they made a huge mistake.
  - braythwayt 8y ago
    Well, there is a general thing where small companies build a thing, and then they scale to the point where the thing doesn't work any more, and they have to make some changes. History has taught me that accepting the inevitability of change is far better than trying to build a thing with so many features that it need never be changed. In this particular case, I find it hard to believe that anybody will be "locked in" to a serverless SQL database in the same sense that, say, being a game company on top of iOS "locks you into the Apple Ecosystem," or writing your app in Rails "locks you into Ruby."
joecot 8y ago
This is extremely frustrating. One of the biggest use cases for wanting to use Aurora serverless is connecting to MySQL using lambda functions -- Function as a Service apps that can still use rdbms. The problem is that RDS endpoints only exist inside vpcs, and cold starting a lambda function with a vpc network interface can take 10+ seconds, making it useless for an API. I had hoped that, given they were making a serverless MySQL service, they'd make sure it actually plays well with lambda. Nope. Same problems regular RDS has. No better method of securing the connection beyond vpc firewall rules. Amazon, it's been a problem since lambda was launched and it hasn't been fixed. Either fix lambda vpc cold start times, or provide a better way to connect lambda to RDS. Just burying the problem only pisses off your customers when they try to buy into your hype. And since it always comes up: * Yes, there is plenty of reason to want to use MySQL with lambda. Wanting to run software on FaaS does not mean wanting to abandon rdbms. For a small app, dynamodb is overkill; for a small app that turns into a large app, dynamodb is a money pit. * No, adding scheduled heartbeat requests to the lambda functions so they never have to cold start is not a real, long term, scalable solution. It's a hack, it doesn't solve the problem if your app actually scales up, and infrastructure shouldn't depend on horrible hacks to function correctly.
- ranman 8y ago
  I’m sorry that you’re frustrated by our incremental improvement. This solves many problems for many people and I’m sorry it doesn’t solve your specific use case. That said, the problems you mentioned are definitely on the roadmap. Give us some time. We’re not burying the problem.
  - joecot 8y ago
    VPC NICs for Lambda were released in February 2016, and API Gateway + Lambda is heavily pushed as a use case. It's been 2 1/2 years, numerous new features have been released since then, both for Lamda and RDS, and this has not been resolved. The problem was apparent as soon as the feature was released. There are hundreds of blog articles about the problem, all of which tell you to just never let your Lambda instances thaw to try to combat it. Numerous frameworks for Lambda include automated plugins to schedule warming the lambda function every 5 minutes. It's been an issue since day 1, and for 2 years a solution has been "coming soon". If my use case is small, that is only true because everyone else gave up on this use case, because it doesn't work, or resigned themselves to using a terrible hack to get it to work. Even though AWS Marketing heavily implies it's a great workflow, and leaves developers to run face first into its problems. To be clear though, are you saying that there's a roadmap for fixing Lambda in VPC, or a roadmap for connecting to Aurora with Lambda without being in a VPC? At this point I'd take the latter and forget about the rest.
oh_hello 8y ago
I'm excited for this. I have a few small, low budget projects that would either require expensive RDS instances or awkward data management to force the use of Dynamo. Now there appears to be a low cost SQL solution for serverless projects.
- Cthulhu_ 8y ago
  You say low cost, but if I'm reading that article right, one of the lowest settings (2 availability units) is 12 cents per hour, so if it's being minimally used 100% of the time, it's still a good $86 per month. You can get a cheap DB instance or hosting outside of Amazon for a fraction of that. And it goes up if the DB gets some actual load.
  - alexbilbie 8y ago
    If your DB load is consistent and the cost of serverless outweights standard instance then yes go for a standard instance. However if your DB load is very spiky then serverless Aurora might be cheaper than running an over provisioned instance.
    - mwarkentin 8y ago
      Just need to give the option to run at 1 unit.. that'd get you in the ballpark of a normal t2.small server..
deleted 8y ago
[deleted]
bastawhiz 8y ago
I have to say, it's really frustrating that Postgres is a second class citizen in the AWS ecosystem. Aurora in general is often far behind on Postgres support relative to MySQL. I hope this changes going forward.
- hasa 8y ago
  Isn't AWS Redshift based on Postgresgl ?
ciferkey 8y ago
Very exciting! I made a simple bot to scrape lunch specials menus off Instagram and post them to my company's slack (http://blog.matthewbrunelle.com/projects/2018/05/07/Soup-Bot.html http://blog.matthewbrunelle.com/projects/2018/05/07/Soup-Bot...). One point point was making custom lambas to spin up/down the DB and everything that was needed to use it (a NAT on EC2, and an elastic IP since the default NAT aww provides is pricey). The bot only needs to run for an hour on weekdays so there's no point keeping everything running 24/7. Excited to try replacing all of that with this!

111 points by tiagobraw 8y ago

I feel like the choice of the name "serverless" is unnecessarily confusing/controversial, but I like the concept. The ability to scale down to zero could certainly be useful for automated testing use cases. As with anything on AWS it will take some testing to discover all the quirks and caveats, but I like the general direction this is headed.

daveFNbuck 8y ago
> The ability to scale down to zero could certainly be useful for automated testing use cases. Isn't that exactly Pricing Example 2?
- talawahdotnet 8y ago
  Yup. I was just saying it is a useful feature.
kondro 8y ago
AWS tends to put the name "serverless" to anything that is designed to work with the rest of their serverless platform. In this case, Aurora Serverless gets the name because it will scale to 0 which is useful for things like Lambda or Batch functions that only get called sporadically.
- candiodari 8y ago
  I don't get why using a multitenant setup for MySQL (like in the PHP days) isn't a much better solution for this, as it doesn't require many instances of MySQL (or many VMs each running a single instance), nor does it require constant resources. This does require constant resource use, and you incur the significant startup delay for what seems to me very little reason. But I guess for Amazon, inefficiency translates to $$$, so wouldn't really make sense to do this. I wonder that more generally about cloud as well. For many things making the backends multitenant would be far cheaper for the customers and just a bit more expensive for the providers ... there's the security aspect (but is it really that different ? Even with VMs the kernel and hypervisors still need to not leak information, and even with Physical servers the "reboot and recovery" and remote control mechanisms are often still avenues for security exploits) This low level separation is easy and cheap to implement for the providers, but (it seems to me) much more expensive for the customers. But I guess that's pretty much true for all cloud stuff. It provides automation, but at an operational cost, which is really not the trade off I'd like to make.
mrep 8y ago
Initially, I was responding to agree with you. However, my first renaming thought was for autoscaling aurora which I realized was incorrect because you still have to deal with things like server security updates. Serverless sounds like the better term to me still because it sounds exactly like what it describes: you never have to deal with a server. Although, I would actually add auto scaling to the name (serverless autoscaling aurora) so users know they don't have to deal with servers nor autoscaling. What do you guys think?
- mrep 8y ago
  Can any marketers comment because I feel like this has gotta be purely due to marketers not wanting their product to become a generic term since litery no company names any product in a way that directly describes their product.
  - evfanknitram 8y ago
    So Azure SQL Database, Azure Virtual Machines, Azure Bot Service does not describe the products? AWS is hilariousy bad at naming. Not all companies are.
    - some_account 8y ago
      I very much prefer AWS names to the Azure naming, but I don't like Windows culture or behavior either so I think it's just general dislike of Microsoft software.
      - evfanknitram 8y ago
        It sounds like you describe something which is irrational.
    - guitarbill 8y ago
      Azure does have a benefit of hindsight, and often their stuff is just named Azure + Obvious Name. On the other hand, they came up with Bing (talk about hilariously bad), and Azure. Which they've chosen to pronounce AZH-ər (Ajer) and not AZ-yoor (Ajur, like in Côte d'Azur). This is non-obvious, and makes them sound like uncultured Americans (like when you pronounce niche as nitch; instead of like cache, which I haven't heard pronounced catch (yet)).
some_account 8y ago
The general trend is serverless because not thinking about scaling is a very nice idea for developers and companies. Once you see your environment adapt to the load on the fly, I don't think you ever want to go back to manually handling it.
hasa 8y ago
Second that. Serverless is a buzzword like AI. But the concept is brilliant from many perspectives. It may save architects and developers from db scaling headaches.

red_dinosaur 8y ago

Assuming this hasn't changed since the preview started: > The cool down period for scale-down is 15 minutes since the last scaling operation. The cool down period for scale-up is 5 minutes since the last scaling operation.

skunkworker 8y ago
My biggest concern is how long will it take to warm up from cold? But it's good knowing that it'll remain warm for 15 min.
- kondro 8y ago
  It says 25 seconds on average. The cold-to-warm state change sounds like it's something you would only ever want to do in batch jobs. But I can see many situations where you have a DB that only needs to actually be awake for short periods of time throughout the day for batch jobs. And even more that only need the barest minimum capacity available 24/7 with large spikes for batch-based processing.
- ranman 8y ago
  > But it's good knowing that it'll remain warm for 15 min. Not sure what you're talking about. It's warm for as long as you want it to be. That's configurable. Read the post.
talawahdotnet 8y ago
It has according to this blog post: https://aws.amazon.com/blogs/aws/aurora-serverless-ga/ https://aws.amazon.com/blogs/aws/aurora-serverless-ga/ "The service currently has autoscaling cooldown periods of 1.5 minutes for scaling up and 5 minutes for scaling down"
mrep 8y ago
From what I have read here / experienced personally, google scaling > amazon scaling > microsoft scaling and each one is an order of magnitude greater in performance (there is an article on microsoft functions vs Amazon lambda which gives you the idea and don't even get me started on google tech versus microsoft/amazon). However, it is the opposite for sales/marketing/support apparently. It'll be interesting to see where the market goes towards in the future.

mooreds 8y ago

For testing and development databases, would prefer to scale to zero. Weird that they don't offer that as an option.

daveFNbuck 8y ago
Isn't that Pricing Example 2?
kondro 8y ago
It does scale to 0 when there are no connections to the DB. Cold-to-warm takes 25 seconds, but for dev/test this shouldn't be a problem.
- mooreds 8y ago
  Ah, I totally misread this line: "Aurora Serverless can scale from a minimum of 2 ACUs to a maximum of 256 ACUs. You can specify the minimum and maximum ACUs your database can consume." I thought it meant you had to always run 2 ACUs. My bad.
ranman 8y ago
literally in the first two paragraphs of the article.

guru4consulting 8y ago

I had assumed that this would be a HTTPS service with a public end-point, available outside a VPC. But, looks like it is still deployed inside the VPC and accessible over JDBC. Did anyone find any configuration option to deploy it outside VPC? Overall, it looks like good old Aurora RDS with an auto start/stop option, and costing based on that start/stop duration.

nahname 8y ago
Aurora requires a VPC, but you can create public subnets to allow for external access to the database. https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-public-private-vpc.html https://docs.aws.amazon.com/AmazonECS/latest/developerguide/...
- guru4consulting 8y ago
  but then, placing the instance in a public subnet is also not a good idea. And moreover, JDBC connections are not optimized for serverless scenarios.. the cost of connecting/dis-connecting are too high when compared to a light-weight HTTP. And any vulnerability can be potentially exploited. I was hoping to see a true HTTP service, highly optimized for Lambda clients.
  - sarvind 8y ago
    The blog post [1] provides additional insights that imply it can work well clients despite its ephemerality, and is optimized for client connection pools. "Scaling operations are transparent to the connected clients and applications since existing connections and session state are transferred to the new nodes." As for the public subnet concern, you can apply security groups to your database instance to ensure only authorized CIDR ranges can call it. Additionally, you can use IAM authentication [2] for DB callers (only recommended for light-weight applications with low concurrency), [1] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Using... [2] https://aws.amazon.com/blogs/aws/aurora-serverless-ga https://aws.amazon.com/blogs/aws/aurora-serverless-ga

paulddraper 8y ago

AWS, why not PostgreSQL? The bare minimum cost for Aurora PostgreSQL is $200/month. Aurora MySQL is an order of magnitude less. Why not PostgreSQL?

ranman 8y ago
It's on the way.
- paulddraper 8y ago
  That's good to hear. Because Aurora PostgreSQL has already behind MySQL for years. Glad to hear this isn't a perpetual condition.

tgp22 8y ago

This definitely works from a cost pov. We use a small instance of AWS Aurora which is mostly idle. If we go this route we will need to see how quick the cold-start is and how quick the ACU scaling is. Serverless SQL DB - definitely a nice way to to go.

dev_dull 8y ago

> Pay only for the database resources you consume, on a per-second basis. > You pay a flat rate per second of ACU usage, with a minimum of 5 minutes of usage each time the database is activated. Cool idea, but an IO operation once every 5 minutes is considered (billing-wise) a full time service. This could unlock a lot of cool potential if it had finer granularity. I can think of some IoT applications that have have infrequent operations which would be cost effective with less minimum time.

smokedmeat 8y ago

Is serverless Aurora available in a multi-az high availability form as well?

ranman 8y ago
It is multi-az yes.

pritambarhate 8y ago

This really is a new paradigm shift for relational databases. In general, on the DB side, you are not able to scale horizontally like application tier. So for the fear of degradation of service, most of us end up overprovisioning on the DB side. With Serverless, auto-scaling Aurora, I think things will change for the DB tier too. The Serverless section in Aurora FAQ [1] is also worth reading. The main gotcha I think is: >> Q: Why isn't my Aurora Serverless DB Cluster automatically scaling? >> Once a scaling operation is initiated, Aurora Serverless attempts to find a scaling point, which is is a point in time at which the database can safely complete scaling. Aurora Serverless might not be able to find a scaling point if you have long-running queries or transactions in progress, or temporary tables or table locks in use. So I think one will need to focus on quick OLTP type queries. Still, I am quite excited to see how this shapes up. [1]: https://aws.amazon.com/rds/aurora/faqs/ https://aws.amazon.com/rds/aurora/faqs/

stephenr 8y ago
The 'main gotcha' seems like the common thing people always gloss over when using IaaS. It solves already solvable problems (albeit with more specialised tools/people) with a 'magic' solution that works until it doesn't, and then requires even more specialised people to maybe solve it. But that's the whole profit model for IaaS: offer 'magic' solutions that are worse than what a qualified ops team will build, for the same money, to small companies who dont actually need the bells and whistles, and hope that by the time they do need the bells and whistles they're locked into your platform too much to change when they inevitably realise they made a huge mistake.
- braythwayt 8y ago
  Well, there is a general thing where small companies build a thing, and then they scale to the point where the thing doesn't work any more, and they have to make some changes. History has taught me that accepting the inevitability of change is far better than trying to build a thing with so many features that it need never be changed. In this particular case, I find it hard to believe that anybody will be "locked in" to a serverless SQL database in the same sense that, say, being a game company on top of iOS "locks you into the Apple Ecosystem," or writing your app in Rails "locks you into Ruby."

joecot 8y ago

This is extremely frustrating. One of the biggest use cases for wanting to use Aurora serverless is connecting to MySQL using lambda functions -- Function as a Service apps that can still use rdbms. The problem is that RDS endpoints only exist inside vpcs, and cold starting a lambda function with a vpc network interface can take 10+ seconds, making it useless for an API. I had hoped that, given they were making a serverless MySQL service, they'd make sure it actually plays well with lambda. Nope. Same problems regular RDS has. No better method of securing the connection beyond vpc firewall rules. Amazon, it's been a problem since lambda was launched and it hasn't been fixed. Either fix lambda vpc cold start times, or provide a better way to connect lambda to RDS. Just burying the problem only pisses off your customers when they try to buy into your hype. And since it always comes up: * Yes, there is plenty of reason to want to use MySQL with lambda. Wanting to run software on FaaS does not mean wanting to abandon rdbms. For a small app, dynamodb is overkill; for a small app that turns into a large app, dynamodb is a money pit. * No, adding scheduled heartbeat requests to the lambda functions so they never have to cold start is not a real, long term, scalable solution. It's a hack, it doesn't solve the problem if your app actually scales up, and infrastructure shouldn't depend on horrible hacks to function correctly.

ranman 8y ago
I’m sorry that you’re frustrated by our incremental improvement. This solves many problems for many people and I’m sorry it doesn’t solve your specific use case. That said, the problems you mentioned are definitely on the roadmap. Give us some time. We’re not burying the problem.
- joecot 8y ago
  VPC NICs for Lambda were released in February 2016, and API Gateway + Lambda is heavily pushed as a use case. It's been 2 1/2 years, numerous new features have been released since then, both for Lamda and RDS, and this has not been resolved. The problem was apparent as soon as the feature was released. There are hundreds of blog articles about the problem, all of which tell you to just never let your Lambda instances thaw to try to combat it. Numerous frameworks for Lambda include automated plugins to schedule warming the lambda function every 5 minutes. It's been an issue since day 1, and for 2 years a solution has been "coming soon". If my use case is small, that is only true because everyone else gave up on this use case, because it doesn't work, or resigned themselves to using a terrible hack to get it to work. Even though AWS Marketing heavily implies it's a great workflow, and leaves developers to run face first into its problems. To be clear though, are you saying that there's a roadmap for fixing Lambda in VPC, or a roadmap for connecting to Aurora with Lambda without being in a VPC? At this point I'd take the latter and forget about the rest.

oh_hello 8y ago

I'm excited for this. I have a few small, low budget projects that would either require expensive RDS instances or awkward data management to force the use of Dynamo. Now there appears to be a low cost SQL solution for serverless projects.

Cthulhu_ 8y ago
You say low cost, but if I'm reading that article right, one of the lowest settings (2 availability units) is 12 cents per hour, so if it's being minimally used 100% of the time, it's still a good $86 per month. You can get a cheap DB instance or hosting outside of Amazon for a fraction of that. And it goes up if the DB gets some actual load.
- alexbilbie 8y ago
  If your DB load is consistent and the cost of serverless outweights standard instance then yes go for a standard instance. However if your DB load is very spiky then serverless Aurora might be cheaper than running an over provisioned instance.
  - mwarkentin 8y ago
    Just need to give the option to run at 1 unit.. that'd get you in the ballpark of a normal t2.small server..

deleted 8y ago

[deleted]

bastawhiz 8y ago

I have to say, it's really frustrating that Postgres is a second class citizen in the AWS ecosystem. Aurora in general is often far behind on Postgres support relative to MySQL. I hope this changes going forward.

hasa 8y ago
Isn't AWS Redshift based on Postgresgl ?

ciferkey 8y ago

Very exciting! I made a simple bot to scrape lunch specials menus off Instagram and post them to my company's slack (http://blog.matthewbrunelle.com/projects/2018/05/07/Soup-Bot.html http://blog.matthewbrunelle.com/projects/2018/05/07/Soup-Bot...). One point point was making custom lambas to spin up/down the DB and everything that was needed to use it (a NAT on EC2, and an elastic IP since the default NAT aww provides is pricey). The bot only needs to run for an hour on weekdays so there's no point keeping everything running 24/7. Excited to try replacing all of that with this!