8 ms·
AWS cancels serverless Postgres service that scales to zero
- superdeeda 3y agoI learned the expensive wya that Aurora Serverless v2 does not, in fact, scale down to 0 when I created three instances for a few days when I was testing out configuration changes. It ended up costing us $300.
- candiddevmike 3y agoFWIW the base cost is around $44/month on paper (plus storage). You're always paying for 0.5 ACU.
- k4rli 3y ago$100 a month for a single Postgres instance is insane
- nijave 3y agoAurora Postgres is an interesting product where AWS made significant changes to Postgres storage architecture, however it has a fairly high fixed cost. It seems in many cases RDS Postgres is cheaper and even when using Aurora, using fixed-size instance classes instead of "serverless" autoscaling classes (with ACUs) is a lot cheaper. Fixed Aurora on-demand instances are roughly 17% the cost of ACUs. Fixed instances are usually >= 30% cheaper on top of that if you reserve. Aurora heavily favors performance and availability over cost.
- qaq 3y agoAWS has a decent implementation transactional distributed block storage that they leverage for a few technically decent offerings. Aurora being one of them.
- zer00eyz 3y agoI used to work for a business that was predominantly used by women. Like clock work, at 3pm PS most weekdays there was a spike in traffic as our customers became active on our system. I would love to see the real numbers behind the costs. Is time of day dependency the real price driver here? Are some customers getting more value than others out of this system?
- 4ggr0 3y ago> [...] predominantly used by women [...] Like clock work, at 3pm PS most weekdays there was a spike in traffic what knowledge gap do i have about the collective womanhood that I don't understand why a service which is predominantly used by women gets traffic spikes at a specific time of day? Is 3PM PST when most women sync their in-built RTC?
- dragonwriter 3y ago3PM is around when school ends for most k-12 schools; if its something someone might check quickly in a free moment at/after work and/or in the car waiting to pickup kids, I can see a big spike from teachers (disproportionately women) and primary child-raisers (disproportionately women) around 3pm.
- zer00eyz 3y agoTo answer, daily peak, 3pm PT to about 11pm PT... That starts 6pm ET, so post work and kids, it's "free time". By 6pm PT(9pm ET) we would be hitting a peak, for the next hour or two and then winding down till 11. It's the prime time for "adult working women" as a demographic.
- 4ggr0 3y agoAh I see. So it's just generally the "adult working parent" prime time, but you had a focus on women because they were the service's target demographic.
- simonw 3y agoThe term "serverless" is so confusing if to me. I had settled on "scale to zero" as my chosen definition for it, because it was the only definition that truly made sense to me and that I very strongly valued. Apparently AWS don't think it means that. I guess it just means "you'll never have to SSH in and upgrade anything"? The other thing that surprises me here is that one of AWS's biggest selling points is how rarely they break existing apps by turning off services developers depend on.
- throwboatyface 3y agoServerless is a paradigm where you pay for capacity, not for a specific number of instances. It's extending "cattle, not pets" beyond VMs to databases. In theory it provides better incentives for both parties: Customers don't have to pick instance size, etc. to try and accomodate their workload. They can buy throughput, etc. in fractions of an actual instance. They can look at their quota usage to determine whether to "scale up" or "scale down". Service providers are incentivized to optimize their software. If you sell instances + a service on top, there's a perverse incentive not to make your code too fast or people will buy fewer instances. In serverless, providers are incentivized to reduce their costs and optimize the entire stack. Scale to zero isn't necessarily a property of serverless - a service provider can agree to keep your data on ice for free, but that's a business decision.
- sspiff 3y agoI mean, the inverse incentive to hobble performance is still there for all the serverless stuff that bills by time spent.
- Freeaqingme 3y agoAlso, it's a way for AWS to reuse very old hardware where customers don't think about the specifications of the underlying hardware.
- Hamuko 3y agoI still consider "serverless" to mean something that does in fact scale to zero. You also don't necessarily need to SSH in and upgrade stuff with a managed service either but RDS isn't exactly serverless even though you don't really need to service the database server yourself.
- mooreds 3y agoThey are end of lifing v1, which could go to zero. V2 can't. Are there any truly serverless SQL databases out there?
- rohansingh 3y agoI've never used it and it's still in beta, but I think Cloudflare D1 would qualify: https://developers.cloudflare.com/d1/ https://developers.cloudflare.com/d1/
- jpalomaki 3y ago"Maximum database size 2 GB (Workers Paid) beta" (but you can have up to 50.000 databases) https://developers.cloudflare.com/d1/platform/limits/ https://developers.cloudflare.com/d1/platform/limits/
- donatj 3y agoOh strange, what a tiny limit. Is it kept in memory on a 32 bit machine? lol. Half our tables are bigger than that
- yencabulator 3y agoActually, yeah, I fully expect them to be running it inside WASM, so 32 bit...
- travem 3y agoI use TiDB Serverless for some smaller projects I am working on. https://www.pingcap.com/tidb-serverless/ https://www.pingcap.com/tidb-serverless/
- dcchambers 3y ago> Are there any truly serverless SQL databases out there? sqlite ;)
- tristan957 3y ago
- CSDude 3y agoServerless V2 is also shit. The minimal one equivalent ACU to a regular instance has lots of latency spikes and much slower than a t3 instance. We also tried serverless Redis. We have a tiny machine at %10 CPU utilization, but sustained queries. It costs 20x more. We were very afraid of the result we just ran it for an hour to get an idea. Serverless, pay per use sounds only good if you have very few or unusual traffic. We just run a monolithic but stateless Spring Boot Application and a dedicated RDS instance with reader in stand by. Only serverless dependency is SQS because it works nice enough. This simple infra has allowed us to iterate and pivot our startup very fast as needed. We still need more customers though.
- RhodesianHunter 3y agoYeah, the only systems likely to see real benefits from serverless are those with infrequent and intermittent traffic.
- Buttons840 3y ago> Serverless, pay per use sounds only good if you have very few or unusual traffic. And most of those cases can be handled by a $5 a month instance. Thus, usually, the most serverless can save you is $5 a month. I suppose the real appeal of serveless is you don't have to worry about maintaining the server, doing updates, etc.
- jvanderbot 3y agoHonest question, why not just use dynamodb? If I'm already mostly server less and AWS, dynamodb seems like a natural choice for a persistent store in a world where cost and zero-scaling is important.
- qvrjuec 3y agoBecause you need SQL. Dynamo can be quite powerful for querying if you use all of its features but it won't be as flexible as just using SQL
- belter 3y agoAlmost any SQL workload you can move to DynamoDB
- e12e 3y agohttps://aws.amazon.com/blogs/database/how-to-determine-if-amazon-dynamodb-is-appropriate-for-your-needs-and-then-plan-your-migration/ https://aws.amazon.com/blogs/database/how-to-determine-if-am... > Some unsuitable workloads for DynamoDB include: > Services that require ad hoc query access. Though it’s possible to use external relational frameworks to implement entity relationships across DynamoDB tables, these are generally cumbersome. > Online analytical processing (OLAP)/data warehouse implementations. These types of applications generally require distribution and the joining of fact and dimension tables that inherently provide a normalized (relational) view of your data. > Binary large object (BLOB) storage. DynamoDB can store binary items up to 400 KB, but DynamoDB is not generally suited to storing documents or images. A better architectural pattern for this implementation is to store pointers to Amazon S3 objects in a DynamoDB table.
- belter 3y agoI said moving the workload to DynamoDB, not doing the same query pattern with DynamoDB ... :-) For each of these workloads you switch the query pattern to accommodate for DynamoDB and provide a solution for your workload and/or app. That is actually the secret of NoSQL. You do the work upfront. So your reading of the recommendations is correct on the surface but incorrect on the fundamental usage aspect of it. Lets have a look at each: > Services that require ad hoc query access. Though it’s possible to use external relational frameworks to implement entity relationships across DynamoDB tables, these are generally cumbersome. A: Dont do this. Don't do ad hoc query access. Define a series of access patterns and lay out your data to support them. You dont get ad hoc query access for Netflix, Amazon or your Airline Travel website... > Online analytical processing (OLAP)/data warehouse implementations. These types of applications generally require distribution and the joining of fact and dimension tables that inherently provide a normalized (relational) view of your data. These are not most SQL Patterns. This is OLAP and was always or should be done with MPP systems, not with your relational database like your SQLServer or your Oracle. So this is going on an edge... > Binary large object (BLOB) storage. DynamoDB can store binary items up to 400 KB, but DynamoDB is not generally suited to storing documents or images. A better architectural pattern for this implementation is to store pointers to Amazon S3 objects in a DynamoDB table. Your Relational Database will not store BLOBs more efficiently than S3 anyway....
- qaq 3y agoI guess a good opportunity for someone to build a good alternative. Although it's hard to get it right because if you scale to 0 people will complain about latency and if you don't it's not really serverless.
- graphe 3y agoWhat alternative does someone need to make to compete with AWS?
- bradhe 3y agoI think the problem is that we're trying to take paradigms that are designed to be serverful (is that a word?) and shoe-horn them into working in a serverless way.
- nikita 3y ago(Neon CEO). It's possible and here is the recipe: https://twitter.com/nikitabase/status/1725394843285766463 https://twitter.com/nikitabase/status/1725394843285766463
- stuartjohnson12 3y agoAlways gotta watch what you say on Hacker News or the CEO of the company you're talking about might just arrive outta nowhere with the elbow drop of justice.
- qaq 3y agoIt helps but does not fully solve it if I need PG to have 512 GB of memory buffers under load to have reasonable latency starting from 0 and going through a number of VM migrations to finally get to properly sized VM with 512 GB of hot data in memory gonna cause a ton of latency.
- nikita 3y agoThe buffers will be cached in remote storage - so it will just work likely. In the future we are considering separating the buffer pool from compute and turning off compute doesn't drop the buffer cache. It's a research project but PolarDB did that for AliCloud.
- slau 3y agoReminder that AWS Aurora claims to have broken the CAP theorem. The literature doesn’t clarify whether they are consistent or available, and when you ask AWS directly, they tell you the CAP theorem isn’t relevant due to the way Aurora is built. They claim 99.99% availability or whatever the figure is, that it’s self healing, allows up to 2 replicas to be down for write availability and 3 replicas to be down for read availability. They promise “millisecond consistency”, whatever that even means. Some of the blogs mention that they use distributed state to extract slivers of consistency across the nodes, which is just scary stuff to hear. I’m seeing more and more usage of Aurora due to the magical component it offers, but I can’t wrap my head around what it’s actually supposed to be, and what the failure model is, like I can with RDS PG and MySQL.
- asylteltine 3y agoCan’t you bypass CAP if you just have a ton of compute in the background handling all of these tasks in real time? Aurora isn’t just some binary running on ec2 it’s probably 10 different things going on
- slau 3y agoCAP is a property of any distributed system. You have to tolerate P (considering it’s distributed), so you can only be CP or AP. CP means you can’t be available, meaning your system will stop replying to queries. AP means you’re always available, but you could receive stale data when querying. It’s irrelevant whether it’s a single binary running on EC2 or a billion services working in unison. It’s a property of the system.
- deleted 3y ago[deleted]
- candiddevmike 3y agoAWS should show us the Jepsen or GTFO
- zokier 3y ago
- nforgerit 3y agoSo if I understand correctly, we're (economically) back at the classic model of continuously running a db in a process, right?
- deleted 3y ago[deleted]
- lijok 3y agoNot yet. Dynamodb is killing it
- joshstrange 3y agoAWS Serverless MySQL/Postgres offerings are straight trash. I used v1 to build a new app but had nothing but problems. Extremely slow starts (from zero), horrible scaling (it would always get stuck), (relatively) huge bills for the smallest capacity, limitations all over the place. After the first year on that I looked into v2 but my costs would have doubled and I didn't believe their promises of faster scaling. I moved to PlanetScale [0] and was very happy ($30/mo covered prod and dev/qa vs well over that for v1 even with having scale to zero on the AWS dev/qa instances). Also you can quickly be forced into paying for RDS Proxy if you are using lambdas/similar which is not cheap (for me). PS doesn't scale to 0 but at the time $30/mo was a decent savings over AWS Aurora Serverless. This year I started to run into some issue with PS mainly around their plans changing (went from pay for reads/writes/storage to pay for compute/storage). Yes, yes, I know they still offer the $30/mo plan but it's billed as "Read/write-based billing for lower-traffic applications" and they dropped all mentions of auto-scaling. That coupled with them sleeping your non-prod DB branches (no auto-wakeup, you had to use the API or console) even after saying that was a feature of the original $30 plan rubbed me the wrong way. Eventually the costs (for what I was getting) were way too out of whack. My app is single-tenant (love it or hate it, it's what it is) so for each customer I was paying $30/mo even though this is event-based software (like in-person, physical events that happen once a year) so for most the year the DB sat there and did nothing. Given all that I looked into Neon [1] (which I had heard of here on HN, but PS support suggested them, kudos to them for recommending a competitor, I always liked their support/staff) and while going from MySQL to Postgres wasn't painless it was way easier than I had anticipated. It was one of the few times Prisma "just worked", I don't think I'd use it again though, that DB engine is so heavy especially in a lambda. I just switched over fully last week to Neon and things seem to have gone smoothly. I can now run multiple databases on the same shared compute and it scales to 0. In fact it's scale up time is absurdly fast, the DB will "wake up" on it's own when you connect to it and unlike AWS Aurora Serverless v1 it comes up in seconds instead of 30-60+ so you don't even have to account for it. With AWS I had to have something poll the backend waiting to see if the DB was awake yet, to fire off my requests, if it was asleep. With Neon I don't even consider it, the first requests just take an extra second or two if that. I don't have any ill will towards PlanetScale and I quite enjoyed their product for almost the whole time I used it. Also their support is very responsive and I loved the branching/merging features (I'll miss those but zero-downtime migrations aren't required for my use-case, just nice to have). In fact if I had written my app to be multi-tenant then I'd probably still be on them since I could just scale up to one of their higher plans. It does seem like Neon is significantly (for me/my workload) cheaper for more compute, I had queries taking _forever_ on PS that come back in a second or less on Neon all while paying less. All that said, I _highly_ recommend checking out Neon if you need "serverless" hosting for Postgres that scales to 0. [0] https://planetscale.com/ https://planetscale.com/ [1] https://neon.tech/ https://neon.tech/
- nikita 3y ago(CEO of neon.tech here) Scaling to zero is tricky. In order to make it work you need to 1. Separate storage and compute which aurora has done 2. Have a proxy between the user and an actual VM in which you host Postgres 3. Spin up this VM after authenticating a connection, stand up Postgres inside the VM and run the query You need to do all that very quickly- ideally in a couple hundred ms. This can only be accomplished with having a warm pool of VM or having extremely fast microVMs like firecracker. I believe AWS aurora just uses ec2 and that’s why scale to 0 is turned off
- htrp 3y agoSo how does your company do it?
- jerrygenser 3y agoreposting Neon ceo twitter post: https://twitter.com/nikitabase/status/1725394843285766463 https://twitter.com/nikitabase/status/1725394843285766463
- mkesper 3y agoScaling from zero did use quite some seconds so there would be some leeway for an alternative implementation.
- JackFr 3y agoI think the big takeaway is that scale-to-zero customers aren't where they're making their money.
- gregw2 3y agoOr that certain "minimum delightful product" features can be abandoned when they are inconvenient to provide.
- andrewfromx 3y agoI started using the smallest VM I can that's free. On google cloud it's 1 GB RAM, 30 GB hard drive, small CPU. I run apt install -y postgresql after I create it and run my golang app and the database ON THE SAME VM! It's amazing how much you can do with that all for free.
- awestroke 3y agoBest solution for small scales, doesn’t work anymore when you need reliability (unless you want to be always on call to restore from backup and are willing to accept data loss)
- tootie 3y agoObviously cloud computing has done wonders for building gigascale infrastructure, but it's so incredibly useful for building microscale too. You can't run half a CPU at home.
- Demiurge 3y agoOf course you can run half a CPU at home. With Ubuntu LXD, you can easily run actual, hardware 50% cpu time, or .5 priority container. Why wouldn't you be able to have complete control over what "microscale" tasks you run on your own hardware, and own OS, that you control? On the contrary, vCPU a completely cloud-centric product oriented concept, intended to abstract away what CPU you're actually running on, and how fast. These abstracts make orchestration and availability easier, but I think the key is a stable, public internet access. If you get a static IP, I don't see what wonders this does for "microscale". In fact, you probably have 90% unused "microcapacity" on all the personal hardware that's often idling, that is a lot less micro than any cloud provider will give you for free.
- donatj 3y agoThe title includes Postgres, but if I’m understanding correctly it’s actually both flavors of Aurora - Postgres and MySQL
- dcchambers 3y agoI have a feeling most people chasing "serverless" postgres or mysql with the ability to scale to zero are perfect candidates for sqlite.
- deleted 3y ago[deleted]
- zerof1l 3y agoWe wanted to switch to Aurora in our company about a year ago. We tried serverless first but decided against it. We ended up using Aurora provisioned. Main reason was cost. Aurora Serverless was 20%-40% more expensive than Aurora provisioned. One area where Aurora is superior to non-Aurora in AWS is replication lag. When running non-Aurora MySQL servers, we had replica lag sometimes reach an hour making read replicas useless. With Aurora provisioned, replica lag is always under 30ms. If your app is database heavy and you end-up using database pretty much 24/7, don't fall for serverless hype and go with Aurora Provisioned.
- 1letterunixname 3y agoIgnore requirements of paying customers because product guy or developer guy knows best. - The Amazon Way(tm)