4 ms·

Are you Netflix? Because is not theres a 99% probability you dont need any of those AWS services and just have a severe case of shiny object syndrome in your or

by esskay 1y ago

Are you Netflix? Because is not theres a 99% probability you dont need any of those AWS services and just have a severe case of shiny object syndrome in your organisation. Plenty of heavy traffic, high redundancy applications exist without the need for AWS (or any other cloud providers) overpriced "bespoke" systems.

rafaelmn 1y ago
To be honest I don't trust myself running a HA PostgreSQL setup with correct backups without spending an exorbitant effort to investigate everything (weeks/months) - do you ? I'm not even sure what effort that would take. I can't remember last time I worked with unmanaged DB in prod where I did not have a dedicated DBA/sysadmin. And I've been doing this for 15 years now. AFAIK Hetzner offers no managed database solution. I know they offer some load balancer so there's that at least. At some point in the scaling journey bare metal might be the right choice, but I get the feeling a lot of people here trivialize it.
- cdelsolar 1y ago
  If youre not Netflix then just sudo yum install postgresql and pg_dump every day, upload to S3. Has worked for me for 20 years at various companies, side projects, startups …
  - rafaelmn 1y ago
    What happens when the server goes down ? How do you update it ?
    - cdelsolar 1y ago
      you stand up another db server and load the last good dump into it i suppose
  - motorest 1y ago
    > If youre not Netflix then just sudo yum install postgresql and pg_dump every day, upload to S3. database services such as DynamoDB support a few backup strategies out of the box, including continuous backups. You just need to flips switch and never bother about it again. > Has worked for me for 20 years at various companies, side projects, startups … That's perfectly fine. There are still developers who don't even use version control at all. Some old habits die hard, even when the whole world moved on.
- esskay 1y ago
  If it requires weeks/months to sort setting that up and backups then you need a news ops person as that's insane. If you're doing it yourself, learn Ansible, you'll do it once and be set forever. You do not need "managed" database services. A managed database is no different from apt install postgesql followed by a scheduled backup. It genuinely is trivial, people seem to have this impression theres some sort of unique special sauce going on at AWS when there really isn't.
  - rafaelmn 1y ago
    But that's the thing - if I have an ops guy who can cover this then sure it makes sense - but who does at an early stage ? As a semi competent dev I can setup a terraform infra and be relatively safe with RDS. I could maybe figure out how to do it on my own in some time - but I don't know what I don't know - and I don't want to do a weekend production DB outage debugging because I messed up the replication setup or something. Maybe I'm getting old but I just don't have the energy to deal with that :)
  - bostik 1y ago
    Backups with wall-g and recurring pg_dump are indeed trivial. (Modulo S3 outage taking so long that your WAL files fill up the disk and you corrupt the entire database.) It's the HA part, especially with a high-volume DB that's challenging.
  - jameshart 1y ago
    That doesn’t give you high availability; it doesn’t give you monitoring and alerting; it doesn’t give you hardware failure detection and replacement; it doesn’t solve access control or networking… Managed databases are a lot more than apt install postgresql.
  - byryan 1y ago
    If you're doing it yourself, learn Ansible, you'll do it once and be set forever. You do not need "managed" database services. A managed database is no different from apt install postgesql followed by a scheduled backup. Genuinely no disrespect, but these statements really make it seem like you have limited experience building an HA scalable system. And no, you don't need to be Netflix or Amazon to build software at scale, or require high availability.
sofixa 1y ago
> Plenty of heavy traffic, high redundancy applications exist without the need for AWS (or any other cloud providers) overpriced "bespoke" systems. And almost all of them need a database, a load balancer, maybe some sort of cache. AWS has got you covered. Maybe some of them need some async periodic reporting tasks. Or to store massive files or datasets and do analysis on them. Or transcode video. Or transform images. Or run another type of database for a third party piece of software. Or run a queue for something. Or capture logs or metrics. And on and on and and on. AWS has got you covered. This is Excel all over again. "Excel is too complex and has too many features, nobody needs more than 20% of Excel. It's just that everyone needs a different 20%".
- esskay 1y ago
  You're right AWS does have you covered. But that doesn't mean thats the only way of doing it. Load balancing is insanely easy to do yourself, databases even easier. Caching, ditto. I think a few people who claim to be in devops could do with learning the basics about how things like Ansible can help them as there's a fair few people who seem to be under the impression AWS is the only, and the best option, which unless you're FAANG really is rarely the case.
  - sofixa 1y ago
    You can spin up a redundant database setup with backups and monitoring and automatic fail over in 10 mins (the time it takes in AWS)? And maintain it? If you've done this a few times before and have it highly automated, sure. But let's not pretend it's "even easier" than "insanely easy". Load balancing is trivial unless you get into global multicast LBs, but AWS have you covered there too.
  - ta1243 1y ago
    You could never run a site like hacker news on a single box somewhere with a backup box a couple of states away. (/s, obviously)
    - sofixa 1y ago
motorest 1y ago
> Are you Netflix? Because is not theres a 99% probability you dont need any of those AWS services and just have a severe case of shiny object syndrome in your organisation. I think you don't even understand the issue you are commenting on. It's irrelevant if you are Netflix or some guy playing with a tutorial. One of the key traits of serverless offerings is how it eliminates the need to manage and maintain a service or even worry about you have enough computational resources. You click a button to provision everything, you configure your clients to consume that service, and you are done. If you stop to think about the amount of work you need to invest to even arrive at a point where you can actually point a client at a service, you'll be looking at what the value of serverless offerings. Ironically, it's the likes of Netflix who can put together a case against using serverless offerings. They can afford to have their own teams managing their own platform services with the service levels they are willing to afford. For everyone else, unless you are in the business of managing and tuning databases or you are heavily motivated to save pocket change on a cloud provider bill, the decision process is neither that clear not favours running your own services.

by esskay 1y ago

rafaelmn 1y ago

To be honest I don't trust myself running a HA PostgreSQL setup with correct backups without spending an exorbitant effort to investigate everything (weeks/months) - do you ? I'm not even sure what effort that would take. I can't remember last time I worked with unmanaged DB in prod where I did not have a dedicated DBA/sysadmin. And I've been doing this for 15 years now. AFAIK Hetzner offers no managed database solution. I know they offer some load balancer so there's that at least. At some point in the scaling journey bare metal might be the right choice, but I get the feeling a lot of people here trivialize it.

cdelsolar 1y ago
If youre not Netflix then just sudo yum install postgresql and pg_dump every day, upload to S3. Has worked for me for 20 years at various companies, side projects, startups …
- rafaelmn 1y ago
  What happens when the server goes down ? How do you update it ?
  - cdelsolar 1y ago
    you stand up another db server and load the last good dump into it i suppose
- motorest 1y ago
  > If youre not Netflix then just sudo yum install postgresql and pg_dump every day, upload to S3. database services such as DynamoDB support a few backup strategies out of the box, including continuous backups. You just need to flips switch and never bother about it again. > Has worked for me for 20 years at various companies, side projects, startups … That's perfectly fine. There are still developers who don't even use version control at all. Some old habits die hard, even when the whole world moved on.
esskay 1y ago
If it requires weeks/months to sort setting that up and backups then you need a news ops person as that's insane. If you're doing it yourself, learn Ansible, you'll do it once and be set forever. You do not need "managed" database services. A managed database is no different from apt install postgesql followed by a scheduled backup. It genuinely is trivial, people seem to have this impression theres some sort of unique special sauce going on at AWS when there really isn't.
- rafaelmn 1y ago
  But that's the thing - if I have an ops guy who can cover this then sure it makes sense - but who does at an early stage ? As a semi competent dev I can setup a terraform infra and be relatively safe with RDS. I could maybe figure out how to do it on my own in some time - but I don't know what I don't know - and I don't want to do a weekend production DB outage debugging because I messed up the replication setup or something. Maybe I'm getting old but I just don't have the energy to deal with that :)
- bostik 1y ago
  Backups with wall-g and recurring pg_dump are indeed trivial. (Modulo S3 outage taking so long that your WAL files fill up the disk and you corrupt the entire database.) It's the HA part, especially with a high-volume DB that's challenging.
- jameshart 1y ago
  That doesn’t give you high availability; it doesn’t give you monitoring and alerting; it doesn’t give you hardware failure detection and replacement; it doesn’t solve access control or networking… Managed databases are a lot more than apt install postgresql.
- byryan 1y ago
  If you're doing it yourself, learn Ansible, you'll do it once and be set forever. You do not need "managed" database services. A managed database is no different from apt install postgesql followed by a scheduled backup. Genuinely no disrespect, but these statements really make it seem like you have limited experience building an HA scalable system. And no, you don't need to be Netflix or Amazon to build software at scale, or require high availability.

sofixa 1y ago

> Plenty of heavy traffic, high redundancy applications exist without the need for AWS (or any other cloud providers) overpriced "bespoke" systems. And almost all of them need a database, a load balancer, maybe some sort of cache. AWS has got you covered. Maybe some of them need some async periodic reporting tasks. Or to store massive files or datasets and do analysis on them. Or transcode video. Or transform images. Or run another type of database for a third party piece of software. Or run a queue for something. Or capture logs or metrics. And on and on and and on. AWS has got you covered. This is Excel all over again. "Excel is too complex and has too many features, nobody needs more than 20% of Excel. It's just that everyone needs a different 20%".

esskay 1y ago
You're right AWS does have you covered. But that doesn't mean thats the only way of doing it. Load balancing is insanely easy to do yourself, databases even easier. Caching, ditto. I think a few people who claim to be in devops could do with learning the basics about how things like Ansible can help them as there's a fair few people who seem to be under the impression AWS is the only, and the best option, which unless you're FAANG really is rarely the case.
- sofixa 1y ago
  You can spin up a redundant database setup with backups and monitoring and automatic fail over in 10 mins (the time it takes in AWS)? And maintain it? If you've done this a few times before and have it highly automated, sure. But let's not pretend it's "even easier" than "insanely easy". Load balancing is trivial unless you get into global multicast LBs, but AWS have you covered there too.
- ta1243 1y ago
  You could never run a site like hacker news on a single box somewhere with a backup box a couple of states away. (/s, obviously)
  - sofixa 1y ago

motorest 1y ago

> Are you Netflix? Because is not theres a 99% probability you dont need any of those AWS services and just have a severe case of shiny object syndrome in your organisation. I think you don't even understand the issue you are commenting on. It's irrelevant if you are Netflix or some guy playing with a tutorial. One of the key traits of serverless offerings is how it eliminates the need to manage and maintain a service or even worry about you have enough computational resources. You click a button to provision everything, you configure your clients to consume that service, and you are done. If you stop to think about the amount of work you need to invest to even arrive at a point where you can actually point a client at a service, you'll be looking at what the value of serverless offerings. Ironically, it's the likes of Netflix who can put together a case against using serverless offerings. They can afford to have their own teams managing their own platform services with the service levels they are willing to afford. For everyone else, unless you are in the business of managing and tuning databases or you are heavily motivated to save pocket change on a cloud provider bill, the decision process is neither that clear not favours running your own services.