5 ms·

Self-hosting is more a question of responsibility I'd say. I am running a couple of SaaS products and self-host at much better performance at a fraction of the

Self-hosting is more a question of responsibility I'd say. I am running a couple of SaaS products and self-host at much better performance at a fraction of the cost of running this on AWS. It's amazing and it works perfectly fine. For client projects, however, I always try and sell them on paying the AWS fees, simply because it shifts the responsibility of the hardware being "up" to someone else. It does not inherently solve the downtime problem, but it allows me to say, "we'll have to wait until they've sorted this out, Ikea and Disney are down, too." Doesn't always work like that and isn't always a tried-and-true excuse, but generally lets me sleep much better at night. With limited budgets, however, it's hard to accept the cost of RDS (and we're talking with at least one staging environment) when comparing it to a very tight 3-node Galera cluster running on Hetzner at barely a couple of bucks a month. Or Cloudflare, titan at the front, being down again today and the past two days (intermittently) after also being down a few weeks ago and earlier this year as well. Also had SQS queues time out several times this week, they picked up again shortly, but it's not like those things ...never happen on managed environments. They happen quite a bit.

bossyTeacher 10mo ago
> Self-hosting is more a question of responsibility I'd say. I am running a couple of SaaS products and self-host at much better performance at a fraction of the cost of running this on AWS It is. You need to answer the question: what are the consecuences of your service being down for lets say 4 hours or some security patch isn't properly applied or you have not followed the best practices in terms of security? Many people are technically unable, lack the time or the resources to be able to confidently address that question, hence paying for someone else to do it. Your time is money though. You are saving money but giving up time. Like everything, it is always cheaper to do it (it being cooking at home, cleaning your home, fixing your own car, etc) yourself (if you don't include the cost of your own time doing the service you normally pay someone else for).
- jbverschoor 10mo ago
  Yea I agree.. better outsource product development, management, and everything else too by that narrative
  - zbentley 10mo ago
    That’s pretty reductive. By that logic the opposite extreme is just as true: if using managed services is just as bad as outsourcing everything else, then a business shouldn’t rent real estate either—every business should build and own their own facility. They should also never contract out janitorial work, nor should they retain outside law firms—they should hire and staff those departments internally, every time, no nuance allowed. You see the issue? Like, I’m all for not procuring things that it makes more sense to own/build (and I know most businesses have piss-poor instincts on which is which—hell, I work for the government! I can see firsthand the consequences of outsourcing decision making to contractors, rather than just outsourcing implementation). But it’s very case-by-case. There’s no general rule like “always prefer self hosting” or “always rent real estate, never buy” that applies broadly enough to be useful.
    - jama211 10mo ago
      So well said, I like the technique of taking their logic and turning it around, never seen that before but it’s smart.
      - antihipocrat 10mo ago
        In my experience it only ends well on the Internet and with philosophically inclined friends.
        jama211 10mo ago
        Anything ending well on the internet is like a mythical unicorn though
    - gopher_space 10mo ago
      I'll be reductive in conversations like this just to help push the pendulum back a little. The prevailing attitude seems (to me) like people find self-hosting mystical and occult, yet there's never been a better time to do it. > But it’s very case-by-case. There’s no general rule like “always prefer self hosting” or “always rent real estate, never buy” that applies broadly enough to be useful. I don't know if anyone remembers that irritating "geek code" thing we were doing a while back, but coming up with some kind of shorthand for whatever context we're talking about would be useful.
- bigstrat2003 10mo ago
  > Like everything, it is always cheaper to do it (it being cooking at home, cleaning your home, fixing your own car, etc) yourself (if you don't include the cost of your own time doing the service you normally pay someone else for). In a business context the "time is money" thing actually makes sense, because there's a reasonable likelihood that the business can put the time to a more profitable use in some other way. But in a personal context it makes no sense at all. Realistically, the time I spend cooking or cleaning was not going to earn me a dime no matter what else I did, therefore the opportunity cost is zero. And this is true for almost everyone out there.
  - _superposition_ 10mo ago
    Lol this made me laugh, there's a reasonable likelihood that time will be filled with meetings.
    - bigstrat2003 10mo ago
      Heh, true. Although in fairness I said the business can repurpose the time to make money, not that they will. I'm splitting hairs, but it seems in keeping with the ethos here. ;)
- PunchyHamster 10mo ago
  You can pay someone else to manage your hardware stack, there are literal companies that will just keep it running, while you just deploy your apps on that. > It is. You need to answer the question: what are the consecuences of your service being down for lets say 4 hours or some security patch isn't properly applied or you have not followed the best practices in terms of security? There is one advantage self hosted setup has here, if you set up VPN, only your employees have access, and you can have server not accessible from the internet. So even in case of zero day that WILL make SaaS company leak your data, you can be safe(r) with self-hosted solution. > Your time is money though. You are saving money but giving up time. The investment compounds. Setting up infra to run a single container for some app takes time and there is good chance it won't pay back for itself. But 2nd service ? Cheaper. 5th ? At that point you probably had it automated enough that it's just pointing it at docker container and tweaking few settings. > Like everything, it is always cheaper to do it (it being cooking at home, cleaning your home, fixing your own car, etc) yourself (if you don't include the cost of your own time doing the service you normally pay someone else for). It's cheaper if you include your own time. You pay a technical person at your company to do it. Saas company does that, then pays sales and PR person to sell it, then pays income tax to it, then it also needs to "pay" investors. Yeah making a service for 4 people in company can be more work than just paying $10/mo to SaaS company. But 20 ? 50 ? 100 ? It quickly gets to point where self hosting (whether actually "self" or by using dedicated servers, or by using cloud) actually pays off
mattmanser 10mo ago
Over 20 year I've had lots of clients on self-hosted, even self-hosting SQL on the same VM as the webserver as you used to in the long distant past for low-usage web apps. I have never, ever, ever had a SQL box go down. I've had a web server go down once. I had someone who probably shouldn't have had access to a server accidentally turn one off once. The only major outage I've had (2/3 hours) was when the box was also self-hosting an email server and I accidentally caused it to flood itself with failed delivery notices with a deploy. I may have cried a little in frustration and panic but it got fixed in the end. I actually find using cloud hosted SQL in some ways harder and more complicated because it's such a confusing mess of cost and what you're actually getting. The only big complication is setting up backups, and that's a one-off task.
- paulryanrogers 10mo ago
  Disks go bad. RAID is nontrivial to set up. Hetzner had a big DC outage that lead to data loss. Off site backups or replication would help, though not always trivial to fail over.
  - alemanek 10mo ago
    As someone who has set this up while not being a DBA or sysadmin. Replication and backups really aren’t that difficult to setup properly with something like Postgres. You can also expose metrics around this to setup alerting if replication lag goes beyond a threshold you set or a backup didn’t complete. You do need to periodically test your backups but that is also good practice. I am not saying something like RDS doesn’t have value but you are paying a huge premium for it. Once you get to more steady state owning your database totally makes sense. A cluster of $10-20 VPSes with NVMe drives can get really good performance and will take you a lot farther than you might expect.
    - andersmurphy 10mo ago
      Even easier with sqlite thanks to litestream.
vb-8448 10mo ago
You can still outsource up to VM level and handle everything else on you own. Obviously it depends on the operational overhead of specific technology.
madeofpalk 10mo ago
> but it allows me to say, "we'll have to wait until they've sorted this out, Ikea and Disney are down, too." From my experience your client’s clients don’t care about this when they’re still otherwise up.
- tjwebbnorfolk 10mo ago
  Yes but the fact that it's "not their fault" keeps the person from getting fired. Don't underestimate the power of CYA
  - HPsquared 10mo ago
    That's real microeconomics.
  - api 10mo ago
    This is a major reason the cloud commands such a premium. It’s a way to make down time someone else’s problem. The other factor is eliminating the “one guy who knows X” problem in IT. What happens if that person leaves or you have to let them go? But with managed infrastructure there’s a pool of people who know how to write terraform or click buttons and manage it and those are more interchangeable than someone’s DIY deployment. Worst case the cloud provider might sell you premium support and help. Might be expensive but you’re not down. Lastly, there’s been an exodus of talent from IT. The problem is that anyone really good can become a coder and make more. So finding IT people at a reasonable cost who know how to really troubleshoot and root cause stuff and engineer good systems is very hard. The good ones command more of a programmer salary which makes the gap with cloud costs much smaller. Might as well just go managed cloud.
    - 01HNNWZ0MV43FF 10mo ago
      That is called "bus factor" or "lottery factor". If the one IT guy gets hit by a bus or wins the lottery and quits, what happens? You want a bus factor of two or more - Two people would have to get hit by a bus for the company to have a big problem
Thaxll 10mo ago
That argument does not hold when there is aws serverless pg available, which cost almost nothing for low traffic and is vastly superior to self hosting regarding observability, security, integration, backup ect... There is no reason to self manage pg for dev / environnement. https://aws.amazon.com/rds/aurora/serverless/ https://aws.amazon.com/rds/aurora/serverless/
- jread 10mo ago
  This was true for RDS serverless v1 which scaled to 0 but is no longer offered. V2 requires a minimum 0.5 ACU hourly commit ($40+ /mo).
  - cobolcomesback 10mo ago
    V2 scales to zero as of last year. https://aws.amazon.com/blogs/database/introducing-scaling-to-0-capacity-with-amazon-aurora-serverless-v2/ https://aws.amazon.com/blogs/database/introducing-scaling-to... It only scales down after a period of inactivity though - it’s not pay-per-request like other serverless offerings. DSQL looks to be more cost effective for small projects if you can deal with the deviations from Postgres.
    - jread 10mo ago
      Ah, good to know, I hadn't seen that V2 update. Looks like a min 5m inactivity to auto-pause (i.e., scale to 0), and any connection attempt (valid or not) resumes the DB.
- starttoaster 10mo ago
  "which cost almost nothing for low traffic" you invented the retort "what about high traffic" within your own message. I don't even necessarily mean user traffic either. But if you constantly have to sync new records over (as could be the case in any kind of timeseries use-case) the internal traffic could rack up costs quickly. "vastly superior to self hosting regarding observability" I'd suggest looking into the cnpg operator for Postgres on Kubernetes. The builtin metrics and official dashboard is vastly superior to what I get from Cloudwatch for my RDS clusters. And the backup mechanism using Barman for database snapshots and WAL backups is vastly superior to AWS DMS or AWS's disk snapshots which aren't portable to a system outside of AWS if you care about avoiding vendor lock-in.
arwhatever 10mo ago
Me: “Why are we switching from NoNameCMS to Salesforce?” Savvy Manager: “NoNameCMS often won’t take our support calls, but if Salesforce goes down it’s in the WSJ the next day.”
- dilyevsky 10mo ago
  This ignores the case when BigVendor is down for your account and your account only and support is mia, which is not that uncommon ime
  - yunwal 10mo ago
    It doesn’t ignore that case, it simply allows them to shift blame whereas the no name vendor does not.
    - ajmurmann 10mo ago
      "Nobody has ever been fired for buying IBM"
      - psychoslave 10mo ago
        https://www.forbes.com/sites/duenablomstrom1/2018/11/30/nobody-gets-fired-for-buying-ibm-but-they-should/ https://www.forbes.com/sites/duenablomstrom1/2018/11/30/nobo...
    - zelphirkalt 10mo ago
      So in the end it's not better for the users at all, it's just for non-technical people to shift blame. Great "business reasoning".
      - WJW 10mo ago
        Nobody in this thread ever claimed it was better for the users. It's better for the people involved in the decision.

bossyTeacher 10mo ago

> Self-hosting is more a question of responsibility I'd say. I am running a couple of SaaS products and self-host at much better performance at a fraction of the cost of running this on AWS It is. You need to answer the question: what are the consecuences of your service being down for lets say 4 hours or some security patch isn't properly applied or you have not followed the best practices in terms of security? Many people are technically unable, lack the time or the resources to be able to confidently address that question, hence paying for someone else to do it. Your time is money though. You are saving money but giving up time. Like everything, it is always cheaper to do it (it being cooking at home, cleaning your home, fixing your own car, etc) yourself (if you don't include the cost of your own time doing the service you normally pay someone else for).

jbverschoor 10mo ago
Yea I agree.. better outsource product development, management, and everything else too by that narrative
- zbentley 10mo ago
  That’s pretty reductive. By that logic the opposite extreme is just as true: if using managed services is just as bad as outsourcing everything else, then a business shouldn’t rent real estate either—every business should build and own their own facility. They should also never contract out janitorial work, nor should they retain outside law firms—they should hire and staff those departments internally, every time, no nuance allowed. You see the issue? Like, I’m all for not procuring things that it makes more sense to own/build (and I know most businesses have piss-poor instincts on which is which—hell, I work for the government! I can see firsthand the consequences of outsourcing decision making to contractors, rather than just outsourcing implementation). But it’s very case-by-case. There’s no general rule like “always prefer self hosting” or “always rent real estate, never buy” that applies broadly enough to be useful.
  - jama211 10mo ago
    So well said, I like the technique of taking their logic and turning it around, never seen that before but it’s smart.
    - antihipocrat 10mo ago
      In my experience it only ends well on the Internet and with philosophically inclined friends.
      - jama211 10mo ago
        Anything ending well on the internet is like a mythical unicorn though
  - gopher_space 10mo ago
    I'll be reductive in conversations like this just to help push the pendulum back a little. The prevailing attitude seems (to me) like people find self-hosting mystical and occult, yet there's never been a better time to do it. > But it’s very case-by-case. There’s no general rule like “always prefer self hosting” or “always rent real estate, never buy” that applies broadly enough to be useful. I don't know if anyone remembers that irritating "geek code" thing we were doing a while back, but coming up with some kind of shorthand for whatever context we're talking about would be useful.
bigstrat2003 10mo ago
> Like everything, it is always cheaper to do it (it being cooking at home, cleaning your home, fixing your own car, etc) yourself (if you don't include the cost of your own time doing the service you normally pay someone else for). In a business context the "time is money" thing actually makes sense, because there's a reasonable likelihood that the business can put the time to a more profitable use in some other way. But in a personal context it makes no sense at all. Realistically, the time I spend cooking or cleaning was not going to earn me a dime no matter what else I did, therefore the opportunity cost is zero. And this is true for almost everyone out there.
- _superposition_ 10mo ago
  Lol this made me laugh, there's a reasonable likelihood that time will be filled with meetings.
  - bigstrat2003 10mo ago
    Heh, true. Although in fairness I said the business can repurpose the time to make money, not that they will. I'm splitting hairs, but it seems in keeping with the ethos here. ;)
PunchyHamster 10mo ago
You can pay someone else to manage your hardware stack, there are literal companies that will just keep it running, while you just deploy your apps on that. > It is. You need to answer the question: what are the consecuences of your service being down for lets say 4 hours or some security patch isn't properly applied or you have not followed the best practices in terms of security? There is one advantage self hosted setup has here, if you set up VPN, only your employees have access, and you can have server not accessible from the internet. So even in case of zero day that WILL make SaaS company leak your data, you can be safe(r) with self-hosted solution. > Your time is money though. You are saving money but giving up time. The investment compounds. Setting up infra to run a single container for some app takes time and there is good chance it won't pay back for itself. But 2nd service ? Cheaper. 5th ? At that point you probably had it automated enough that it's just pointing it at docker container and tweaking few settings. > Like everything, it is always cheaper to do it (it being cooking at home, cleaning your home, fixing your own car, etc) yourself (if you don't include the cost of your own time doing the service you normally pay someone else for). It's cheaper if you include your own time. You pay a technical person at your company to do it. Saas company does that, then pays sales and PR person to sell it, then pays income tax to it, then it also needs to "pay" investors. Yeah making a service for 4 people in company can be more work than just paying $10/mo to SaaS company. But 20 ? 50 ? 100 ? It quickly gets to point where self hosting (whether actually "self" or by using dedicated servers, or by using cloud) actually pays off

mattmanser 10mo ago

Over 20 year I've had lots of clients on self-hosted, even self-hosting SQL on the same VM as the webserver as you used to in the long distant past for low-usage web apps. I have never, ever, ever had a SQL box go down. I've had a web server go down once. I had someone who probably shouldn't have had access to a server accidentally turn one off once. The only major outage I've had (2/3 hours) was when the box was also self-hosting an email server and I accidentally caused it to flood itself with failed delivery notices with a deploy. I may have cried a little in frustration and panic but it got fixed in the end. I actually find using cloud hosted SQL in some ways harder and more complicated because it's such a confusing mess of cost and what you're actually getting. The only big complication is setting up backups, and that's a one-off task.

paulryanrogers 10mo ago
Disks go bad. RAID is nontrivial to set up. Hetzner had a big DC outage that lead to data loss. Off site backups or replication would help, though not always trivial to fail over.
- alemanek 10mo ago
  As someone who has set this up while not being a DBA or sysadmin. Replication and backups really aren’t that difficult to setup properly with something like Postgres. You can also expose metrics around this to setup alerting if replication lag goes beyond a threshold you set or a backup didn’t complete. You do need to periodically test your backups but that is also good practice. I am not saying something like RDS doesn’t have value but you are paying a huge premium for it. Once you get to more steady state owning your database totally makes sense. A cluster of $10-20 VPSes with NVMe drives can get really good performance and will take you a lot farther than you might expect.
  - andersmurphy 10mo ago
    Even easier with sqlite thanks to litestream.

vb-8448 10mo ago

You can still outsource up to VM level and handle everything else on you own. Obviously it depends on the operational overhead of specific technology.

madeofpalk 10mo ago

> but it allows me to say, "we'll have to wait until they've sorted this out, Ikea and Disney are down, too." From my experience your client’s clients don’t care about this when they’re still otherwise up.

tjwebbnorfolk 10mo ago
Yes but the fact that it's "not their fault" keeps the person from getting fired. Don't underestimate the power of CYA
- HPsquared 10mo ago
  That's real microeconomics.
- api 10mo ago
  This is a major reason the cloud commands such a premium. It’s a way to make down time someone else’s problem. The other factor is eliminating the “one guy who knows X” problem in IT. What happens if that person leaves or you have to let them go? But with managed infrastructure there’s a pool of people who know how to write terraform or click buttons and manage it and those are more interchangeable than someone’s DIY deployment. Worst case the cloud provider might sell you premium support and help. Might be expensive but you’re not down. Lastly, there’s been an exodus of talent from IT. The problem is that anyone really good can become a coder and make more. So finding IT people at a reasonable cost who know how to really troubleshoot and root cause stuff and engineer good systems is very hard. The good ones command more of a programmer salary which makes the gap with cloud costs much smaller. Might as well just go managed cloud.
  - 01HNNWZ0MV43FF 10mo ago
    That is called "bus factor" or "lottery factor". If the one IT guy gets hit by a bus or wins the lottery and quits, what happens? You want a bus factor of two or more - Two people would have to get hit by a bus for the company to have a big problem

Thaxll 10mo ago

That argument does not hold when there is aws serverless pg available, which cost almost nothing for low traffic and is vastly superior to self hosting regarding observability, security, integration, backup ect... There is no reason to self manage pg for dev / environnement. https://aws.amazon.com/rds/aurora/serverless/ https://aws.amazon.com/rds/aurora/serverless/

jread 10mo ago
This was true for RDS serverless v1 which scaled to 0 but is no longer offered. V2 requires a minimum 0.5 ACU hourly commit ($40+ /mo).
- cobolcomesback 10mo ago
  V2 scales to zero as of last year. https://aws.amazon.com/blogs/database/introducing-scaling-to-0-capacity-with-amazon-aurora-serverless-v2/ https://aws.amazon.com/blogs/database/introducing-scaling-to... It only scales down after a period of inactivity though - it’s not pay-per-request like other serverless offerings. DSQL looks to be more cost effective for small projects if you can deal with the deviations from Postgres.
  - jread 10mo ago
    Ah, good to know, I hadn't seen that V2 update. Looks like a min 5m inactivity to auto-pause (i.e., scale to 0), and any connection attempt (valid or not) resumes the DB.
starttoaster 10mo ago
"which cost almost nothing for low traffic" you invented the retort "what about high traffic" within your own message. I don't even necessarily mean user traffic either. But if you constantly have to sync new records over (as could be the case in any kind of timeseries use-case) the internal traffic could rack up costs quickly. "vastly superior to self hosting regarding observability" I'd suggest looking into the cnpg operator for Postgres on Kubernetes. The builtin metrics and official dashboard is vastly superior to what I get from Cloudwatch for my RDS clusters. And the backup mechanism using Barman for database snapshots and WAL backups is vastly superior to AWS DMS or AWS's disk snapshots which aren't portable to a system outside of AWS if you care about avoiding vendor lock-in.

arwhatever 10mo ago

Me: “Why are we switching from NoNameCMS to Salesforce?” Savvy Manager: “NoNameCMS often won’t take our support calls, but if Salesforce goes down it’s in the WSJ the next day.”

dilyevsky 10mo ago
This ignores the case when BigVendor is down for your account and your account only and support is mia, which is not that uncommon ime
- yunwal 10mo ago
  It doesn’t ignore that case, it simply allows them to shift blame whereas the no name vendor does not.
  - ajmurmann 10mo ago
    "Nobody has ever been fired for buying IBM"
    - psychoslave 10mo ago
      https://www.forbes.com/sites/duenablomstrom1/2018/11/30/nobody-gets-fired-for-buying-ibm-but-they-should/ https://www.forbes.com/sites/duenablomstrom1/2018/11/30/nobo...
  - zelphirkalt 10mo ago
    So in the end it's not better for the users at all, it's just for non-technical people to shift blame. Great "business reasoning".
    - WJW 10mo ago
      Nobody in this thread ever claimed it was better for the users. It's better for the people involved in the decision.