5 ms·

I do laugh at people that say "Using the cloud means less downtime than your own server", then things like this come along :D

HatchedLake721 5y ago
It’s still less downtime than your own server and you have hundreds of engineers 24/7 ready to diagnose and fix issues
- iso1210 5y ago
  Really, downtime of google so far this year: More than 0 minutes. Downtime of my own server: 0 minutes. Of course I'm not trying to run a massively scalable service coping with millions of customers, because I don't need that.
  - mekkkkkk 5y ago
    You think that's how statistics works? Obviously it's possible to keep your own server running with 0 downtime. But you are exposed to a much higher risk of severe downtimes longer than the cloud provider would likely be. Be it hardware failure, grid, ISP, whatever.
    - iso1210 5y ago
      Claim was "It’s still less downtime than your own server" Claim disproven
      - mekkkkkk 5y ago
        That claim was a direct snippet from your original comment. So unless you meant that you laugh at people who are talking specifically about your personal server, I'd say you are being selectively literal.
        iso1210 5y ago
        Cloud kool-aid drinkers say "The cloud is always better" I say that in many cases your own server is better. My company runs its own on-prem confluence, it's been taken down for updates on a regular basis at a known maintenence time. That's far better than losing it because a cloud based one was hosted on GCP this morning when we actually need the data Obviously in many other cases the cloud is better. You wouldn't want to server a million cusotmers across the world from a single server in your own basement. That's not the only model.
        mekkkkkk 5y ago
        I agree completely with that added nuance. Cloud infra is definitely overused and is sometimes seen as the only option these days, even for the simplest of projects. Having your own metal is often times much more convenient and cheaper. Plus it's much more enjoyable to work with. My only gripe was with your gross over-simplification that read like "hurr durr, my server hasn't gone down this year, so self-hosting has better uptime than Google". It's such an unnecessary and baseless argument.
        iso1210 5y ago
        Vast vast majority of arguments you hear on HN is that it's impossible to run your own equipment. That's as ridiculous as saying you could run dropbox off a raspberry pi, but tends to get pushed to the top, and the schadenfreude when these events inevitability come along is too good to miss. Every few months another major outage of another cloud provider hits the headlines, meanwhile millions of small companies have no problem with the uptime of their 'legacy' services. I was at a farm a few weeks ago, the farmer had a server in a closet. It did break on occasion, when there was a power outage. His desktop and internet broke too, so what would the point in his server working. If it was hosted on google it wouldn't have been working this morning, despite his computer being fine. If you build your business processes around accepting failure, it's not a problem. It's far easier to keep at least one out of 3 machines online for 99.999% of the time than to keep a single service running for the same time.
    - remram 5y ago
      To be fair, this is exactly how statistics work. The average downtime of self-hosted servers might be higher (arguably) but many people are under Google's yearly downtime (aka variance is high).
- tester34 5y ago
  >It’s still less downtime than your own server what makes you think so, actually?
  - michaelt 5y ago
    In my experience, it's very easy to achieve high uptime through luck. If I only run a single server, and it has a 10% chance of failing in any given year, I have a 90% chance of achieving 100% uptime in a given year. In my experience it's also very easy to think you've got all your bases covered when actually you haven't. I'm protected against mains power failures by a UPS and a generator - but a UPS switchgear fault can cut my power even without a mains power outage. My server has dual power supplies and network cards - but that won't help me if a clumsy worker sent to replace the server above mine unplugs mine by mistake. And so on. If I think I'm doing a better job than a billion-dollar corporation with hundreds of thousands of servers, does that mean I am? Or is it more likely I'm fooling myself?
    - lostmsu 5y ago
      Do you have some external ping test running continuously? If not, you really have no idea.
      - iso1210 5y ago
        Of course, how else would I know the exact failure times, my service monitoring only polls every few minutes. That doesn't help me with my knowing how my service is performing though, I'm not offering a ping service. More importantly, I do know I had an error loading twitter at Thu 11 Nov 04:17:01 GMT 2021, however my websites (and google and hackernews) were working. At 17:53:01 GMT Twitter took over 2 seconds to load it's first http page, far beyond the normal 400ms. Google took 23 seconds to load this morning, BBC News just 0.071. On the other hand I also need to provide services which can't cope with outages measured in milliseconds. Good luck with complaining to a cloud provider that your traffic vanished for 4 seconds. Those services thus have multiple connections on independent hardware and circuits with no single point of failure
    - 5y ago
- southerntofu 5y ago
  This is so misinformed by industry propaganda. Modern cloud services are very often unavailable from a region without any apparent reasons. Or the service appears to be available but some specific feature doesn't work. Or it's available but just really slow or dropping packets. When you have a "simple" (already quite complex) BGP(TCP(HTTP)) tunnel, chances are things just work and it's easy enough to diagnose issues. Between anycast, auto-scaling, and WAFs (among others) you have added so many layers of complexity that the chance of a "random" error somewhere in the stack has dramatically increased and diagnosis is close to impossible. It used to be simple that web services are either up or down. Now it's not so obvious anymore, and that's definitely not being reflected in the 5 9's SLAs falsely promised by cloud providers. I can say with confidence most selfhosted systems i've been close to have much better uptime than modern cloud services and are much cheaper to service and maintain.
  - exikyut 5y ago
    I agree with specific dimensions of this sentiment. Firstly, I think that the GP comment ever so slightly misapplies the localized awareness available in smaller environments to the cloud. Yes, there are tons more engineers to look at stuff, but those engineers are a) already bogged down keeping up with internal infra, politics and bureaucratic machinery, and b) quite some distance further away from actual errors occurring on the ground because everything's aggregated to the hilt so the stats remain comprehensible at the higher scale. I also agree that the newer stuff that is less mature has an order of magnitude more intrinsic leaky abstractions than old designs, which I do think were more hygienic and espoused more effective separation of concerns than what is used today. Also, not to nitpick, but the "classical" canonical definition would probably look closer to BGP(TCP(TLSv1.3(HTTP/1.1))), with the modern equivalent being BGP(UDP(HTTP2)) and the future being BGP(UDP(QUIC)). I do agree that the rapid consolidation of HTTP and TLS, without wide-scale awareness and slow, methodological development of general introspection tooling, does make things net worse in general. I suspect infra will still be using HTTP/1.1 for a long time into the future until this materially changes.
i5heu 5y ago
Right? My private website on my RPI is running now 2 Years without a problem and only minimal downtime due to rebooting for the new kernels. It is amazing how much uptime you can achieve with a 5$ Computer in comparison to a 1730000000000$ (1,73 tera $) Company. Even if you compensate for dynamic content.
- exikyut 5y ago
  (Google is currently processing multiple terabits of traffic per second using several billion dollars' worth of distributed infrastructure.)
  - Nextgrid 5y ago
    And yet it doesn't mean his RPi-powered website needs to ever sustain that kind of traffic, so he doesn't have to take the additional risk of running a distributed system. In contrast, in the cloud you typically take on the risk of the underlying distributed platform (optimized for managing thousands of VMs, etc) even though you only need a single machine for yourself.
  - iso1210 5y ago
    Actually it's not, and that's the problem
    - exikyut 5y ago
      FWIW I vaguely recall some kind of website that showed approximate bandwidth use... it was something like a cross between statcounter and alexa except for network stats and the like. I think it purported Google to be hovering around 15Gbps. That sounded humongously wrong, like I'd expect global traffic <-> Google to at least be a couple terabits, right? I'd be very curious if there's a way to actually ballpark the number. Or maybe it is in fact possible to implement services that reliably track this sort of thing, and my futile searches earlier just weren't finding them...
  - i5heu 5y ago
    Yeah but google has also 27,169 software engineers.

HatchedLake721 5y ago

It’s still less downtime than your own server and you have hundreds of engineers 24/7 ready to diagnose and fix issues

iso1210 5y ago
Really, downtime of google so far this year: More than 0 minutes. Downtime of my own server: 0 minutes. Of course I'm not trying to run a massively scalable service coping with millions of customers, because I don't need that.
- mekkkkkk 5y ago
  You think that's how statistics works? Obviously it's possible to keep your own server running with 0 downtime. But you are exposed to a much higher risk of severe downtimes longer than the cloud provider would likely be. Be it hardware failure, grid, ISP, whatever.
  - iso1210 5y ago
    Claim was "It’s still less downtime than your own server" Claim disproven
    - mekkkkkk 5y ago
      That claim was a direct snippet from your original comment. So unless you meant that you laugh at people who are talking specifically about your personal server, I'd say you are being selectively literal.
      - iso1210 5y ago
        Cloud kool-aid drinkers say "The cloud is always better" I say that in many cases your own server is better. My company runs its own on-prem confluence, it's been taken down for updates on a regular basis at a known maintenence time. That's far better than losing it because a cloud based one was hosted on GCP this morning when we actually need the data Obviously in many other cases the cloud is better. You wouldn't want to server a million cusotmers across the world from a single server in your own basement. That's not the only model.
        mekkkkkk 5y ago
        I agree completely with that added nuance. Cloud infra is definitely overused and is sometimes seen as the only option these days, even for the simplest of projects. Having your own metal is often times much more convenient and cheaper. Plus it's much more enjoyable to work with. My only gripe was with your gross over-simplification that read like "hurr durr, my server hasn't gone down this year, so self-hosting has better uptime than Google". It's such an unnecessary and baseless argument.
        iso1210 5y ago
        Vast vast majority of arguments you hear on HN is that it's impossible to run your own equipment. That's as ridiculous as saying you could run dropbox off a raspberry pi, but tends to get pushed to the top, and the schadenfreude when these events inevitability come along is too good to miss. Every few months another major outage of another cloud provider hits the headlines, meanwhile millions of small companies have no problem with the uptime of their 'legacy' services. I was at a farm a few weeks ago, the farmer had a server in a closet. It did break on occasion, when there was a power outage. His desktop and internet broke too, so what would the point in his server working. If it was hosted on google it wouldn't have been working this morning, despite his computer being fine. If you build your business processes around accepting failure, it's not a problem. It's far easier to keep at least one out of 3 machines online for 99.999% of the time than to keep a single service running for the same time.
  - remram 5y ago
    To be fair, this is exactly how statistics work. The average downtime of self-hosted servers might be higher (arguably) but many people are under Google's yearly downtime (aka variance is high).
tester34 5y ago
>It’s still less downtime than your own server what makes you think so, actually?
- michaelt 5y ago
  In my experience, it's very easy to achieve high uptime through luck. If I only run a single server, and it has a 10% chance of failing in any given year, I have a 90% chance of achieving 100% uptime in a given year. In my experience it's also very easy to think you've got all your bases covered when actually you haven't. I'm protected against mains power failures by a UPS and a generator - but a UPS switchgear fault can cut my power even without a mains power outage. My server has dual power supplies and network cards - but that won't help me if a clumsy worker sent to replace the server above mine unplugs mine by mistake. And so on. If I think I'm doing a better job than a billion-dollar corporation with hundreds of thousands of servers, does that mean I am? Or is it more likely I'm fooling myself?
  - lostmsu 5y ago
    Do you have some external ping test running continuously? If not, you really have no idea.
    - iso1210 5y ago
      Of course, how else would I know the exact failure times, my service monitoring only polls every few minutes. That doesn't help me with my knowing how my service is performing though, I'm not offering a ping service. More importantly, I do know I had an error loading twitter at Thu 11 Nov 04:17:01 GMT 2021, however my websites (and google and hackernews) were working. At 17:53:01 GMT Twitter took over 2 seconds to load it's first http page, far beyond the normal 400ms. Google took 23 seconds to load this morning, BBC News just 0.071. On the other hand I also need to provide services which can't cope with outages measured in milliseconds. Good luck with complaining to a cloud provider that your traffic vanished for 4 seconds. Those services thus have multiple connections on independent hardware and circuits with no single point of failure
  - 5y ago
southerntofu 5y ago
This is so misinformed by industry propaganda. Modern cloud services are very often unavailable from a region without any apparent reasons. Or the service appears to be available but some specific feature doesn't work. Or it's available but just really slow or dropping packets. When you have a "simple" (already quite complex) BGP(TCP(HTTP)) tunnel, chances are things just work and it's easy enough to diagnose issues. Between anycast, auto-scaling, and WAFs (among others) you have added so many layers of complexity that the chance of a "random" error somewhere in the stack has dramatically increased and diagnosis is close to impossible. It used to be simple that web services are either up or down. Now it's not so obvious anymore, and that's definitely not being reflected in the 5 9's SLAs falsely promised by cloud providers. I can say with confidence most selfhosted systems i've been close to have much better uptime than modern cloud services and are much cheaper to service and maintain.
- exikyut 5y ago
  I agree with specific dimensions of this sentiment. Firstly, I think that the GP comment ever so slightly misapplies the localized awareness available in smaller environments to the cloud. Yes, there are tons more engineers to look at stuff, but those engineers are a) already bogged down keeping up with internal infra, politics and bureaucratic machinery, and b) quite some distance further away from actual errors occurring on the ground because everything's aggregated to the hilt so the stats remain comprehensible at the higher scale. I also agree that the newer stuff that is less mature has an order of magnitude more intrinsic leaky abstractions than old designs, which I do think were more hygienic and espoused more effective separation of concerns than what is used today. Also, not to nitpick, but the "classical" canonical definition would probably look closer to BGP(TCP(TLSv1.3(HTTP/1.1))), with the modern equivalent being BGP(UDP(HTTP2)) and the future being BGP(UDP(QUIC)). I do agree that the rapid consolidation of HTTP and TLS, without wide-scale awareness and slow, methodological development of general introspection tooling, does make things net worse in general. I suspect infra will still be using HTTP/1.1 for a long time into the future until this materially changes.

i5heu 5y ago

Right? My private website on my RPI is running now 2 Years without a problem and only minimal downtime due to rebooting for the new kernels. It is amazing how much uptime you can achieve with a 5$ Computer in comparison to a 1730000000000$ (1,73 tera $) Company. Even if you compensate for dynamic content.

exikyut 5y ago
(Google is currently processing multiple terabits of traffic per second using several billion dollars' worth of distributed infrastructure.)
- Nextgrid 5y ago
  And yet it doesn't mean his RPi-powered website needs to ever sustain that kind of traffic, so he doesn't have to take the additional risk of running a distributed system. In contrast, in the cloud you typically take on the risk of the underlying distributed platform (optimized for managing thousands of VMs, etc) even though you only need a single machine for yourself.
- iso1210 5y ago
  Actually it's not, and that's the problem
  - exikyut 5y ago
    FWIW I vaguely recall some kind of website that showed approximate bandwidth use... it was something like a cross between statcounter and alexa except for network stats and the like. I think it purported Google to be hovering around 15Gbps. That sounded humongously wrong, like I'd expect global traffic <-> Google to at least be a couple terabits, right? I'd be very curious if there's a way to actually ballpark the number. Or maybe it is in fact possible to implement services that reliably track this sort of thing, and my futile searches earlier just weren't finding them...
- i5heu 5y ago
  Yeah but google has also 27,169 software engineers.