5 ms·

Once a month is way too often. Even if your downtime requirements are "so high", once a month is ridiculous. For the sake of this example, let's assume you hav

by akulbe 12y ago

Once a month is way too often. Even if your downtime requirements are "so high", once a month is ridiculous. For the sake of this example, let's assume you have customers that depend on your server being up, for the product/service they're paying you for. Tell the customers that you plan on rebooting once a month, and see how they feel about it.

noblethrasher 12y ago
A server is just a name. Just because a host machine needs to go down doesn't mean that the "server" has to go down.
Someone1234 12y ago
In that scenario I'd never have just a single server, be it Linux, Windows, BSD, or anything else. Being in a situation where you're completely unable to restart a machine without losing money, is being in a situation where you're living on a knife edge regardless. A single hardware failure and you're in deep trouble. I've been in exactly that situation (the entire company rested on a single Ubuntu server), it was really unpleasant and management didn't do a thing about it until we started getting sector errors which the RAID controller's error handling was struggling to cope with (after that incident and the downtime, management shelled out for four physical boxes, a SAN, VMWare licenses, and a setup with 15 minute rollbacks and dynamic migrations if a physical server died).
- mreiland 12y ago
  his point remains. being forced to run 2 servers so your server doesn't go down once/month is a bit ridiculous. No, it isn't untenable, yes you can work around it, but that's a problem that shouldn't need to be worked around.
  - tempestn 12y ago
    Agreed. Up to a certain point, a single server is going to be considerably more cost effective than multiple machines, as well as requiring less devOps time. (Especially nowadays when you can easily lease a server with, say, 16 cores and 48GB of RAM.) I can recall having to restart our production server exactly once in the past 12 months for a kernel update. Every month would be an irritation for sure.
  - WorldWideWayne 12y ago
    Is it normal to update server software like PostgreSQL or Redis on Linux without stopping the running process during the update? Maybe Linux doesn't need to restart the OS as often, but you're still going to have down time, right?
    - mreiland 12y ago
      You can update the software without pulling it down. In order to use the update you need to restart, but there's a large difference between restarting a service and restarting an OS. It isn't really analogous.
      - WorldWideWayne 12y ago
        It takes 10 seconds to restart Windows these days, so it's almost analagous.
        acdha 12y ago
        Do you have any benchmarks to support that claim? I have Windows 8 and 10 systems which take minutes to reboot on an SSD which beg to differ.
        WorldWideWayne 12y ago
        You can see it when you do a video search for: Windows fast boot. Here are a couple: https://www.youtube.com/watch?v=p_vUZR2RtRs https://www.youtube.com/watch?v=p_vUZR2RtRs https://www.youtube.com/watch?v=xkKoggifsE4 https://www.youtube.com/watch?v=xkKoggifsE4 (And you don't need to log in for your background services to start.)
        acdha 12y ago
        Neither of those support your original claim. The author is focused on one specific feature, which is nice but excludes the entire shutdown process and booting up until the Windows logo appears. Anyone who's run a server knows that just the BIOS initialization is often much longer than 10 seconds. And, of course, many services take a noticeable amount of time to startup so time-to-available is longer than time-to-login-screen. Worse, the conversation was actually about the time needed to install updates. Now that Windows logs out before installing updates, you have a fairly sizable time delay – tens of minutes after a large update even on an SSD – while the service is unavailable but before the system starts rebooting. How does Fast Boot work? Well, it's actually not booting at all – it's logging out all active users and hibernating: http://blogs.msdn.com/b/b8/archive/2011/09/08/delivering-fast-boot-times-in-windows-8.aspx http://blogs.msdn.com/b/b8/archive/2011/09/08/delivering-fas... That's a really nice bit of work in normal usage but it means that you have to do the standard cold boot process when you're getting a system update. That means that a sysadmin would need to review each update to know whether it would trigger a fast or slow boot, assuming that a patch for e.g. SQL Server wouldn't always trigger a cold boot to be conservative. The alternative is what everyone's been saying, namely that you have to assume that a reboot is slow and have n > 1 servers if uptime matters.
SomeStupidPoint 12y ago
Popular (gaming) services have weekly downtimes to patch servers, and while users are annoyed, they seem to get over it and keep playing just fine.
- TillE 12y ago
  It's completely insane to have multi-hour downtime every week the way Blizzard does. Between that and their regionally segregated Battle.net (there's no good excuse that you can't play Hearthstone with anyone anywhere), they're basically stuck on server architectures from the 90s.

by akulbe 12y ago

noblethrasher 12y ago

A server is just a name. Just because a host machine needs to go down doesn't mean that the "server" has to go down.

Someone1234 12y ago

In that scenario I'd never have just a single server, be it Linux, Windows, BSD, or anything else. Being in a situation where you're completely unable to restart a machine without losing money, is being in a situation where you're living on a knife edge regardless. A single hardware failure and you're in deep trouble. I've been in exactly that situation (the entire company rested on a single Ubuntu server), it was really unpleasant and management didn't do a thing about it until we started getting sector errors which the RAID controller's error handling was struggling to cope with (after that incident and the downtime, management shelled out for four physical boxes, a SAN, VMWare licenses, and a setup with 15 minute rollbacks and dynamic migrations if a physical server died).

mreiland 12y ago
his point remains. being forced to run 2 servers so your server doesn't go down once/month is a bit ridiculous. No, it isn't untenable, yes you can work around it, but that's a problem that shouldn't need to be worked around.
- tempestn 12y ago
  Agreed. Up to a certain point, a single server is going to be considerably more cost effective than multiple machines, as well as requiring less devOps time. (Especially nowadays when you can easily lease a server with, say, 16 cores and 48GB of RAM.) I can recall having to restart our production server exactly once in the past 12 months for a kernel update. Every month would be an irritation for sure.
- WorldWideWayne 12y ago
  Is it normal to update server software like PostgreSQL or Redis on Linux without stopping the running process during the update? Maybe Linux doesn't need to restart the OS as often, but you're still going to have down time, right?
  - mreiland 12y ago
    You can update the software without pulling it down. In order to use the update you need to restart, but there's a large difference between restarting a service and restarting an OS. It isn't really analogous.
    - WorldWideWayne 12y ago
      It takes 10 seconds to restart Windows these days, so it's almost analagous.
      - acdha 12y ago
        Do you have any benchmarks to support that claim? I have Windows 8 and 10 systems which take minutes to reboot on an SSD which beg to differ.
        WorldWideWayne 12y ago
        You can see it when you do a video search for: Windows fast boot. Here are a couple: https://www.youtube.com/watch?v=p_vUZR2RtRs https://www.youtube.com/watch?v=p_vUZR2RtRs https://www.youtube.com/watch?v=xkKoggifsE4 https://www.youtube.com/watch?v=xkKoggifsE4 (And you don't need to log in for your background services to start.)
        acdha 12y ago
        Neither of those support your original claim. The author is focused on one specific feature, which is nice but excludes the entire shutdown process and booting up until the Windows logo appears. Anyone who's run a server knows that just the BIOS initialization is often much longer than 10 seconds. And, of course, many services take a noticeable amount of time to startup so time-to-available is longer than time-to-login-screen. Worse, the conversation was actually about the time needed to install updates. Now that Windows logs out before installing updates, you have a fairly sizable time delay – tens of minutes after a large update even on an SSD – while the service is unavailable but before the system starts rebooting. How does Fast Boot work? Well, it's actually not booting at all – it's logging out all active users and hibernating: http://blogs.msdn.com/b/b8/archive/2011/09/08/delivering-fast-boot-times-in-windows-8.aspx http://blogs.msdn.com/b/b8/archive/2011/09/08/delivering-fas... That's a really nice bit of work in normal usage but it means that you have to do the standard cold boot process when you're getting a system update. That means that a sysadmin would need to review each update to know whether it would trigger a fast or slow boot, assuming that a patch for e.g. SQL Server wouldn't always trigger a cold boot to be conservative. The alternative is what everyone's been saying, namely that you have to assume that a reboot is slow and have n > 1 servers if uptime matters.

SomeStupidPoint 12y ago

Popular (gaming) services have weekly downtimes to patch servers, and while users are annoyed, they seem to get over it and keep playing just fine.

TillE 12y ago
It's completely insane to have multi-hour downtime every week the way Blizzard does. Between that and their regionally segregated Battle.net (there's no good excuse that you can't play Hearthstone with anyone anywhere), they're basically stuck on server architectures from the 90s.