6 ms·

A machine staying up for almost 3 years is irresponsible in this day and age. Yeah, I remember people having uptime competitions on Slashdot and the like some

A machine staying up for almost 3 years is irresponsible in this day and age. Yeah, I remember people having uptime competitions on Slashdot and the like some decades back, but you only need to look at the ssh logs of a 5 minutes old machine to realize this is a terrible idea in modern times.

sokoloff 3y ago
Air gapped machines and kernel live patching both exist.
- cpach 3y ago
  And how many people use that? Most servers today are not air-gapped.
  - j16sdiz 3y ago
    Most server don't do that, but those that do are not crazy
  - agentgumshoe 3y ago
    How many examples will you need before you say "oh ok, I can see some valid concerns."? I've worked in places where expensive Lab equipment is running off outdated PCs/servers because updates aren't available and they will absolutely stay on for as long as possible. We're not all silicon valley, things can be expensive and difficult to replace...
  - BasedAnon 3y ago
    I have kernel live patching on my mother's computer because it means she has to know how to do less
mnw21cam 3y ago
Kernel bugs are rare. Most (almost every single) vulnerability can be patched without rebooting.
- 0x0 3y ago
  They're not that rare. Also, there are a lot of other updates that in practice should be followed up with a reboot. For example, any library consumed by systemd (such as openssl) usually requires pid1 to relaunch. For example, debian released an openssl update just yesterday. You can run "checkrestart -v" to try to figure out how to restart every affected app but you'll quickly run into systemd's init process running with the old vulnerable library loaded, and then you might as well just reboot to get a clean "checkrestart -v". Even just relaunching non-pid-1 applications like dbus can quickly create a mess where sshd logins get a delay if you're not careful to also reload everything that depends on it.
  - Denvercoder9 3y ago
    > For example, any library consumed by systemd (such as openssl) usually requires pid1 to relaunch. That does not require a reboot, `systemctl daemon-reexec` is enough.
    - znpy 3y ago
      nice username, can i ask you what did you see?
oynqr 3y ago
You don't need full system resets to get security updates. Kexec, live patching, userspace reboot.
hardware2win 3y ago
I dont understand opinons like this Just because it would be dangerous for your nodejs web_app.exe running on ubuntu behind apache fully exposed on the internet then there are billion other ways to use computers, like even air gapped systems. So, dont try to justify obvious flaw
- dale_glass 3y ago
  I mean, hardware is cheap enough that any server of importance should be individually disposable. Yeah, you can do stuff to maximize uptime but if it needs to stay up that badly you have to consider the case of the hardware needing to be turned off at some point. > So, dont try to justify obvious flaw I'm not, it's a bug and should be fixed. But I think if anything is powered for 3 years straight it's a bit concerning. Otherwise you're liable to find things like that somebody started something by hand 2 years ago, and at a critical moment nobody quite remember what the command was.
  - defrost 3y ago
    > But I think if anything is powered for 3 years straight it's a bit concerning. Pretty much why Pawsey has an Annual High Voltage inspection shutdown [1] > Otherwise you're liable to find things like [..] TBH that's not really been an issue of note at any of the big iron farms I've been around since the 1980s .. generally there's a disciplined approach to maintaining 24/7/365 operation (that includes scheduled downtime for equipment checks) part of which is process documentation and justification and soft means of freezing | migrating processes+data etc. [1] https://status.pawsey.org.au/incidents/tk5n5y965r5j https://status.pawsey.org.au/incidents/tk5n5y965r5j
  - PedroBatista 3y ago
    You live in your own World with other people. Please just keep in mind there are many other Worlds with other people and laws of the Universe. I don't know if you're young or don't know much about history but what you describe is a fairly recent way of looking at things, it's not the only one and I guarantee you it will become "out of fashion".
shpx 3y ago
1042 days ought to be enough for anybody
- Tajnymag 3y ago
  Not for a server
  - mihaic 3y ago
    Not sure if he was sarcastically referencing "640kb should be enough for anything"
- ant6n 3y ago
  > 1042 days ought to be enough for anybody "640K ought to be enough for anybody."
- mhuffman 3y ago
  Spoken like a true AWS user!
kuratkull 3y ago
Are you perhaps a Windows user? In the Linux world updates don't necessarily require reboots.
- dale_glass 3y ago
  Actually as of late, Linux has been moving towards rebooting for update. Yeah, you technically can replace on-disk files while services are running. In practice this can cause trouble if an application wants to read an updated file at the wrong time, and library dependencies can require restarting a lot of stuff. For ages people would install an update containing a security fix in glibc or libz or something, and keep on running the vulnerable version of the services that use them. At that point you might as well reboot. Modern Fedora has a very Windows-like mechanism where you reboot to update. You reboot, the system installs updates, then reboots again.
  - viraptor 3y ago
    While Fedora did move towards that, it's not the only way. A lot of systems which require high reliability are built to reload correctly. At a generic system level, for example upgrading Nixos will pull new packages and put them next to the current ones, then reexec where possible. Nginx can replace its master process (SIGUSR2). Telephony software can often reexec and keep connecting open. Etc. Outside of desktops it's not that uncommon to do seamless live reloads of the whole system.
  - justinclift 3y ago
    > Actually as of late, Linux has been moving ... That's a pretty broad generalisation. Which distro's are you meaning?
    - magicalhippo 3y ago
      KDE Neon has done this. Before I had to reboot anyway because usually the desktop was full of random crashes if I updated without rebooting.
ly3xqhl8g9 3y ago
3 years is irresponsible? To quote Logan Roy, you, software developers, "are not serious people" [1]. Just out of curiosity looked for a list of longest running electrical devices [2]: 1840 - The Oxford Electric Bell 1871 – Souter Lighthouse in South Shields, UK 1896 – The Isle of Man’s Manx Electric Railway 1902 – The Centennial Bulb Apparently, "The Centennial Bulb has seen just two interruptions: for a week in 1937 when the Firehouse was refurbished, and in May 2013 when it was off for nine and a half hours due to a failed power supply." [1] https://www.youtube.com/watch?v=LZTaXjt2Ggk https://www.youtube.com/watch?v=LZTaXjt2Ggk [2] https://www.drax.com/electrification/4-of-the-longest-running-electrical-objects/ https://www.drax.com/electrification/4-of-the-longest-runnin...
- deleted 3y ago
  [deleted]
- dale_glass 3y ago
  Those are completely trivial complexity-wise compared to a modern server, and many don't have a real function, and mostly are artificially maintained as a curiosity. I mean, the centennial bulb barely glows, that's why it still works. The hotter the filament gets the faster it evaporates, so a light bulb that barely makes any light can stay working forever.
  - ly3xqhl8g9 3y ago
    Sure, was looking for electrical devices, a better example of what great engineering can achieve I suppose it's the Pons Fabricius [1], bridge built 2,085 years ago, still in use. The problem is, if we can't expect software to run essentially forever, to update without 'restarts', and so forth, how are we ever going to achieve neural chip implants, artificial organs, synthetic agents mining ore in outer space, and so on? Software is not a gear mechanism, a rack and pinion, there is absolutely no reason to restart an 'operating system' or to ever lose state, however we became accustomed and we commit these sort of crimes daily, restarts and refreshes. [1] https://en.wikipedia.org/wiki/Pons_Fabricius https://en.wikipedia.org/wiki/Pons_Fabricius
cesarb 3y ago
> A machine staying up for almost 3 years is irresponsible in this day and age. [...] but you only need to look at the ssh logs of a 5 minutes old machine to realize this is a terrible idea in modern times. You don't need to reboot a machine to update ssh. You only need to reboot the machine to update the kernel; for everything else, you just have to restart the corresponding user-space processes (and even PID1 can re-exec itself). Most kernel vulnerabilities are not remotely exploitable, so as long as you can trust your user-space processes (and keep them updated), it should be safe enough.
- kjs3 3y ago
  As I recall, machines made by Tandem Computers, among other highly fault tolerant machines that have regrettably fallen out of fashion, didn't have to reboot even to replace the kernel. They didn't run Linux, tho.

sokoloff 3y ago

Air gapped machines and kernel live patching both exist.

cpach 3y ago
And how many people use that? Most servers today are not air-gapped.
- j16sdiz 3y ago
  Most server don't do that, but those that do are not crazy
- agentgumshoe 3y ago
  How many examples will you need before you say "oh ok, I can see some valid concerns."? I've worked in places where expensive Lab equipment is running off outdated PCs/servers because updates aren't available and they will absolutely stay on for as long as possible. We're not all silicon valley, things can be expensive and difficult to replace...
- BasedAnon 3y ago
  I have kernel live patching on my mother's computer because it means she has to know how to do less

mnw21cam 3y ago

Kernel bugs are rare. Most (almost every single) vulnerability can be patched without rebooting.

0x0 3y ago
They're not that rare. Also, there are a lot of other updates that in practice should be followed up with a reboot. For example, any library consumed by systemd (such as openssl) usually requires pid1 to relaunch. For example, debian released an openssl update just yesterday. You can run "checkrestart -v" to try to figure out how to restart every affected app but you'll quickly run into systemd's init process running with the old vulnerable library loaded, and then you might as well just reboot to get a clean "checkrestart -v". Even just relaunching non-pid-1 applications like dbus can quickly create a mess where sshd logins get a delay if you're not careful to also reload everything that depends on it.
- Denvercoder9 3y ago
  > For example, any library consumed by systemd (such as openssl) usually requires pid1 to relaunch. That does not require a reboot, `systemctl daemon-reexec` is enough.
  - znpy 3y ago
    nice username, can i ask you what did you see?

oynqr 3y ago

You don't need full system resets to get security updates. Kexec, live patching, userspace reboot.

hardware2win 3y ago

I dont understand opinons like this Just because it would be dangerous for your nodejs web_app.exe running on ubuntu behind apache fully exposed on the internet then there are billion other ways to use computers, like even air gapped systems. So, dont try to justify obvious flaw

dale_glass 3y ago
I mean, hardware is cheap enough that any server of importance should be individually disposable. Yeah, you can do stuff to maximize uptime but if it needs to stay up that badly you have to consider the case of the hardware needing to be turned off at some point. > So, dont try to justify obvious flaw I'm not, it's a bug and should be fixed. But I think if anything is powered for 3 years straight it's a bit concerning. Otherwise you're liable to find things like that somebody started something by hand 2 years ago, and at a critical moment nobody quite remember what the command was.
- defrost 3y ago
  > But I think if anything is powered for 3 years straight it's a bit concerning. Pretty much why Pawsey has an Annual High Voltage inspection shutdown [1] > Otherwise you're liable to find things like [..] TBH that's not really been an issue of note at any of the big iron farms I've been around since the 1980s .. generally there's a disciplined approach to maintaining 24/7/365 operation (that includes scheduled downtime for equipment checks) part of which is process documentation and justification and soft means of freezing | migrating processes+data etc. [1] https://status.pawsey.org.au/incidents/tk5n5y965r5j https://status.pawsey.org.au/incidents/tk5n5y965r5j
- PedroBatista 3y ago
  You live in your own World with other people. Please just keep in mind there are many other Worlds with other people and laws of the Universe. I don't know if you're young or don't know much about history but what you describe is a fairly recent way of looking at things, it's not the only one and I guarantee you it will become "out of fashion".

shpx 3y ago

1042 days ought to be enough for anybody

Tajnymag 3y ago
Not for a server
- mihaic 3y ago
  Not sure if he was sarcastically referencing "640kb should be enough for anything"
ant6n 3y ago
> 1042 days ought to be enough for anybody "640K ought to be enough for anybody."
mhuffman 3y ago
Spoken like a true AWS user!

kuratkull 3y ago

Are you perhaps a Windows user? In the Linux world updates don't necessarily require reboots.

dale_glass 3y ago
Actually as of late, Linux has been moving towards rebooting for update. Yeah, you technically can replace on-disk files while services are running. In practice this can cause trouble if an application wants to read an updated file at the wrong time, and library dependencies can require restarting a lot of stuff. For ages people would install an update containing a security fix in glibc or libz or something, and keep on running the vulnerable version of the services that use them. At that point you might as well reboot. Modern Fedora has a very Windows-like mechanism where you reboot to update. You reboot, the system installs updates, then reboots again.
- viraptor 3y ago
  While Fedora did move towards that, it's not the only way. A lot of systems which require high reliability are built to reload correctly. At a generic system level, for example upgrading Nixos will pull new packages and put them next to the current ones, then reexec where possible. Nginx can replace its master process (SIGUSR2). Telephony software can often reexec and keep connecting open. Etc. Outside of desktops it's not that uncommon to do seamless live reloads of the whole system.
- justinclift 3y ago
  > Actually as of late, Linux has been moving ... That's a pretty broad generalisation. Which distro's are you meaning?
  - magicalhippo 3y ago
    KDE Neon has done this. Before I had to reboot anyway because usually the desktop was full of random crashes if I updated without rebooting.

ly3xqhl8g9 3y ago

3 years is irresponsible? To quote Logan Roy, you, software developers, "are not serious people" [1]. Just out of curiosity looked for a list of longest running electrical devices [2]: 1840 - The Oxford Electric Bell 1871 – Souter Lighthouse in South Shields, UK 1896 – The Isle of Man’s Manx Electric Railway 1902 – The Centennial Bulb Apparently, "The Centennial Bulb has seen just two interruptions: for a week in 1937 when the Firehouse was refurbished, and in May 2013 when it was off for nine and a half hours due to a failed power supply." [1] https://www.youtube.com/watch?v=LZTaXjt2Ggk https://www.youtube.com/watch?v=LZTaXjt2Ggk [2] https://www.drax.com/electrification/4-of-the-longest-running-electrical-objects/ https://www.drax.com/electrification/4-of-the-longest-runnin...

deleted 3y ago
[deleted]
dale_glass 3y ago
Those are completely trivial complexity-wise compared to a modern server, and many don't have a real function, and mostly are artificially maintained as a curiosity. I mean, the centennial bulb barely glows, that's why it still works. The hotter the filament gets the faster it evaporates, so a light bulb that barely makes any light can stay working forever.
- ly3xqhl8g9 3y ago
  Sure, was looking for electrical devices, a better example of what great engineering can achieve I suppose it's the Pons Fabricius [1], bridge built 2,085 years ago, still in use. The problem is, if we can't expect software to run essentially forever, to update without 'restarts', and so forth, how are we ever going to achieve neural chip implants, artificial organs, synthetic agents mining ore in outer space, and so on? Software is not a gear mechanism, a rack and pinion, there is absolutely no reason to restart an 'operating system' or to ever lose state, however we became accustomed and we commit these sort of crimes daily, restarts and refreshes. [1] https://en.wikipedia.org/wiki/Pons_Fabricius https://en.wikipedia.org/wiki/Pons_Fabricius

cesarb 3y ago

> A machine staying up for almost 3 years is irresponsible in this day and age. [...] but you only need to look at the ssh logs of a 5 minutes old machine to realize this is a terrible idea in modern times. You don't need to reboot a machine to update ssh. You only need to reboot the machine to update the kernel; for everything else, you just have to restart the corresponding user-space processes (and even PID1 can re-exec itself). Most kernel vulnerabilities are not remotely exploitable, so as long as you can trust your user-space processes (and keep them updated), it should be safe enough.

kjs3 3y ago
As I recall, machines made by Tandem Computers, among other highly fault tolerant machines that have regrettably fallen out of fashion, didn't have to reboot even to replace the kernel. They didn't run Linux, tho.