3 ms·

Heroku reboots servers every night no matter what stack is running on them. Same idea. The problem is that you merely borrowed yourself some time. As time goes

Heroku reboots servers every night no matter what stack is running on them. Same idea. The problem is that you merely borrowed yourself some time. As time goes on, more inefficiencies/bugs of this nature will creep in unnoticed, some will perhaps silently corrupt data before it is noticed (!), and it will be vastly more difficult at that point to troubleshoot 10 bugs of varying degrees of severity and frequency all happening at the same time causing you to have to reboot said servers at faster and faster intervals which simultaneously makes it harder to diagnose them individually. > It's shockingly stable. Well of course it is. You're "turning it off and then on again," the classic way to return to a known-good state. It is not a root-cause fix though, it is a band-aid.

lanstin 2y ago
Also, it means you are married to the reboot process. If you loose control of your memory management process too much, you'll never be able to fix it absent a complete rewrite. I worked at a place that had a lot of (c++) CGI programs with a shocking level of disregard for freeing memory, but that was ok because when the CGI request was over the process restarted. But then they reused that same code in SOA/long lived services, but they could never have one worker process handle more than 10 requests due to memory leaks (and inability to re-initialize all the memory used in a request). So they could never use in-process caching or any sort of optimization that long-lived processes could enable.
- pmarreck 2y ago
  I never considered "having to reboot" as "introducing another dependency" (in the sense of wanting to keep those at a minimum) but sure enough, it is. Also, great point about (depending on your architecture) losing the ability to do things like cache results
- andrewf 2y ago
  You kinda want some machines to reboot less frequently than others, so issues don't creep up on you. You also want some machines to reboot much more frequently than others, so you catch boot issues before they affect your entire fleet.
  - lanstin 2y ago
    Not rebooting after software upgrade is an oft repeated mistake.

lanstin 2y ago

Also, it means you are married to the reboot process. If you loose control of your memory management process too much, you'll never be able to fix it absent a complete rewrite. I worked at a place that had a lot of (c++) CGI programs with a shocking level of disregard for freeing memory, but that was ok because when the CGI request was over the process restarted. But then they reused that same code in SOA/long lived services, but they could never have one worker process handle more than 10 requests due to memory leaks (and inability to re-initialize all the memory used in a request). So they could never use in-process caching or any sort of optimization that long-lived processes could enable.

pmarreck 2y ago
I never considered "having to reboot" as "introducing another dependency" (in the sense of wanting to keep those at a minimum) but sure enough, it is. Also, great point about (depending on your architecture) losing the ability to do things like cache results
andrewf 2y ago
You kinda want some machines to reboot less frequently than others, so issues don't creep up on you. You also want some machines to reboot much more frequently than others, so you catch boot issues before they affect your entire fleet.
- lanstin 2y ago
  Not rebooting after software upgrade is an oft repeated mistake.