3 ms·

Process upgrades are a variant of fail-over on either hardware death or bug. I recommend treating upgrades as a chance to test your failure recovery processes.

Process upgrades are a variant of fail-over on either hardware death or bug. I recommend treating upgrades as a chance to test your failure recovery processes. If you really can't afford someone getting a "connection refused" what happens when the machine's network connection dies?

alexk 12y ago
For us it's about not dropping your customers requests when we can, obviously that happens sometimes, but there's no need to do that if you can avoid it during normal deployment cycles.
_Soulou 12y ago
You're right, but in some cases, for example a 'git push' to deploy an application, we don't want the SSH connection to be cut when upgrading our SSH server, this is not an option. If the server crashes the current connections will be lost, but it should be the only scenario when this occures.
- laumars 12y ago
  You can cut an SSH connection while SSHed in providing the server comes back up in a timely manner as SSH does some clever keep alive tricks to resume broken connections. I often restarted sshd or even the network interfaces themselves while SSHed into some Linux boxes.
  - _Soulou 12y ago
    That's because when you connect to your SSH, it forked and there is one process which is handling your connection, then when you restart the daemon, it only restarts the master, no harm done to the existing connections.
    - laumars 12y ago
      That doesn't help when you restart the network interfaces though. Normal TCP/IP behaviour would be for the connection to terminate, however SSH can reattach itself.
      - _Soulou 12y ago
        I don't want to say anything wrong on this and I don't have the knowledge on this, but it's possible than the OS is doing a lot of work on this to avoid cutting connections. Do you have any details on this?
        laumars 12y ago
        Sure, not a problem: "A keep-alive is a small piece of data transmitted between a client and a server to ensure that the connection is still open or to keep the connection open. Many protocols implement this as a way of cleaning up dead connections to the server. If a client does not respond, the connection is closed. SSH does not enable this by default. There are pros and cons to this. A major pro is that under a lot of conditions if you disconnect from the Internet, your connection will be usable when you reconnect. For those who drop out of WiFi a lot, this is a major plus when you discover you don't need to login again." Source: http://www.symkat.com/ssh-tips-and-tricks-you-need http://www.symkat.com/ssh-tips-and-tricks-you-need There's probably better sources out there, that was just one of the top results in Google as, if I'm honest, I'm not an expert on this either.
        bluecmd 12y ago
        Devil is in the details, but the source actually talks against you here: This happens because your router or firewall is trying to clean up dead connections. It's seeing that no data has been transmitted in N seconds and falsely assumes that the connection is no longer in use. To rectify this you can add a Keep-Alive. This will ensure that your connection stays open to the server and the firewall doesn't close it. In other words: What keep-alive does is that it prevents routers/middle-ware-boxes to forget that the connection exists in the first place. This is not needed on a clean internet connection where everything is treated as stateless and simple routing is everything that is done.

alexk 12y ago

For us it's about not dropping your customers requests when we can, obviously that happens sometimes, but there's no need to do that if you can avoid it during normal deployment cycles.

_Soulou 12y ago

You're right, but in some cases, for example a 'git push' to deploy an application, we don't want the SSH connection to be cut when upgrading our SSH server, this is not an option. If the server crashes the current connections will be lost, but it should be the only scenario when this occures.

laumars 12y ago
You can cut an SSH connection while SSHed in providing the server comes back up in a timely manner as SSH does some clever keep alive tricks to resume broken connections. I often restarted sshd or even the network interfaces themselves while SSHed into some Linux boxes.
- _Soulou 12y ago
  That's because when you connect to your SSH, it forked and there is one process which is handling your connection, then when you restart the daemon, it only restarts the master, no harm done to the existing connections.
  - laumars 12y ago
    That doesn't help when you restart the network interfaces though. Normal TCP/IP behaviour would be for the connection to terminate, however SSH can reattach itself.
    - _Soulou 12y ago
      I don't want to say anything wrong on this and I don't have the knowledge on this, but it's possible than the OS is doing a lot of work on this to avoid cutting connections. Do you have any details on this?
      - laumars 12y ago
        Sure, not a problem: "A keep-alive is a small piece of data transmitted between a client and a server to ensure that the connection is still open or to keep the connection open. Many protocols implement this as a way of cleaning up dead connections to the server. If a client does not respond, the connection is closed. SSH does not enable this by default. There are pros and cons to this. A major pro is that under a lot of conditions if you disconnect from the Internet, your connection will be usable when you reconnect. For those who drop out of WiFi a lot, this is a major plus when you discover you don't need to login again." Source: http://www.symkat.com/ssh-tips-and-tricks-you-need http://www.symkat.com/ssh-tips-and-tricks-you-need There's probably better sources out there, that was just one of the top results in Google as, if I'm honest, I'm not an expert on this either.
        bluecmd 12y ago
        Devil is in the details, but the source actually talks against you here: This happens because your router or firewall is trying to clean up dead connections. It's seeing that no data has been transmitted in N seconds and falsely assumes that the connection is no longer in use. To rectify this you can add a Keep-Alive. This will ensure that your connection stays open to the server and the firewall doesn't close it. In other words: What keep-alive does is that it prevents routers/middle-ware-boxes to forget that the connection exists in the first place. This is not needed on a clean internet connection where everything is treated as stateless and simple routing is everything that is done.