4 ms·

Again, thanks for the objections, they're making me think hard about the design choices. > Lucky you if your SSH client never failed to connect. How do you thi

Again, thanks for the objections, they're making me think hard about the design choices. > Lucky you if your SSH client never failed to connect. How do you think it reports a network failure? Currently posixcube handles this by checking the return code and reporting the error. > You see, there are tools that allow you to easily tell between network failure and remote command failure. RPC protocols are an example. ssh does have some basic differentiation: "ssh exits with the exit status of the remote command or with 255 if an error occurred." Although, of course, 255 might be returned by a remote command (not sure how likely that is), so it's still ambiguous. I guess you're saying that if it's an SSH error, then that's pretty safe to retry and it would be a nice feature, and I agree so I've opened this issue: https://github.com/myplaceonline/posixcube/issues/6 https://github.com/myplaceonline/posixcube/issues/6 > Now let's get practical: find me such a library. I guess it would just be a matter of, first, checking for 255, and next, checking the output of the command to see if it's an SSH error for sure? > We were talking about configuring servers. You basically tell me that I would have to pay for "agentlessness" with tracking state of each and every machine I would possibly want to reconfigure, and remembering to check if the machine was booted to re-issue a reconfiguration command. So you bring up a good point. On DigitalOcean, for example, their Fedora images use extlinux for booting and I can easily update the Linux command line (e.g. crashkernel), reboot, and it picks it up. Their Ubuntu distribution uses grub, and update-grub works fine, but for some reason, it requires a hard shutdown instead of a reboot to pick up the change. The way I handle this now is here: https://github.com/myplaceonline/myplaceonline_posixcubes/blob/824dbdbde086a173692d88c6066c9e5ec393315c/core_begin/core_begin.sh#L255 https://github.com/myplaceonline/myplaceonline_posixcubes/bl... We echo this instruction to the user. This prints a red-text error message and stops execution. This seems even better than an agent-based execution because with an agent-based architecture, the agent is now gone (hard powered off); whereas, this style of orchestration in the front-end can use a script on top of posixcube.sh that waits a minute, remotely calls DO to start the droplet, and then continues execution (using the same script since it's essentially idempotent). The only other option would be to have a centralized server that orchestrates all of this, which posixcube avoids. > Sorry, I'm lost here: what re-running? The command didn't get the chance to run in the first place. I don't know where did you get the idea that I propose running the command again if it got through to the remote server and failed there. See above, I'll need to try it, but I think the ssh error code for "command didn't get through" along with some simple heuristics on the command output /might/ work; I'll investigate in issue #6.

dozzie 10y ago
> ssh does have some basic differentiation: "ssh exits with the exit status of the remote command or with 255 if an error occurred." Although, of course, 255 might be returned by a remote command (not sure how likely that is), so it's still ambiguous. If one uses SSH for running such scripts, then it is not unlikely that the script itself would have another call to `ssh' to an even-more-remote machine. And then there is the case of not expecting that a command would ever exit with 255 code. My trust to a system would be quite damaged if I happened to run into such a bug that a proper remote error would be mistaken for a network error. How can I, as the script writer, be sure that I never run into this situation?
- schizoidboy 10y ago
  I agree the ambiguous return code is a major limitation of SSH. I think I'll just avoid any retry logic and treat every failure the same: fail immediately and let the user to decide how to retry. Since posixcube does most of its execution over a single SSH connection (packaging all commands/cubes into a cube_exec.sh which is executed remotely), I think the immediate need for automatic retry on failing to connect over SSH isn't large.

dozzie 10y ago

> ssh does have some basic differentiation: "ssh exits with the exit status of the remote command or with 255 if an error occurred." Although, of course, 255 might be returned by a remote command (not sure how likely that is), so it's still ambiguous. If one uses SSH for running such scripts, then it is not unlikely that the script itself would have another call to `ssh' to an even-more-remote machine. And then there is the case of not expecting that a command would ever exit with 255 code. My trust to a system would be quite damaged if I happened to run into such a bug that a proper remote error would be mistaken for a network error. How can I, as the script writer, be sure that I never run into this situation?

schizoidboy 10y ago
I agree the ambiguous return code is a major limitation of SSH. I think I'll just avoid any retry logic and treat every failure the same: fail immediately and let the user to decide how to retry. Since posixcube does most of its execution over a single SSH connection (packaging all commands/cubes into a cube_exec.sh which is executed remotely), I think the immediate need for automatic retry on failing to connect over SSH isn't large.