4 ms·

First, I just want to say that I appreciate this conversation and debate. It's making me think hard about certain concepts, so thank you. > If you ask this que

First, I just want to say that I appreciate this conversation and debate. It's making me think hard about certain concepts, so thank you. > If you ask this question, it means you've probably never configured SSH in bigger environments, because you don't imagine how easy is to accidentally cut off your access. Many moons ago, I was mucking with sshd configuration, and locked myself out. It was the first and only time I ever had to truly hack - my own system. The system had a Wordpress installation which, at least back then, allowed you to modify the PHP code through the web console. I knew I had gcc on the box, so I found a NULL-dereference root escalation exploit for my kernel version, jammed a bunch of C code into a php system call, and step by step, escalated, and then overwrote my ssh config (all through escaped C code in PHP!). It was fun. So I totally hear ya, updating sshd configuration can be disastrous, and it's certainly required in certain situations. Posixcube is certainly not ready for massive enterprise usage, but I don't see why it couldn't be. For this particular point, it seems that no matter what technology you're using, SSH can always get mucked up, and I see no inherent reason why shell scripts are worse. > You actually have provided me another important point: how to tell that the command reached its destination server, but failed there? This is vastly different case than when a command didn't get through to the server, so it can be safely re-sent. (And I was talking about the latter case.) The way this is handled now is that all cube_* APIs check all return codes and if non-zero, they print the full stack and `exit $code` which is passed back through SSH into the client and reported and the script stops. In the latter case of safely re-sending a command that didn't even get to the server, I don't think that's much of an issue here, because posixcube doesn't execute one command at a time (since the overhead of re-establishing the SSH connection would be massive), but actually generates a shell script on the fly (cube_exec.sh) which has all the commands you want to execute, uploads that as well as all of the cubes, and then executes cube_exec.sh on the other side. > But now you have a queue which needs to be managed and that also can break. This is not an improvement. Setup with a dedicated agent that works in "pull" mode (instead of one that was not designed for this task, but just happened to be already installed) doesn't have the problem that needs solving with queue in the first place. Are you talking about a use case where an administrator is not monitoring a deployment (i.e. fire and forget)?

dozzie 10y ago
> Posixcube is certainly not ready for massive enterprise usage, but I don't see why it couldn't be. Because its architecture doesn't fit the job. The same situation is with Ansible or Salt being used for maintaining configuration of servers. > For this particular point, it seems that no matter what technology you're using, SSH can always get mucked up, and I see no inherent reason why shell scripts are worse. I don't say shell scripts are worse. I say the "push" architecture over SSH is worse for configuring machines than with an agent in "pull" mode. You could easily write an agent that works on a cron-like schedule, downloads commands to be run, (verify their origin with a digital signature,) runs them, and reports the results back, all in shell. > Are you talking about a use case where an administrator is not monitoring a deployment (i.e. fire and forget)? No, I'm not talking about deployment. Deployment is a one-off operation (even if similar is repeated in the future). I'm talking about managing configuration, which is what Chef, Puppet, and CFEngine are for. If you were trying to use Chef for deployment, then no wonder you got a terrible experience.
- schizoidboy 10y ago
  > Because its architecture doesn't fit the job. The same situation is with Ansible or Salt being used for maintaining configuration of servers. > I don't say shell scripts are worse. I say the "push" architecture is worse for configuring machines than with an agent in "pull" mode. You could easily write an agent that works on a cron-like schedule, downloads commands to be run, (verify their origin with a digital signature,) runs them, and reports the results back, all in shell. I'm still not completely understanding, so let me see if I can rephrase your objection and please correct me if I'm getting it wrong: A "pull" architecture for configuration management is better than a "push" architecture because updating configuration shouldn't require synchronous human interaction. If so, then one example of a counterargument is needing to reload/restart a service after updating its configuration file. What if there's an error reloading/restarting the service? Does the agent then email the administrator? And what if the emails fail?
  - dozzie 10y ago
    > A "pull" architecture for configuration management is better than a "push" architecture because updating configuration shouldn't require synchronous human interaction. One may put it this way as one of the reasons, yes. > If so, then one example of a counterargument is needing to reload/restart a service after updating its configuration file. What if there's an error reloading/restarting the service? Does the agent then email the administrator? And what if the emails fail? You're on a good track here. There are several things not quite there yet, though. First, e-mail reports don't scale. For a dozen of servers the volume is still readable if every report fits on a single screen, but a move in any direction (report's length or number of servers) makes the situation unsustainable. Second, all of the operations should be reported, not just the failures, as you want to be able to see what was done, when, and in what order. Usually it's not interesting, as most logs are, but there are cases when you need this information and you learn about the need after the fact. Also, there are abnormalities that are reported as successes (e.g. a config that gets changed on every scheduled execution that gets replaced by something else, thus creating the need for updating the config again). Third, the operation logs should be structured. Plain text doesn't cut it, as you can't process them with a machine in a reliable and convenient way. And fourth, you were right when you asked what if the report transport breaks down. This is covered by having reports from each execution; a simple automaton (that either queries execution reports from their store or tracks them as they come) can easily tell that a server went missing. Such an automaton would be a part of reports receiver or reports store. By the way, reading execution reports from a push tool doesn't scale for the same reason as e-mail: too much manual reading. And if the tool stops on the first error to be encountered, tool's use will be annoying in the long run (how to recover and resume operations?)

dozzie 10y ago

> Posixcube is certainly not ready for massive enterprise usage, but I don't see why it couldn't be. Because its architecture doesn't fit the job. The same situation is with Ansible or Salt being used for maintaining configuration of servers. > For this particular point, it seems that no matter what technology you're using, SSH can always get mucked up, and I see no inherent reason why shell scripts are worse. I don't say shell scripts are worse. I say the "push" architecture over SSH is worse for configuring machines than with an agent in "pull" mode. You could easily write an agent that works on a cron-like schedule, downloads commands to be run, (verify their origin with a digital signature,) runs them, and reports the results back, all in shell. > Are you talking about a use case where an administrator is not monitoring a deployment (i.e. fire and forget)? No, I'm not talking about deployment. Deployment is a one-off operation (even if similar is repeated in the future). I'm talking about managing configuration, which is what Chef, Puppet, and CFEngine are for. If you were trying to use Chef for deployment, then no wonder you got a terrible experience.

schizoidboy 10y ago
> Because its architecture doesn't fit the job. The same situation is with Ansible or Salt being used for maintaining configuration of servers. > I don't say shell scripts are worse. I say the "push" architecture is worse for configuring machines than with an agent in "pull" mode. You could easily write an agent that works on a cron-like schedule, downloads commands to be run, (verify their origin with a digital signature,) runs them, and reports the results back, all in shell. I'm still not completely understanding, so let me see if I can rephrase your objection and please correct me if I'm getting it wrong: A "pull" architecture for configuration management is better than a "push" architecture because updating configuration shouldn't require synchronous human interaction. If so, then one example of a counterargument is needing to reload/restart a service after updating its configuration file. What if there's an error reloading/restarting the service? Does the agent then email the administrator? And what if the emails fail?
- dozzie 10y ago
  > A "pull" architecture for configuration management is better than a "push" architecture because updating configuration shouldn't require synchronous human interaction. One may put it this way as one of the reasons, yes. > If so, then one example of a counterargument is needing to reload/restart a service after updating its configuration file. What if there's an error reloading/restarting the service? Does the agent then email the administrator? And what if the emails fail? You're on a good track here. There are several things not quite there yet, though. First, e-mail reports don't scale. For a dozen of servers the volume is still readable if every report fits on a single screen, but a move in any direction (report's length or number of servers) makes the situation unsustainable. Second, all of the operations should be reported, not just the failures, as you want to be able to see what was done, when, and in what order. Usually it's not interesting, as most logs are, but there are cases when you need this information and you learn about the need after the fact. Also, there are abnormalities that are reported as successes (e.g. a config that gets changed on every scheduled execution that gets replaced by something else, thus creating the need for updating the config again). Third, the operation logs should be structured. Plain text doesn't cut it, as you can't process them with a machine in a reliable and convenient way. And fourth, you were right when you asked what if the report transport breaks down. This is covered by having reports from each execution; a simple automaton (that either queries execution reports from their store or tracks them as they come) can easily tell that a server went missing. Such an automaton would be a part of reports receiver or reports store. By the way, reading execution reports from a push tool doesn't scale for the same reason as e-mail: too much manual reading. And if the tool stops on the first error to be encountered, tool's use will be annoying in the long run (how to recover and resume operations?)