3 ms·
>> Now let's get practical: find me such a library. > Certainly. I'm most familiar with Python, and for that, both Paramiko and asyncssh support returning the
by dozzie 10y ago
>> Now let's get practical: find me such a library.
> Certainly. I'm most familiar with Python, and for that, both Paramiko and asyncssh support returning the exit status of a command, distinct from any network failure.
Oh, that's something new. From what I heard, Paramiko used `ssh' under the
hood, but apparently I was wrong.
> ([...] The ssh binary is mostly a porcelain interface to the SSH protocol;
libraries are a better tool for building on top of.)
There's still a big problem on the client side itself. SSHv2 is a complicated
protocol, so client libraries' APIs are not pretty or force you to use
OpenSSH-like home (or both, like Paramiko does).
>> with tracking state of each and every machine I would possibly want to reconfigure, and remembering to check if the machine was booted to re-issue a reconfiguration command.
> Perhaps this is just a difference in our production environments. We strive very hard to have all our machines online at all times.
You must then have very homogenous environment with a small team. Sounds like
a startup of sorts. Anything more loose (e.g. more than one team working on
the servers, or split between who installs the machines in racks and their
OSes on disk and who manages the OSes) and you won't have the luxury of
keeping everything up and running all the time.
>> Sorry, I'm lost here: what re-running?
> My larger point was about non idempotent commands. If your infrastructure retries commands, are you just very careful to not have non-idempotent stuff?
And what does it have anything to do with re-sending the commands when they
didn't reach their target machine?
> Again, if you can structure the command to be idempotent, you can work
around this, but in my experience there always seems to be some patch or
database upgrade that this can't be accomplished for.
Uhm... What? Where did you get the idea that upgrading database is managing
configuration? It's a deployment operation. A different category.
- deathanatos 10y ago> And what does it have anything to do with re-sending the commands when they didn't reach their target machine? It has to do with whether its at all safe to run the command twice. If you don't hear back from a machine that your command ran, you don't know if the command reached the machine or not, in general. If you decide to not re-run the command, it may be that it never made it, and you never run the command. If you decide to re-run the command, it may have reached the machine, and you've now run the command twice. Consider: 1. You send command $X to a remote machine. 2. The remote receives and executes $X successfully. 3. Just prior to sending back that $X was successful, the remote dies. From the point of view of the deployment machine, you never hear back from the remote. You don't even need two machines to get into this state: even on a single command, if you run $X and the machine crashes just as $X is about to return whether or not it succeeded or failed, you're out of luck. For specific commands, such as ones that log their progress and can recover from crashes, you can determine after a crash the correct procedure automatically. But not every arbitrary Unix administration command falls into that bucket, especially in the weird and wacky things that need to be done to keep production machines alive. > Uhm... What? Where did you get the idea that upgrading database is managing configuration? It's a deployment operation. A different category. We used our deployment infra for all sorts of fun stuff.