4 ms·

It's always good to see people challenging existing solutions .. this is how innovation happens. But you need to mention other good alternatives like fabric, pa

It's always good to see people challenging existing solutions .. this is how innovation happens. But you need to mention other good alternatives like fabric, paramiko, etc to list of alternatives... and plain old shell scripting. Some of the other options in orgalog seem like a combination of rsync, autossh (which has a heartbeat protocol built in), xargs, and plain old bash might be a better option. For ad hoc jobs like this, this seems like how I'd go about synchronizing local and remote directories on multiple servers in parallel: for host in host1 host2 host3 do rsync -rpae ssh /my/directory/ $host:/remote/directory/ & done wait That will synchronize a local directory with a remote directory, in parallel, and it's readable and uses software you probably already have installed. Or, if you want to limit parallelization to no more than 16 boxes simultaneously, use in combination with xargs (ie something like xargs --max-procs 16 -n 1 and call a short bash function with the given hostname). Orgalog doesn't seem to support limiting to n hosts at a time? Here's how to run the example uptime on hosts in parallel, but with batch limits: orgalog: orgalorg -o <host>... -x -C whoami bash: echo host1 host2 ... | xargs -d' ' -P2 -n1 -I{} ssh {} uptime Another orgalog example: orgalorg -o <host>... -p -i ~/.ssh/id_rsa.pub -C tee -a ~/.ssh/authorized_keys bash: echo host1 host2 ... | xargs -d' ' -P2 -n1 -I{} scp ~/.ssh/authorized_keys {} # or, Userify cloud or On Site for >1 users Of course, I'd just use Userify[1] in a local installation for that (disclaimer: CTO) but you could just modify the rsync example above or just use my old cloudadmins script[2] if your needs are small. This was obviously a big effort just looking at the dozens of go files! I don't want to disparage what was obviously a huge amount of work from someone who spent a lot of time. A cautionary note for anyone thinking about this: always focus on automating non-security-sensitive tasks first and try to use the smallest amount of magic necessary to do a trick. Rolling your own crypto is never a good idea. Clearly the OP is trying a lot of new ideas and we should expect good things in the future. 1. https://userify.com https://userify.com 2. https://github.com/jamiesonbecker/cloudadmins https://github.com/jamiesonbecker/cloudadmins

seletskiy 10y ago
Of course I'm aware of `for` loops in shell. But note, that your solution lacks: valid termination of commands by `CTRL-C`, aggregated progress indication, password authentication, various fail modes (like try to connect to all servers first), elevating privileges from user account to sudo (which is must have when you're logging in using your username, not root), no integration into other unix-way tools. Also, you will little bit stuck trying to run `tail -f` via xargs ssh. So, that's the reason for writing production-grade tool.
- deleted 10y ago
  [deleted]
- jamiesonbecker 10y ago
  It's clear that you've put a lot of thought into this! If I was going for production grade as opposed to a quick ad hoc job, I'd probably automate with a script to handle failures and send logs somewhere (thus wouldn't need or want control-C or progress indicators) ---- * valid termination of commands by `CTRL-C` Good point. I'd probably use pkill for that, if my ad hoc job went haywire :) ---- * aggregated progress indication You are correct, this is missing in my solution (nicely aggregated, anyway) -- but again if I was going to do this in production (i.e., automated), I'd probably not be doing it like this anyway.. I'd write a bash script, check status codes, timeouts, etc. For example, delivering sorted output via log files: for host in host1 host2 host3 do rsync -rpae ssh /my/directory/ $host:/remote/directory/ > rsync-$host.log 2>rsync-host.err & done and to syslog (hopefully a remote loghost) for host in host1 host2 host3 do rsync -rpae ssh /my/directory/ $host:/remote/directory/ | logger -t "myscript-$host" 2>&1 & done (note: --tag parameters for logger vary between RHEL and Debian-based distributions.. see the man page for specifics.) ---- * password authentication Please never do that. We don't even provide that as an option at Userify (https://userify.com https://userify.com, SSH key management). SSH is one of the few places where passwords are completely obsolete. ---- * various fail modes (like try to connect to all servers first) That seems like a simple thing to accomplish; are you saying orgalog connects to all servers first before connecting to all servers again? ---- * elevating privileges from user account to sudo (which is must have when you're logging in using your username, not root) Sure, that's easy.. echo host1 host2 ... | xargs -d' ' -P2 -n1 -I{} ssh {} sudo uptime If that doesn't work for you, you're probably using a version of red hat with a broken sudoers file (which goes back nearly ten years!) but they're fixing it in the next release: https://bugzilla.redhat.com/show_bug.cgi?id=1020147 https://bugzilla.redhat.com/show_bug.cgi?id=1020147 In the meantime, just comment out the Defaults requiretty line to get the same fix. Also, for the tty-requiring solutions that you mention, SSH has a command-line switch already: ssh -t and/or ssh -tt. `man 1 ssh` for details. Userify configures sudo on remote systems, btw. (SSH key management, https://userify.com https://userify.com, blatant plug: CTO) ---- * no integration into other unix-way tools I don't quite understand what you are saying.. I'd suggest that it's completely integrated into other unix tools, because it uses them instead of inventing a monolithic framework.

seletskiy 10y ago

Of course I'm aware of `for` loops in shell. But note, that your solution lacks: valid termination of commands by `CTRL-C`, aggregated progress indication, password authentication, various fail modes (like try to connect to all servers first), elevating privileges from user account to sudo (which is must have when you're logging in using your username, not root), no integration into other unix-way tools. Also, you will little bit stuck trying to run `tail -f` via xargs ssh. So, that's the reason for writing production-grade tool.

deleted 10y ago
[deleted]
jamiesonbecker 10y ago
It's clear that you've put a lot of thought into this! If I was going for production grade as opposed to a quick ad hoc job, I'd probably automate with a script to handle failures and send logs somewhere (thus wouldn't need or want control-C or progress indicators) ---- * valid termination of commands by `CTRL-C` Good point. I'd probably use pkill for that, if my ad hoc job went haywire :) ---- * aggregated progress indication You are correct, this is missing in my solution (nicely aggregated, anyway) -- but again if I was going to do this in production (i.e., automated), I'd probably not be doing it like this anyway.. I'd write a bash script, check status codes, timeouts, etc. For example, delivering sorted output via log files: for host in host1 host2 host3 do rsync -rpae ssh /my/directory/ $host:/remote/directory/ > rsync-$host.log 2>rsync-host.err & done and to syslog (hopefully a remote loghost) for host in host1 host2 host3 do rsync -rpae ssh /my/directory/ $host:/remote/directory/ | logger -t "myscript-$host" 2>&1 & done (note: --tag parameters for logger vary between RHEL and Debian-based distributions.. see the man page for specifics.) ---- * password authentication Please never do that. We don't even provide that as an option at Userify (https://userify.com https://userify.com, SSH key management). SSH is one of the few places where passwords are completely obsolete. ---- * various fail modes (like try to connect to all servers first) That seems like a simple thing to accomplish; are you saying orgalog connects to all servers first before connecting to all servers again? ---- * elevating privileges from user account to sudo (which is must have when you're logging in using your username, not root) Sure, that's easy.. echo host1 host2 ... | xargs -d' ' -P2 -n1 -I{} ssh {} sudo uptime If that doesn't work for you, you're probably using a version of red hat with a broken sudoers file (which goes back nearly ten years!) but they're fixing it in the next release: https://bugzilla.redhat.com/show_bug.cgi?id=1020147 https://bugzilla.redhat.com/show_bug.cgi?id=1020147 In the meantime, just comment out the Defaults requiretty line to get the same fix. Also, for the tty-requiring solutions that you mention, SSH has a command-line switch already: ssh -t and/or ssh -tt. `man 1 ssh` for details. Userify configures sudo on remote systems, btw. (SSH key management, https://userify.com https://userify.com, blatant plug: CTO) ---- * no integration into other unix-way tools I don't quite understand what you are saying.. I'd suggest that it's completely integrated into other unix tools, because it uses them instead of inventing a monolithic framework.