4 ms·

Heh, I made a similar thing (mostly unreleased/undocumented at [0], original idea at [1]) that would take your Go code, build a binary wrapper for it using the

Heh, I made a similar thing (mostly unreleased/undocumented at [0], original idea at [1]) that would take your Go code, build a binary wrapper for it using the target OS arch, SSH to the target, SCP the file over, run the binary, connect stdout/stderr/stdin with the remote, allow the remotely running binary to request files from the home binary, and delete itself once complete. One thing I found is SCP/SFTP is a really slow transfer protocol and an alternative should be made available as an option (but leave SSH as the default). I believe that we are getting to a point where config management might as well be in a programming language instead of a bunch of ad-hoc scripts in templates that defer to dynamic language scripts and become a mess. 0 - https://github.com/cretz/systrument https://github.com/cretz/systrument 1 - https://github.com/cretz/software-ideas/issues/1 https://github.com/cretz/software-ideas/issues/1

akhilcacharya 8y ago
Is the purpose to use a faster machine for a particular compute job?
- kodablah 8y ago
  Nope, just to deploy to or configure a remote machine. So you write some Go code as though it's running locally that may run some apt commands, or may extract a tarball, or may setup a systemd script or whatever. Then the tool builds that binary and runs it on that remote machine (fetching resources lazily from the home computer such as the tarball you might need), then it deletes itself. I've used it for multiple server things that require more of an upgrade approach than a ephemeral container approach...I welcome anyone to take the idea and run with it.
  - tenken 8y ago
    So it's a virus? ... because it sounds like a virus :D
drbawb 8y ago
>One thing I found is SCP/SFTP is a really slow transfer protocol Just FYI, SFTP is an OK protocol hampered by really poor implementations. Daniel Stenberg (author of `curl`) has a really good write-up about why this is the case.[1] -- What it boils down to is SFTP is "chunk" oriented, not file oriented, and most implementations wait for an acknowledgement of each chunk before fetching the next one. The end result is you're artificially band-limited by your latency to the remote host. -- If an implementation (like libssh2) in-flights many chunks at once SFTP can go much faster. Sadly a lot of SFTP clients don't seem to have gotten the memo. [1]: https://daniel.haxx.se/blog/2010/12/08/making-sftp-transfers-fast/ https://daniel.haxx.se/blog/2010/12/08/making-sftp-transfers...
rollcat 8y ago
> I believe that we are getting to a point where config management might as well be in a programming language instead of a bunch of ad-hoc scripts in templates that defer to dynamic language scripts and become a mess. That's the idea behind my brainchild, judo[1]. [1]: https://github.com/rollcat/judo https://github.com/rollcat/judo
- Kimitri 8y ago
  Judo seems like a great automation option for some simpler tasks where Ansible could be considered overkill. For me, that covers almost 100% of my remote automation needs. :) Nice!
  - kchhina 8y ago
    Another infrastructure as code (not code as in YAML, etc.), but programming language 'code' tool is Pulumi[0]. [0] https://www.pulumi.com/ https://www.pulumi.com/

akhilcacharya 8y ago

Is the purpose to use a faster machine for a particular compute job?

kodablah 8y ago
Nope, just to deploy to or configure a remote machine. So you write some Go code as though it's running locally that may run some apt commands, or may extract a tarball, or may setup a systemd script or whatever. Then the tool builds that binary and runs it on that remote machine (fetching resources lazily from the home computer such as the tarball you might need), then it deletes itself. I've used it for multiple server things that require more of an upgrade approach than a ephemeral container approach...I welcome anyone to take the idea and run with it.
- tenken 8y ago
  So it's a virus? ... because it sounds like a virus :D

drbawb 8y ago

>One thing I found is SCP/SFTP is a really slow transfer protocol Just FYI, SFTP is an OK protocol hampered by really poor implementations. Daniel Stenberg (author of `curl`) has a really good write-up about why this is the case.[1] -- What it boils down to is SFTP is "chunk" oriented, not file oriented, and most implementations wait for an acknowledgement of each chunk before fetching the next one. The end result is you're artificially band-limited by your latency to the remote host. -- If an implementation (like libssh2) in-flights many chunks at once SFTP can go much faster. Sadly a lot of SFTP clients don't seem to have gotten the memo. [1]: https://daniel.haxx.se/blog/2010/12/08/making-sftp-transfers-fast/ https://daniel.haxx.se/blog/2010/12/08/making-sftp-transfers...

rollcat 8y ago

> I believe that we are getting to a point where config management might as well be in a programming language instead of a bunch of ad-hoc scripts in templates that defer to dynamic language scripts and become a mess. That's the idea behind my brainchild, judo[1]. [1]: https://github.com/rollcat/judo https://github.com/rollcat/judo

Kimitri 8y ago
Judo seems like a great automation option for some simpler tasks where Ansible could be considered overkill. For me, that covers almost 100% of my remote automation needs. :) Nice!
- kchhina 8y ago
  Another infrastructure as code (not code as in YAML, etc.), but programming language 'code' tool is Pulumi[0]. [0] https://www.pulumi.com/ https://www.pulumi.com/