5 ms·

That's a very cool deployment method to just use rsync like that! Simple and very composable. And now I feel self-conscious about my pile of AWS and Docker!

toast0 5y ago
It's 'easy' to do it with rsync and Make as long as you don't have a lot of dependencies. Once you start pulling in dependencies, you have to figure out how to get them installed, hopefully before you roll code that uses them. If the dependencies are small, you can pull them into your source (more or less) and deploy them that way, but that may make tracking upstream harder. (otoh, a lot of things I personally used like that were more of pull this probably abandoned thing that's close to what we need, and modify it quite a bit to fit our needs; there wasn't a need or desire to follow upstream after that)
- tiffanyh 5y ago
  Just curious, would your deployment method be any different today? If so, how?
  - toast0 5y ago
    If I was going to do it again, I would probably work on a user management script a lot sooner (you don't really need it, but it would have been nice, and the earlier you do it, the less crufty differences need to be tolerated) and maybe make build on one host, deploy on many or at least push from workstation to colo once and fan out; might have been nice to have that around 100 hosts instead of much later. Of course, today, my fleet is down to less than ten hosts, depending on exactly how you count, and I'm usually the only user, so I can do whatever.
- dolmen 5y ago
  rsync+make is the deployment+run. But this assumes the source of rsync is already static. Handling dependencies is done in a build process, upstream. Fetching dependencies at deployment time is a direct path to deployment failures (especially if the dependency management relies on external resources or non-locked versionning).
motoboi 5y ago
Containers are just tar.gz files, you know? The whole layers thing it’s just an optimization. You can actually very simply run those tar.gz files without docker involved, just cgroups. But then you’ll have to write some daemon scripts to start, stop, restart, etc Follow this path and soon you’ll have a (worst) custom docker. Try to create a network out of those containers and soon a (worst) SDN network appears. Try to expand that to optimal node usage and soon a (worst) Kubernetes appears. My point here is: it’s just software packaged with their dependencies. The rest seems inconsequential, but it’s actually the hard part.
- lamontcg 5y ago
  "worst" might not really be all that bad. K8s may be the "worst" orchestrator for you, and you may not actually need all or any of that complexity/functionality. But starting with containers viewed as an RPM/tarball with some extra sauce (union filesystem and cgroups/jail) is a way better mental model than the whole "immutable infrastructure" meme to me.
- z3t4 5y ago
  You can do a lot with just SystemD today. Would not get surprised if SystemD would eat up Docker too.
  - Art9681 5y ago
    I'm personally interested in Unikernels. That tech seems to be a good contender for replacing K8s. Every app an appliance running in a hypervisor.
    - dolmen 5y ago
      I even imagine a GOOS runtime for compiling a Go program that would talk directly to the hypervisor... Many services I write use only network resources, no disk.
- treesknees 5y ago
  This is essentially what fly.io does, they unpack the container image into a micro-VM and execute it directly. Calling it worse is subjective, you certainly need to account for things you would never worry about by using traditional Docker. But for some companies it ends up working much better than Docker. https://fly.io/blog/docker-without-docker/ https://fly.io/blog/docker-without-docker/

toast0 5y ago

It's 'easy' to do it with rsync and Make as long as you don't have a lot of dependencies. Once you start pulling in dependencies, you have to figure out how to get them installed, hopefully before you roll code that uses them. If the dependencies are small, you can pull them into your source (more or less) and deploy them that way, but that may make tracking upstream harder. (otoh, a lot of things I personally used like that were more of pull this probably abandoned thing that's close to what we need, and modify it quite a bit to fit our needs; there wasn't a need or desire to follow upstream after that)

tiffanyh 5y ago
Just curious, would your deployment method be any different today? If so, how?
- toast0 5y ago
  If I was going to do it again, I would probably work on a user management script a lot sooner (you don't really need it, but it would have been nice, and the earlier you do it, the less crufty differences need to be tolerated) and maybe make build on one host, deploy on many or at least push from workstation to colo once and fan out; might have been nice to have that around 100 hosts instead of much later. Of course, today, my fleet is down to less than ten hosts, depending on exactly how you count, and I'm usually the only user, so I can do whatever.
dolmen 5y ago
rsync+make is the deployment+run. But this assumes the source of rsync is already static. Handling dependencies is done in a build process, upstream. Fetching dependencies at deployment time is a direct path to deployment failures (especially if the dependency management relies on external resources or non-locked versionning).

motoboi 5y ago

Containers are just tar.gz files, you know? The whole layers thing it’s just an optimization. You can actually very simply run those tar.gz files without docker involved, just cgroups. But then you’ll have to write some daemon scripts to start, stop, restart, etc Follow this path and soon you’ll have a (worst) custom docker. Try to create a network out of those containers and soon a (worst) SDN network appears. Try to expand that to optimal node usage and soon a (worst) Kubernetes appears. My point here is: it’s just software packaged with their dependencies. The rest seems inconsequential, but it’s actually the hard part.

lamontcg 5y ago
"worst" might not really be all that bad. K8s may be the "worst" orchestrator for you, and you may not actually need all or any of that complexity/functionality. But starting with containers viewed as an RPM/tarball with some extra sauce (union filesystem and cgroups/jail) is a way better mental model than the whole "immutable infrastructure" meme to me.
z3t4 5y ago
You can do a lot with just SystemD today. Would not get surprised if SystemD would eat up Docker too.
- Art9681 5y ago
  I'm personally interested in Unikernels. That tech seems to be a good contender for replacing K8s. Every app an appliance running in a hypervisor.
  - dolmen 5y ago
    I even imagine a GOOS runtime for compiling a Go program that would talk directly to the hypervisor... Many services I write use only network resources, no disk.
treesknees 5y ago
This is essentially what fly.io does, they unpack the container image into a micro-VM and execute it directly. Calling it worse is subjective, you certainly need to account for things you would never worry about by using traditional Docker. But for some companies it ends up working much better than Docker. https://fly.io/blog/docker-without-docker/ https://fly.io/blog/docker-without-docker/