4 ms·

> Building is as easy as: > go build > Testing is as easy as > go test > Deploying the app is as easy as: > scp app user@host: > ssh use

> Building is as easy as: > go build > Testing is as easy as > go test > Deploying the app is as easy as: > scp app user@host: > ssh user@host “nohup ./app” I suppose we'll see in 1-3 years whether the decision to skip any/all build and deploy tools in favor of staking a business on someone scp'ing to prod servers was a good idea. I have a very strong assumption, but who knows.

SteveNuts 5y ago
In the "our business teeters on the edge of solvency every month" stage of a company lifecycle, does it really matter that much? Plus the article never mentions "we'll never need to add more ci/cd", they're simply saying they don't need it right now.
- simonjgreen 5y ago
  Just to re-enforce this, this article is all about context. It's making it very clear that in this organisations context this method of deployment and production is absolutely fine. And that's the key thing, you can keep things this simple for as long as your risk assessment says so. The key for me is making sure you are taking the time to reassess those early life decisions so when the time comes you can course-adjust
simonjgreen 5y ago
I'm pretty sure we've already answered that question over the last 20-30 years, and it turns out that in most cases: yes, it's absolutely fine.
wizzwizz4 5y ago
Yeah, we should use deploy tools instead. $ cat deploy.sh #!/bin/sh scp app user@host; ssh user@host "nohup ./app" At some point you have to use something to get it to the production server. Why not scp?
- viraptor 5y ago
  Because push instead of pull deployment means reboots and now servers have to be handled manually. Because this way doesn't leave explicit audit/deployment logs.
- void_mint 5y ago
  The reason to use build and deployment tools has nothing to do with _how_ the tools actually work. Deployment issues are mostly issues of consistency and access. Who runs the build/deploy, from where, with what arguments, when? Using a CI/CD pipeline isn't just wrapping an scp call in a bash script, they provide tools for managing the when, the where, the why. Do you think using only scp will prevent you from deploying the wrong artifact? Or deploying to all-but-one production node? Or scping but forgetting to run the nohup? Honestly setting up a github action to do this takes very little time, doesn't depend on containers, doesn't really add complexity to your stack. CircleCI exists and is equally easy. To suggest that your above-posted script is just as good is, to me, mostly just NIH + developer bikeshedding. YAGNI is great for being honest about the needs of an org and ignoring tools or frameworks that aren't appropriate, but there really are tools that exist that add consistency in ways that don't really increase complexity all that much.
Jiejeing 5y ago
Down the road, pulling the binary in a container for deployment is trivial, so I don’t see a downside here. Of course nobody will advocate that scping a binary on each production server for deployment is viable in the long run, but that should not be your primary concern when going through the early phases of a product. Not focusing on docker from the start forces you to have a clean build and runtime software chain, and reduces the amount of magic required (ironically, going to containers was supposed to be the opposite, but many server-side packages are now docker-only because the build and/or runtime requirements are so messed up, e.g. discourse).
- jayd16 5y ago
  I just don't get it. Docker is just not that hard to set up. It takes a day or two and you get a lot of flexibility you can use down the line. Sure you could convert later, but now you have to move your whole workflow over during a time you're in dire need of the new features instead of casually from the start. Now instead of your artifact history being consistent, its truncated at the point you switched over. I guess I don't understand whats so complicated about docker?
  - dijit 5y ago
    Bridge networking, probably. Overlay filesystem performance across distros? Attaching a debugger to a process? Idk. Just some ideas off the top of my head. I don’t think it’s unimaginable to consider how docker is an increase in complexity and understanding.
    - akvadrako 5y ago
      > Bridge networking, probably. You can just use --net=host if you don't like it. > Overlay filesystem performance across distros? I've used docker on several distros and never noticed an issue with this. > Attaching a debugger to a process? This is also not an issue if you don't use bridge networking.
    - trinovantes 5y ago
      Getting VSCode debugger to hook into a process running in a docker container running inside WSL was insanely frustrating. In the end, I stopped using Docker during development and only use it for deploying to staging/production.
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
grogenaut 5y ago
Twitch made it to acquisition with amazon with basically this system for both a rails app and about 20 other go apps. I'd know, I helped transition us to ecs later. For go apps, docker isn't really doing much over golang binaries. The bigger transition was more just moving us away from any unneeded dependence on system level apps. This happened at the same time we were dockerizing. I think the ECS abstraction is better than the simple daemon scripts we were using and scp for deploys, but we made it work up to around 30k servers before we hit scale limits. We could have pretty easily taken it even further forward but ecs is simpler and better integrated with the rest of AWS and our build tools. It only matters for a few special teams. The real issue with SCP at some point was the parallelism of deploys: opening 30k ssh connections single threaded or with bash starts bogging down at some point, and we were at the point of writing a massively parallel version when we switched to ECS for much of this (also around the time ECS became stable).
- void_mint 5y ago
  I posted below, but to me this post isn't really about Docker at all, as are my issues with it. You don't need to use Docker to see the need for sane build and deploy tooling. Who is running this scp? Where do they get the build? Do they have to know which hosts to scp to? Questions/problems like this are not solved at all, which generally need to be solved one way or another to scale any production system.
  - grogenaut 5y ago
    This was years ago. it was bare metal. The answer was that it got the exe from the build machine which it was running from. It knew where to deploy as we were on bare metal and used consul to list the hosts.
dehrmann 5y ago
I'd almost argue that the main benefit to not using docker at a scale where scp works is you won't be tempted to use kubernetes.

SteveNuts 5y ago

In the "our business teeters on the edge of solvency every month" stage of a company lifecycle, does it really matter that much? Plus the article never mentions "we'll never need to add more ci/cd", they're simply saying they don't need it right now.

simonjgreen 5y ago
Just to re-enforce this, this article is all about context. It's making it very clear that in this organisations context this method of deployment and production is absolutely fine. And that's the key thing, you can keep things this simple for as long as your risk assessment says so. The key for me is making sure you are taking the time to reassess those early life decisions so when the time comes you can course-adjust

simonjgreen 5y ago

I'm pretty sure we've already answered that question over the last 20-30 years, and it turns out that in most cases: yes, it's absolutely fine.

wizzwizz4 5y ago

Yeah, we should use deploy tools instead. $ cat deploy.sh #!/bin/sh scp app user@host; ssh user@host "nohup ./app" At some point you have to use something to get it to the production server. Why not scp?

viraptor 5y ago
Because push instead of pull deployment means reboots and now servers have to be handled manually. Because this way doesn't leave explicit audit/deployment logs.
void_mint 5y ago
The reason to use build and deployment tools has nothing to do with _how_ the tools actually work. Deployment issues are mostly issues of consistency and access. Who runs the build/deploy, from where, with what arguments, when? Using a CI/CD pipeline isn't just wrapping an scp call in a bash script, they provide tools for managing the when, the where, the why. Do you think using only scp will prevent you from deploying the wrong artifact? Or deploying to all-but-one production node? Or scping but forgetting to run the nohup? Honestly setting up a github action to do this takes very little time, doesn't depend on containers, doesn't really add complexity to your stack. CircleCI exists and is equally easy. To suggest that your above-posted script is just as good is, to me, mostly just NIH + developer bikeshedding. YAGNI is great for being honest about the needs of an org and ignoring tools or frameworks that aren't appropriate, but there really are tools that exist that add consistency in ways that don't really increase complexity all that much.

Jiejeing 5y ago

Down the road, pulling the binary in a container for deployment is trivial, so I don’t see a downside here. Of course nobody will advocate that scping a binary on each production server for deployment is viable in the long run, but that should not be your primary concern when going through the early phases of a product. Not focusing on docker from the start forces you to have a clean build and runtime software chain, and reduces the amount of magic required (ironically, going to containers was supposed to be the opposite, but many server-side packages are now docker-only because the build and/or runtime requirements are so messed up, e.g. discourse).

jayd16 5y ago
I just don't get it. Docker is just not that hard to set up. It takes a day or two and you get a lot of flexibility you can use down the line. Sure you could convert later, but now you have to move your whole workflow over during a time you're in dire need of the new features instead of casually from the start. Now instead of your artifact history being consistent, its truncated at the point you switched over. I guess I don't understand whats so complicated about docker?
- dijit 5y ago
  Bridge networking, probably. Overlay filesystem performance across distros? Attaching a debugger to a process? Idk. Just some ideas off the top of my head. I don’t think it’s unimaginable to consider how docker is an increase in complexity and understanding.
  - akvadrako 5y ago
    > Bridge networking, probably. You can just use --net=host if you don't like it. > Overlay filesystem performance across distros? I've used docker on several distros and never noticed an issue with this. > Attaching a debugger to a process? This is also not an issue if you don't use bridge networking.
  - trinovantes 5y ago
    Getting VSCode debugger to hook into a process running in a docker container running inside WSL was insanely frustrating. In the end, I stopped using Docker during development and only use it for deploying to staging/production.

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

grogenaut 5y ago

Twitch made it to acquisition with amazon with basically this system for both a rails app and about 20 other go apps. I'd know, I helped transition us to ecs later. For go apps, docker isn't really doing much over golang binaries. The bigger transition was more just moving us away from any unneeded dependence on system level apps. This happened at the same time we were dockerizing. I think the ECS abstraction is better than the simple daemon scripts we were using and scp for deploys, but we made it work up to around 30k servers before we hit scale limits. We could have pretty easily taken it even further forward but ecs is simpler and better integrated with the rest of AWS and our build tools. It only matters for a few special teams. The real issue with SCP at some point was the parallelism of deploys: opening 30k ssh connections single threaded or with bash starts bogging down at some point, and we were at the point of writing a massively parallel version when we switched to ECS for much of this (also around the time ECS became stable).

void_mint 5y ago
I posted below, but to me this post isn't really about Docker at all, as are my issues with it. You don't need to use Docker to see the need for sane build and deploy tooling. Who is running this scp? Where do they get the build? Do they have to know which hosts to scp to? Questions/problems like this are not solved at all, which generally need to be solved one way or another to scale any production system.
- grogenaut 5y ago
  This was years ago. it was bare metal. The answer was that it got the exe from the build machine which it was running from. It knew where to deploy as we were on bare metal and used consul to list the hosts.

dehrmann 5y ago

I'd almost argue that the main benefit to not using docker at a scale where scp works is you won't be tempted to use kubernetes.