9 ms·

I wouldn't use docker even as an established company at this point. I've been using it for several years and it adds complexity rather than removing it in the l

by hughrr 5y ago

I wouldn't use docker even as an established company at this point. I've been using it for several years and it adds complexity rather than removing it in the long run. And yes I include Kubernetes in that as well. They work very well for specific narrow sets of requirements but are unnecessary and require headspace and headcount to look after. It's better to religiously build simplicity, repeatability and fast automation into everything you do without it. That has a much better ROI on a product build because you have to do all those things anyway to make docker and kubernetes effective. The key thing to do is to build on technology that is portable, compact and repeatable which means at the moment something entirely self-contained (Go, .Net Core, Java, Python with some caveats come to mind)

PaywallBuster 5y ago
Likewise, I'd also prefer not jumping on the hype and have tried to avoid K8s so far, but Something I'd take docker is: deployment speed. Baking images and deploying new ASG makes production deployments take around ~ 15 min. Docker based solution could probably cut it down to 5 mins. Also more inclined to use a simpler/hands off service like ECS Fargate than straight kubernetes. Also seems like over half the jobs being currently posted require k8s skills, might be difficult to avoid it for much longer
- hughrr 5y ago
  I am doing blue/green deployments with ansible in 30 seconds from package deploy across 20 nodes. YMMV but docker doesn't inherently make anything faster itself.
  - stfp 5y ago
    Out of curiosity, what kind of artifacts are you deploying and how are you building them?
    - hughrr 5y ago
      Build in Jenkins. self-contained ReadyToRun .Net core apps. Couple of smaller Go apps. I rather like Go. Target package contains one file! Config is inherited from the environment and cloud provider as the targets are Amazon Linux 2 based.
      - ecnahc515 5y ago
        Compiled executables are pretty simple to deploy without containers. The benefit of containers is that it can help standardize that type of benefit to other runtimes like Python/Ruby where it's much more challenging to just ship an artifact. In addition to that, with cgroups and namespaces you get a lot of other benefits, which can be used with eg: systemd/etc but it's nice when one tool handles all of it.
        hughrr 5y ago
        Yes exactly. It's a solution to a problem. My point is it is better to avoid the problem than employ a solution.
        meowface 5y ago
        It's not easy when the only way to avoid the problem you describe is to rewrite your whole project(s) in a language like Go. I like Go and have used standalone binaries instead of Docker containers when deploying Go applications, but I don't always work with Go applications. As other posters said, Docker largely exists to help with any situation where you don't have a single binary. These situations are still very common for many developers and devops engineers. And in those cases, Docker can be very helpful.
  - PaywallBuster 5y ago
    each VM can take 1-2 min to deploy launch vm: 1-2 min config/deploy: 3-5 min (ansible) create image: 2/3 min launch new asg: 2-5 min Containers greatly reduce most of these steps
    - hughrr 5y ago
      We just leave the instances running. Why burn the universe a thousand times a day?
      - PaywallBuster 5y ago
        that's one of the reasons to pay for cloud providers. You can adjust your paid resources to actual usage. This is one way to do it. (search Immutable Infrastructure) For this application, traffic is quite stable so it doesn't come up often. But once or twice, we had some bug causing performance regressions. They were not visible on the first day, but as usage grew over time the cluster started growing with the problem. So it allowed the application to stay online (at an additional cost) until devs pick up the problem and have a solution ready, probably coming up in the following sprint only.
        hughrr 5y ago
        yeah aware of that. We do that as well. But scheduled scaling.
    - deleted 5y ago
      [deleted]
- bryanrasmussen 5y ago
  >Baking images and deploying new ASG makes production deployments take around ~ 15 min. >Docker based solution could probably cut it down to 5 mins. this reminds me of when I was working on the Danish government electronic invoicing project. The political driver of the project had made some statement along the line that a minute was going to be saved on every invoice sent (which I mean there would be millions and millions of minutes saved every month), and someone else made the observation in my hearing that he doubted that time saved would be monetizable. Which I took to mean that when small amounts of time are saved those are likely just swallowed by meaningless other tasks and not used effectively. I doubt the 10 minutes saved per deploy by using Docker would be monetizable.
  - PaywallBuster 5y ago
    probably )) For our biggest app, Drupal, deployments are kind of hands on. Dev(s) are waiting for the deployment, do some post deployment steps, check logs for any issues, make announcements, etc and if we had to rollback, the easy solution is to find the previous deployment and run it again, speed would be probably be beneficial here
  - rahimnathwani 5y ago
    When they talked about time saved, were they referring to: A) effort (aka 'man-hours'), or B) duration (aka 'wall clock time') ?
- hashkb 5y ago
  Fargate provisioning is really slow. Minute or two at best, in my experience. I went back to ASG and Codebuild. Packer for AMI rebuilds.
  - kcartlidge 5y ago
    +1 Packer doesn't get enough love.
    - hughrr 5y ago
      Try building windows images on it. That will balance the love out a bit :)
j45 5y ago
Bash scripts are very capable these days… and for many years.
- intev 5y ago
  I think the whole point of this exercise is to find out where to draw the line in terms of abstracting out deployment steps and increasing deployment speed. The vast majority will agree that it is NOT at bash scripts. Once you get to a few micro services, good luck maintaining that. C++ has been amazingly powerful for years, but there's a reason popularity isn't exploding...
  - j45 5y ago
    Bash scripts are pretty good in the beginning. I have scaled systems and they remain helpful as scale is built. A lot of bash commands are automated in Devops tools too.
- void_mint 5y ago
  Why not build everything in Assembly?
  - Brian_K_White 5y ago
    Oh let me clear that up for you. This article and this comment discussion assumes a certain familiarity with computers and system administration, which, to ask a question like that, you must not have. So a bash script is this thing that's very functional, flexible, legible, can be written quickly and easily, can be maintained and updated quickly and easily, and is even portable across platforms, and it gives you all that without a teetering fragile tower of library and framework dependencies that most other automation systems rely on. Every system already has bash or some equivalent Bourne shell for free. Assembly has none of those qualities. Are there any other mysteries we can clear up? I mean we all had to start somewhere and "there are no stupid questions" right?
    - void_mint 5y ago
mountainriver 5y ago
I find this kinda insane. The days before docker I would spend a lot of time trying to reproduce environments. Docker isn’t hard to use and doesn’t add much complexity, but the payoff is huge in terms of having portable environments.
- slindz 5y ago
  Some languages definitely benefit from portable environments, others don't really need it. My anecdata is that horsing around with Docker was still needed and when things went wrong, they went REALLY wrong. As with all things, figure out what you need, pick the right tool for the job, and all that good stuff.
- hughrr 5y ago
  Try producing two kubernetes clusters with exactly the same configuration with full management and monitoring stack and you will discover how difficult it is to scale this complexity up. Docker works for most trivial cases but for complex systems it becomes a shit show very quickly which is hard to reason about when you need to, which is whenever it is inevitably broken.
  - xupybd 5y ago
    Yes but a legacy app with no need for kubernetes scale gains huge benefits from docker. When you have to compile your libs from source because they are so old no packages exist anymore, then docker greatly aids both development and deployment.
  - meowface 5y ago
    A lot of Docker criticism I've seen on HN and elsewhere equivocates between Docker and Kubernetes. I've never used Kubernetes, but I use Docker very frequently and find it a massive improvement over the previous state of affairs. Kubernetes does seem complex and difficult to learn, but Docker doesn't.
  - mountainriver 5y ago
    I do this every day and have done it for the last 5 years. It’s far more sane than any alternatives I’ve seen
maccard 5y ago
How do you deploy a golang app? How do you handle rollbacks? Maybe not applicable but how do you scale up? The advantage of something like docker is it unlocks tools like k8s, ecs, azure container instances, DO app platform etc.
- hughrr 5y ago
  deploy = basically scp and do a blue/green switch over. rollbacks = deploy the last version Scale up = add some more instances and redeploy the current version (not an issue as our load is fairly consistent) All of those tools, which I have used extensively, increase complexity tenfold. Just considering kubernetes resource allocation and CNI is an art which requires full time staff to manage. Fargate is the only reasonable abstraction I have seen so far on this front.
  - maccard 5y ago
    I'm actually going to skip most of your reply to this: > Fargate is the only reasonable abstraction I have seen so far on this front. This is the starting point (actually Azure [0] and DigitalOcean [1] are even simpler), not setting up an autoscaling group of ec2 instances and considering resource usage. If you build on top of any of those three platforms, you write a dockerfile, write a service file, and point DNS at a load balancer and you're done. [0] https://docs.microsoft.com/en-gb/azure/container-instances/ https://docs.microsoft.com/en-gb/azure/container-instances/ [1] https://docs.digitalocean.com/products/app-platform/ https://docs.digitalocean.com/products/app-platform/
croo 5y ago
> The key thing to do is to build on technology that is portable, compact and repeatable You mean like docker?... I agree with the k8s part though. Commiting your enviroment to git and reproducing it anywhere is great. Orchestrating thousands of that is horrible and adds too many extra layers.
brundolf 5y ago
I would remove Python from this list. Getting Python services that I don't directly work on running on my dev machine is the only case where Docker has been a lifesaver for me. Not having to deal with Python's insane amount of environment dependency (even with virtualenvs and all that business) is worth the bulk of running a Linux VM on my Mac. If I didn't know better, I'd say Python was the entire reason technologies like Docker were invented in the first place.
- hughrr 5y ago
  Yes hence python “with some limitations” was mentioned.
hsn915 5y ago
> It's better to religiously build simplicity, repeatability and fast automation into everything you do Yea. Totally agree. This should be common sense. Unfortunately it's not. > The key thing to do is to build on technology that is portable, compact and repeatable which means at the moment something entirely self-contained (Go, .Net Core, Java, Python with some caveats come to mind) Is there really anything other than Go that can achieve this? .Net/Java/Python all require the environment to be setup a certain way. And you have the problem of managing versions. Does python even let you produce a self-contained application?
- hughrr 5y ago
  Yes. Use pyinstaller.

by hughrr 5y ago

PaywallBuster 5y ago

Likewise, I'd also prefer not jumping on the hype and have tried to avoid K8s so far, but Something I'd take docker is: deployment speed. Baking images and deploying new ASG makes production deployments take around ~ 15 min. Docker based solution could probably cut it down to 5 mins. Also more inclined to use a simpler/hands off service like ECS Fargate than straight kubernetes. Also seems like over half the jobs being currently posted require k8s skills, might be difficult to avoid it for much longer

hughrr 5y ago
I am doing blue/green deployments with ansible in 30 seconds from package deploy across 20 nodes. YMMV but docker doesn't inherently make anything faster itself.
- stfp 5y ago
  Out of curiosity, what kind of artifacts are you deploying and how are you building them?
  - hughrr 5y ago
    Build in Jenkins. self-contained ReadyToRun .Net core apps. Couple of smaller Go apps. I rather like Go. Target package contains one file! Config is inherited from the environment and cloud provider as the targets are Amazon Linux 2 based.
    - ecnahc515 5y ago
      Compiled executables are pretty simple to deploy without containers. The benefit of containers is that it can help standardize that type of benefit to other runtimes like Python/Ruby where it's much more challenging to just ship an artifact. In addition to that, with cgroups and namespaces you get a lot of other benefits, which can be used with eg: systemd/etc but it's nice when one tool handles all of it.
      - hughrr 5y ago
        Yes exactly. It's a solution to a problem. My point is it is better to avoid the problem than employ a solution.
        meowface 5y ago
        It's not easy when the only way to avoid the problem you describe is to rewrite your whole project(s) in a language like Go. I like Go and have used standalone binaries instead of Docker containers when deploying Go applications, but I don't always work with Go applications. As other posters said, Docker largely exists to help with any situation where you don't have a single binary. These situations are still very common for many developers and devops engineers. And in those cases, Docker can be very helpful.
- PaywallBuster 5y ago
  each VM can take 1-2 min to deploy launch vm: 1-2 min config/deploy: 3-5 min (ansible) create image: 2/3 min launch new asg: 2-5 min Containers greatly reduce most of these steps
  - hughrr 5y ago
    We just leave the instances running. Why burn the universe a thousand times a day?
    - PaywallBuster 5y ago
      that's one of the reasons to pay for cloud providers. You can adjust your paid resources to actual usage. This is one way to do it. (search Immutable Infrastructure) For this application, traffic is quite stable so it doesn't come up often. But once or twice, we had some bug causing performance regressions. They were not visible on the first day, but as usage grew over time the cluster started growing with the problem. So it allowed the application to stay online (at an additional cost) until devs pick up the problem and have a solution ready, probably coming up in the following sprint only.
      - hughrr 5y ago
        yeah aware of that. We do that as well. But scheduled scaling.
  - deleted 5y ago
    [deleted]
bryanrasmussen 5y ago
>Baking images and deploying new ASG makes production deployments take around ~ 15 min. >Docker based solution could probably cut it down to 5 mins. this reminds me of when I was working on the Danish government electronic invoicing project. The political driver of the project had made some statement along the line that a minute was going to be saved on every invoice sent (which I mean there would be millions and millions of minutes saved every month), and someone else made the observation in my hearing that he doubted that time saved would be monetizable. Which I took to mean that when small amounts of time are saved those are likely just swallowed by meaningless other tasks and not used effectively. I doubt the 10 minutes saved per deploy by using Docker would be monetizable.
- PaywallBuster 5y ago
  probably )) For our biggest app, Drupal, deployments are kind of hands on. Dev(s) are waiting for the deployment, do some post deployment steps, check logs for any issues, make announcements, etc and if we had to rollback, the easy solution is to find the previous deployment and run it again, speed would be probably be beneficial here
- rahimnathwani 5y ago
  When they talked about time saved, were they referring to: A) effort (aka 'man-hours'), or B) duration (aka 'wall clock time') ?
hashkb 5y ago
Fargate provisioning is really slow. Minute or two at best, in my experience. I went back to ASG and Codebuild. Packer for AMI rebuilds.
- kcartlidge 5y ago
  +1 Packer doesn't get enough love.
  - hughrr 5y ago
    Try building windows images on it. That will balance the love out a bit :)

j45 5y ago

Bash scripts are very capable these days… and for many years.

intev 5y ago
I think the whole point of this exercise is to find out where to draw the line in terms of abstracting out deployment steps and increasing deployment speed. The vast majority will agree that it is NOT at bash scripts. Once you get to a few micro services, good luck maintaining that. C++ has been amazingly powerful for years, but there's a reason popularity isn't exploding...
- j45 5y ago
  Bash scripts are pretty good in the beginning. I have scaled systems and they remain helpful as scale is built. A lot of bash commands are automated in Devops tools too.
void_mint 5y ago
Why not build everything in Assembly?
- Brian_K_White 5y ago
  Oh let me clear that up for you. This article and this comment discussion assumes a certain familiarity with computers and system administration, which, to ask a question like that, you must not have. So a bash script is this thing that's very functional, flexible, legible, can be written quickly and easily, can be maintained and updated quickly and easily, and is even portable across platforms, and it gives you all that without a teetering fragile tower of library and framework dependencies that most other automation systems rely on. Every system already has bash or some equivalent Bourne shell for free. Assembly has none of those qualities. Are there any other mysteries we can clear up? I mean we all had to start somewhere and "there are no stupid questions" right?
  - void_mint 5y ago

mountainriver 5y ago

I find this kinda insane. The days before docker I would spend a lot of time trying to reproduce environments. Docker isn’t hard to use and doesn’t add much complexity, but the payoff is huge in terms of having portable environments.

slindz 5y ago
Some languages definitely benefit from portable environments, others don't really need it. My anecdata is that horsing around with Docker was still needed and when things went wrong, they went REALLY wrong. As with all things, figure out what you need, pick the right tool for the job, and all that good stuff.
hughrr 5y ago
Try producing two kubernetes clusters with exactly the same configuration with full management and monitoring stack and you will discover how difficult it is to scale this complexity up. Docker works for most trivial cases but for complex systems it becomes a shit show very quickly which is hard to reason about when you need to, which is whenever it is inevitably broken.
- xupybd 5y ago
  Yes but a legacy app with no need for kubernetes scale gains huge benefits from docker. When you have to compile your libs from source because they are so old no packages exist anymore, then docker greatly aids both development and deployment.
- meowface 5y ago
  A lot of Docker criticism I've seen on HN and elsewhere equivocates between Docker and Kubernetes. I've never used Kubernetes, but I use Docker very frequently and find it a massive improvement over the previous state of affairs. Kubernetes does seem complex and difficult to learn, but Docker doesn't.
- mountainriver 5y ago
  I do this every day and have done it for the last 5 years. It’s far more sane than any alternatives I’ve seen

maccard 5y ago

How do you deploy a golang app? How do you handle rollbacks? Maybe not applicable but how do you scale up? The advantage of something like docker is it unlocks tools like k8s, ecs, azure container instances, DO app platform etc.

hughrr 5y ago
deploy = basically scp and do a blue/green switch over. rollbacks = deploy the last version Scale up = add some more instances and redeploy the current version (not an issue as our load is fairly consistent) All of those tools, which I have used extensively, increase complexity tenfold. Just considering kubernetes resource allocation and CNI is an art which requires full time staff to manage. Fargate is the only reasonable abstraction I have seen so far on this front.
- maccard 5y ago
  I'm actually going to skip most of your reply to this: > Fargate is the only reasonable abstraction I have seen so far on this front. This is the starting point (actually Azure [0] and DigitalOcean [1] are even simpler), not setting up an autoscaling group of ec2 instances and considering resource usage. If you build on top of any of those three platforms, you write a dockerfile, write a service file, and point DNS at a load balancer and you're done. [0] https://docs.microsoft.com/en-gb/azure/container-instances/ https://docs.microsoft.com/en-gb/azure/container-instances/ [1] https://docs.digitalocean.com/products/app-platform/ https://docs.digitalocean.com/products/app-platform/

croo 5y ago

> The key thing to do is to build on technology that is portable, compact and repeatable You mean like docker?... I agree with the k8s part though. Commiting your enviroment to git and reproducing it anywhere is great. Orchestrating thousands of that is horrible and adds too many extra layers.

brundolf 5y ago

I would remove Python from this list. Getting Python services that I don't directly work on running on my dev machine is the only case where Docker has been a lifesaver for me. Not having to deal with Python's insane amount of environment dependency (even with virtualenvs and all that business) is worth the bulk of running a Linux VM on my Mac. If I didn't know better, I'd say Python was the entire reason technologies like Docker were invented in the first place.

hughrr 5y ago
Yes hence python “with some limitations” was mentioned.

hsn915 5y ago

> It's better to religiously build simplicity, repeatability and fast automation into everything you do Yea. Totally agree. This should be common sense. Unfortunately it's not. > The key thing to do is to build on technology that is portable, compact and repeatable which means at the moment something entirely self-contained (Go, .Net Core, Java, Python with some caveats come to mind) Is there really anything other than Go that can achieve this? .Net/Java/Python all require the environment to be setup a certain way. And you have the problem of managing versions. Does python even let you produce a self-contained application?

hughrr 5y ago
Yes. Use pyinstaller.