6 ms·

Hey just curious: With a dockerfile is there any point to using Ansible/Puppet/Chef any more? I have used those tools to keep 'how machines are installed' in c

Hey just curious: With a dockerfile is there any point to using Ansible/Puppet/Chef any more? I have used those tools to keep 'how machines are installed' in code in the past, but with dockerfiles these days I don't get the point anymore. Seems like the future will be having a few dockerfiles (staging, prod, dev, db, etc) then just finding a place to run them (digital ocean, docker swarm, aws container service - or your macbook).

dullgiulio 9y ago
I don't think it's a good idea to have different dockerfiles for your staging, dev and prod environments. The point of using Docker containers is that you get to deploy to production your dev environment as is.
bdcravens 9y ago
You still have to bootstrap the instances/droplets/etc with Docker, Kubernetes, etc before Dockerfiles can be used. Unless you're starting with images where those are already baked in, Ansible/Chef/Puppet are good ways to accomplish this on a clean OS.
- zimbatm 9y ago
  It's debatable. Ansible/Chef/Puppet themselves need to be bootstrapped as well. Bash is perfectly fine if the machine can be installed with less than 100 lines of code and immutable infrastructure is being used (so no need for idempotent operations). Just make sure to add ShellCheck to the CI pipeline.
  - icebraining 9y ago
    Ansible/Chef/Puppet themselves need to be bootstrapped as well. Nope, Ansible just needs SSH access.
    - Whitestrake 9y ago
      Remote hosts need Python, but you can use the raw command functionality to figure out what the remote package manager is and install Python as required.
    - zimbatm 9y ago
      Not when using AutoScaling Groups. Then the machine will be provisioned on boot and therefor ansible would have to be installed on the machine.
      - icebraining 9y ago
        No, you'd use Ansible to prepare the new image once, then you'd use that image to scale. Bootstrapping everything each time a VM is created doesn't make much sense.
  - tdurden 9y ago
    > Bash is perfectly fine if the machine can be installed with less than 100 lines of code It is always tough to predict the future; something < 100 lines can easily grow over time. It is nice to start with a tool like Chef/Ansible/Puppet which can more easily grow as the project grows vs Bash.
    - zimbatm 9y ago
      I agree that it's hard to predict the future, that's why I'm not trying to add unnecessary dependencies. 100 lines of code are not hard to rewrite in anything else if necessary.
btmiller 9y ago
For single-provider solutions, I'd say that's debatable. You might be fine with <insert cloud provider here>'s PaaS spin-up services for Kubernetes. The instant you want to span multiple cloud providers, you'll rub up against all the minor differences between each of their implementations, and I think that's where the value in SaltStack/Ansible/Chef/Puppet is in modern times. Manually orchestrate your infrastructure once, then deploy it to any cloud provider.
manigandham 9y ago
No, you don't need those tools anymore. Cook a base VM image that has the docker engine and anything else you need. Do this once and spin up VMs you need after that. This is what images are for, and they're far easier, faster, more reliable and more secure than dealing with all this other junk.
- icebraining 9y ago
  How do you do the cooking of the base VM image?
  - sevagh 9y ago
    Check out Packer (https://www.packer.io/intro/ https://www.packer.io/intro/)
  - manigandham 9y ago
    All the major clouds already have API commands to convert a VM into a reusable image - so a simple startup script + this API command is all you need. Even better since you can script all this too. If anything changes, update the startup script, then run this "make new base image" script and you're all set. Update VMs as needed. Really doesn't get much more complicated than that and using tools like ansible just to setup your VMs so you can run docker containers later is unnecessary.
    - icebraining 9y ago
      You make Ansible sound like an ICBM launching system. In practice, you do a simple "apt install ansible" once (on your machine) and then you write a simple startup script in it. There's nothing extra to be "unnecessary", it's a 1:1 replacement.
      - manigandham 9y ago
        Why do you need that then when you can just use the startup script directly with the VM? Unnecessary = not needed to function. Replacing functionality is just preference, which is fine, but specifically not necessary.
zimbatm 9y ago
I'm doing just that (for a SaaS). 1. Build an AMI with Packer that contains Ubuntu + Docker + aws-cli. 2. Build and publish docker images for all the services that will run on the instances. (one image per service, not per environment like you suggested) 3. Configure the EC2 instance or AutoScaling Groups with userdata. The userdata usually just contains a `docker run ...` instruction. I used to have Chef, run test kitchen, bake AMIs, cycle all the machines. Now I have a single AMI and do most of the testing on the docker side and then some on staging. In both cases I am using terraform to manage the overall infrastructure (and userdata scripts). The only issue is how to manage secrets as userdata are readable with the DescribeInstance ACL. The best solution that I found is to upload the secrets on S3+KMS and use IAM Role on the machine to retrieve them. Or use Hashicorp Vault.
riquito 9y ago
I moved away from dockerfiles to Packer to build images for multiple environments. Even if you don't need that, ansible from dockerfile is a good choice to avoid writing hard to manage/reuse scripts in yml

dullgiulio 9y ago

I don't think it's a good idea to have different dockerfiles for your staging, dev and prod environments. The point of using Docker containers is that you get to deploy to production your dev environment as is.

bdcravens 9y ago

You still have to bootstrap the instances/droplets/etc with Docker, Kubernetes, etc before Dockerfiles can be used. Unless you're starting with images where those are already baked in, Ansible/Chef/Puppet are good ways to accomplish this on a clean OS.

zimbatm 9y ago
It's debatable. Ansible/Chef/Puppet themselves need to be bootstrapped as well. Bash is perfectly fine if the machine can be installed with less than 100 lines of code and immutable infrastructure is being used (so no need for idempotent operations). Just make sure to add ShellCheck to the CI pipeline.
- icebraining 9y ago
  Ansible/Chef/Puppet themselves need to be bootstrapped as well. Nope, Ansible just needs SSH access.
  - Whitestrake 9y ago
    Remote hosts need Python, but you can use the raw command functionality to figure out what the remote package manager is and install Python as required.
  - zimbatm 9y ago
    Not when using AutoScaling Groups. Then the machine will be provisioned on boot and therefor ansible would have to be installed on the machine.
    - icebraining 9y ago
      No, you'd use Ansible to prepare the new image once, then you'd use that image to scale. Bootstrapping everything each time a VM is created doesn't make much sense.
- tdurden 9y ago
  > Bash is perfectly fine if the machine can be installed with less than 100 lines of code It is always tough to predict the future; something < 100 lines can easily grow over time. It is nice to start with a tool like Chef/Ansible/Puppet which can more easily grow as the project grows vs Bash.
  - zimbatm 9y ago
    I agree that it's hard to predict the future, that's why I'm not trying to add unnecessary dependencies. 100 lines of code are not hard to rewrite in anything else if necessary.

btmiller 9y ago

For single-provider solutions, I'd say that's debatable. You might be fine with <insert cloud provider here>'s PaaS spin-up services for Kubernetes. The instant you want to span multiple cloud providers, you'll rub up against all the minor differences between each of their implementations, and I think that's where the value in SaltStack/Ansible/Chef/Puppet is in modern times. Manually orchestrate your infrastructure once, then deploy it to any cloud provider.

manigandham 9y ago

No, you don't need those tools anymore. Cook a base VM image that has the docker engine and anything else you need. Do this once and spin up VMs you need after that. This is what images are for, and they're far easier, faster, more reliable and more secure than dealing with all this other junk.

icebraining 9y ago
How do you do the cooking of the base VM image?
- sevagh 9y ago
  Check out Packer (https://www.packer.io/intro/ https://www.packer.io/intro/)
- manigandham 9y ago
  All the major clouds already have API commands to convert a VM into a reusable image - so a simple startup script + this API command is all you need. Even better since you can script all this too. If anything changes, update the startup script, then run this "make new base image" script and you're all set. Update VMs as needed. Really doesn't get much more complicated than that and using tools like ansible just to setup your VMs so you can run docker containers later is unnecessary.
  - icebraining 9y ago
    You make Ansible sound like an ICBM launching system. In practice, you do a simple "apt install ansible" once (on your machine) and then you write a simple startup script in it. There's nothing extra to be "unnecessary", it's a 1:1 replacement.
    - manigandham 9y ago
      Why do you need that then when you can just use the startup script directly with the VM? Unnecessary = not needed to function. Replacing functionality is just preference, which is fine, but specifically not necessary.

zimbatm 9y ago

I'm doing just that (for a SaaS). 1. Build an AMI with Packer that contains Ubuntu + Docker + aws-cli. 2. Build and publish docker images for all the services that will run on the instances. (one image per service, not per environment like you suggested) 3. Configure the EC2 instance or AutoScaling Groups with userdata. The userdata usually just contains a `docker run ...` instruction. I used to have Chef, run test kitchen, bake AMIs, cycle all the machines. Now I have a single AMI and do most of the testing on the docker side and then some on staging. In both cases I am using terraform to manage the overall infrastructure (and userdata scripts). The only issue is how to manage secrets as userdata are readable with the DescribeInstance ACL. The best solution that I found is to upload the secrets on S3+KMS and use IAM Role on the machine to retrieve them. Or use Hashicorp Vault.

riquito 9y ago

I moved away from dockerfiles to Packer to build images for multiple environments. Even if you don't need that, ansible from dockerfile is a good choice to avoid writing hard to manage/reuse scripts in yml