6 ms·

We use Ansible to orchestrate our infrastructure and application deployment. Only in one particular stack are we using ansible how it is "supposed" to be used (

We use Ansible to orchestrate our infrastructure and application deployment. Only in one particular stack are we using ansible how it is "supposed" to be used (SSH:ing into a machine and updating/installing Prometheus). The infrastructure is provisioned using 90% CloudFormation and the other 10% is Ansible commands for when CF is not enough or when AWS CLI commands allows for better flexibility (i.e. generating SSH keys for EC2 machines and storing the private key in a central store). We make heavy use of the `cloudformation` Ansible-module.

mcintyre1994 7y ago
I'm looking to do something similar - spin up EC2 instances, deploy a script to them, run that script with various parameters, have it all defined in some template somewhere in our repo. Did you look at EC2 UserData at all when you decided to use Ansible? I asked about this use case on the AWS Slack and someone pointed me at it. I haven't had a look yet but if you've used it as well as Ansible then I'd be curious to know how they compare/how you ended up with Ansible + CF instead.
- GuyPostington 7y ago
  I used Ansible in pull mode (https://docs.ansible.com/ansible/latest/cli/ansible-pull.html https://docs.ansible.com/ansible/latest/cli/ansible-pull.htm...) in my UserData for several years. I've since left that company, but the implementation still powers their infrastructure. I had Ansible emit slack notifications when it started and completed. If the initial deploy ever failed, the node would eventually fail a healthcheck and that server would be destroyed and throw a different slack notification via an autoscaling group lifecycle monitor.
- jturpin 7y ago
  We use the userdata to call ansible-pull so that we can just terminate and re-create instances when we want to re-provision them. We use Terraform for the actual orchestration of the cloud environment.
  - ghthor 7y ago
    That's the way to handle it. Use terraform to describe the shape and where the holes are, and use something like ansible to configure/provision the resource after it comes up and fits into the hole.
gregmac 7y ago
Are you using Ansible to orchestrate the setup of a particular EC2 instance, or to actually provision AWS resources, or both? If you're doing AWS resources, where is ansible running from? I have a few deployments using CloudFormation, and in those use CloudFormation::Init [1] to actually setup the EC2 instances (deploy packages, apps, configure local firewall, etc). This kind of grew over time and now involves some (mildly) complex bash and/or powershell scripts which do things that could probably be done with Ansible, but when I started it wasn't obvious how complex that part would eventually get. The split of Ansible vs CloudFormation responsibility also wasn't clear to me, and I favored the simplicity of a single tool, so even though I considered using Ansible, I ended up sticking to CloudFormation::Init. In retrospect, CloudFormation::Init is pretty much a stand-alone tool, and the only connection it has to AWS and AWS CloudFormation is really just that it reads the metadata from the EC2 API to get its actual configuration (which is just a JSON or YAML blob). [1] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-init.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...
- jturpin 7y ago
  We just use ansible for the provisioning of EC2 instances, and Terraform for the actual orchestration of the resources. Userdata is used to call ansible-pull. Theoretically we can use Ansible for everything, but Terraform's planning makes the runs a lot safer, and in general it just seems a little nicer for orchestration.
- thecopy 7y ago
  No, we are running Ansible Playbooks on the localhost. We store configurations in group_vars and have differnet environments vars which can override the global defaults ones (e.g. instnaces types) (when running the playbook one needs to provide the env, e.g. `-e env=dev`), we then give these varialbes to the CF stacks as parameters. We also use Ansible Vault to store sensitive information in Git, and push these to e.g. AWS SSM Parameter Store as encrypted SecureStrings to provide the software with secrets (we provide the apps with the name of the SSM parameter key) EDIT: To clarify: the ansible-commands runs locally on your machine, or locally on the CI/CD agent - not on any remote machine/EC2 instance.

mcintyre1994 7y ago

I'm looking to do something similar - spin up EC2 instances, deploy a script to them, run that script with various parameters, have it all defined in some template somewhere in our repo. Did you look at EC2 UserData at all when you decided to use Ansible? I asked about this use case on the AWS Slack and someone pointed me at it. I haven't had a look yet but if you've used it as well as Ansible then I'd be curious to know how they compare/how you ended up with Ansible + CF instead.

GuyPostington 7y ago
I used Ansible in pull mode (https://docs.ansible.com/ansible/latest/cli/ansible-pull.html https://docs.ansible.com/ansible/latest/cli/ansible-pull.htm...) in my UserData for several years. I've since left that company, but the implementation still powers their infrastructure. I had Ansible emit slack notifications when it started and completed. If the initial deploy ever failed, the node would eventually fail a healthcheck and that server would be destroyed and throw a different slack notification via an autoscaling group lifecycle monitor.
jturpin 7y ago
We use the userdata to call ansible-pull so that we can just terminate and re-create instances when we want to re-provision them. We use Terraform for the actual orchestration of the cloud environment.
- ghthor 7y ago
  That's the way to handle it. Use terraform to describe the shape and where the holes are, and use something like ansible to configure/provision the resource after it comes up and fits into the hole.

gregmac 7y ago

Are you using Ansible to orchestrate the setup of a particular EC2 instance, or to actually provision AWS resources, or both? If you're doing AWS resources, where is ansible running from? I have a few deployments using CloudFormation, and in those use CloudFormation::Init [1] to actually setup the EC2 instances (deploy packages, apps, configure local firewall, etc). This kind of grew over time and now involves some (mildly) complex bash and/or powershell scripts which do things that could probably be done with Ansible, but when I started it wasn't obvious how complex that part would eventually get. The split of Ansible vs CloudFormation responsibility also wasn't clear to me, and I favored the simplicity of a single tool, so even though I considered using Ansible, I ended up sticking to CloudFormation::Init. In retrospect, CloudFormation::Init is pretty much a stand-alone tool, and the only connection it has to AWS and AWS CloudFormation is really just that it reads the metadata from the EC2 API to get its actual configuration (which is just a JSON or YAML blob). [1] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-init.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

jturpin 7y ago
We just use ansible for the provisioning of EC2 instances, and Terraform for the actual orchestration of the resources. Userdata is used to call ansible-pull. Theoretically we can use Ansible for everything, but Terraform's planning makes the runs a lot safer, and in general it just seems a little nicer for orchestration.
thecopy 7y ago
No, we are running Ansible Playbooks on the localhost. We store configurations in group_vars and have differnet environments vars which can override the global defaults ones (e.g. instnaces types) (when running the playbook one needs to provide the env, e.g. `-e env=dev`), we then give these varialbes to the CF stacks as parameters. We also use Ansible Vault to store sensitive information in Git, and push these to e.g. AWS SSM Parameter Store as encrypted SecureStrings to provide the software with secrets (we provide the apps with the name of the SSM parameter key) EDIT: To clarify: the ansible-commands runs locally on your machine, or locally on the CI/CD agent - not on any remote machine/EC2 instance.