5 ms·

Maybe a stupid redundant slightly offtopic question, but I'll ask anyways: What is the best way to couple Ansible with some git repository to automatically app

by corty 5y ago

Maybe a stupid redundant slightly offtopic question, but I'll ask anyways: What is the best way to couple Ansible with some git repository to automatically apply changes while still being able to trace and debug?

edaemon 5y ago
This definitely isn't the best way because it's far from painless, but we use GitHub Actions to run some Ansible playbooks. When there's a problem that needs debugging we add a tmate[0] session with mxschmitt's action[1] so we can step through the playbook, etc. [0]: https://tmate.io/ https://tmate.io/ [1]: https://github.com/mxschmitt/action-tmate https://github.com/mxschmitt/action-tmate
- Datagenerator 5y ago
  Or Jenkins pipelined steps which might include setting up the pipenv to include the setup of Ansible in your git as requirement file. So it starts with Groovy and switches to Ansible etc
WFHRenaissance 5y ago
Build/Buy a real CI/CD system for this TBH. Once you hit a certain level of complexity, this sort of deployment strategy will mostly just amplify pain.
xorcist 5y ago
Tower? Open sourced by Red Hat as AWX, it is a central Ansible hub for exactly that purpose.
- Spivak 5y ago
  Tower/AWX are bad products and they should feel bad. It’s actually easier to run Ansible via any CI system than it is to use Tower. I get what the devs are trying to do but Tower wants to own so much of the process that it’s basically not Ansible anymore. You pretty much have to write your code specifically for Tower (which means a total rewrite in mosts shops because who starts with Tower?)
  - xorcist 5y ago
    That type of criticism is a bit broad. While it has all the hallmarks of a commercial product trying to fit every type of customer, on the other hand the whole point is having a centralized architecture for Ansible. This is something desirable as soon as you outgrow using it for ad-hoc jobs, and there is a payoff compared to just scripting in bash or dockerfiles. Of course the code has to be adapter for it, but it's not anything like a complete rewrite.
ratorx 5y ago
Ansible doesn’t have the right model for painless auto application, because it’s only idempotent in some cases and not others (e.g. removing config). It’s a good tool to set up a server from scratch to a desired state, but not to go between two versions of desired state. So a git-based deployment workflow is doomed for failure as soon as you start actually using the history (or removing config).
- notabee 5y ago
  We've been using the git module to pull down the desired version of the repo and then using synchronize to copy it over. That could be used to overwrite whatever was previously being copied from the git repo. So, can you explain where that doesn't work for a git based deployment? It's a pattern we use a lot. (A slight tangent though, systemd-tmpfiles can ruin your day if it partially deletes the repo, because that makes the git module crap out.)
  - ratorx 5y ago
    If I understood your use case correctly, I think you’re deploying an application with it? I meant more about server configuration things with Ansible configuration which is version controlled and auto-applied on push. In that case, say you are using Ansible to set up a daemon A on a server. Now you want to remove the setup for daemon A and move to daemon B. Just changing the configuration is not enough (since nothing stops daemon A and cleans up it’s configuration). And after you’re in the new state (let’s say daemon B was just an experiment), if you decide to revert the commit that added daemon B and push, nothing will stop daemon B.
    - notabee 5y ago
      I'm still not seeing the difficulty with doing that. Basically any daemons that you're setting up using distro packages or systemd service units (which, even though I'm not strictly in love with systemd, is what you should do for any service daemons you set up), it's just a matter of telling the service/systemd module and probably the copy or template module that you used to created the config files that they should be "state: disabled" or "state: absent" respectively. It'll disable and remove config of whatever service you've set up. What I will also do some times is create an array variable of services to enable and services to disable, and all that's required to switch is moving a daemon's name from one list to another. So, unless you're doing something outside of the modules like copying a bunch of stuff with the shell module and launching these putative daemons without some supervisor like systemd, it's pretty trivial to tell Ansible to reverse whatever it did. And if you are just launching processes using a bash script or something, why? It can get more complicated if it's e.g. a dev box and people are using ssh to go in and tinker directly, but generally Ansible will still just remove/create whatever needs to be there if it needs to be changed, or do nothing. Depending on how robust you want to make Ansible against people doing unexpected things, you can use trap doors like "meta: end_host" to bail out on errors if people are causing problems with manual edits, but that's better solved as a people/process problem than trying to make Ansible handle all possible unexpected situations. In the end it's also usually just easier to use Ansible to bake a new VM or container configuration and deploy that instead of mutating an existing one over and over again.
nijave 5y ago
For small deploys you could set a cron that does `git pull && ansible-playbook`. You can setup your cron to email failures. This can be setup with the ansible playbook and bootstrapped by running over ssh instead of local. Cron can email failures For more than a few servers, other answers are probably better
- mdaniel 5y ago
  > set a cron that does `git pull && ansible-playbook` AKA ansible-pull, which can greatly simplify the auth story since the machine only needs outbound access and auth to the repo (if required): https://docs.ansible.com/ansible/2.10/cli/ansible-pull.html https://docs.ansible.com/ansible/2.10/cli/ansible-pull.html
  - nijave 5y ago
    Didn't realize that was built in to Ansible. Depending on how ephemeral your hosts are, you could scale this to quite a bit of hosts and use host-reported health checks to determine if things are working correctly e.g. HTTP POST a "this went ok" to some web server or metrics/monitoring service

by corty 5y ago

edaemon 5y ago

This definitely isn't the best way because it's far from painless, but we use GitHub Actions to run some Ansible playbooks. When there's a problem that needs debugging we add a tmate[0] session with mxschmitt's action[1] so we can step through the playbook, etc. [0]: https://tmate.io/ https://tmate.io/ [1]: https://github.com/mxschmitt/action-tmate https://github.com/mxschmitt/action-tmate

Datagenerator 5y ago
Or Jenkins pipelined steps which might include setting up the pipenv to include the setup of Ansible in your git as requirement file. So it starts with Groovy and switches to Ansible etc

WFHRenaissance 5y ago

Build/Buy a real CI/CD system for this TBH. Once you hit a certain level of complexity, this sort of deployment strategy will mostly just amplify pain.

xorcist 5y ago

Tower? Open sourced by Red Hat as AWX, it is a central Ansible hub for exactly that purpose.

Spivak 5y ago
Tower/AWX are bad products and they should feel bad. It’s actually easier to run Ansible via any CI system than it is to use Tower. I get what the devs are trying to do but Tower wants to own so much of the process that it’s basically not Ansible anymore. You pretty much have to write your code specifically for Tower (which means a total rewrite in mosts shops because who starts with Tower?)
- xorcist 5y ago
  That type of criticism is a bit broad. While it has all the hallmarks of a commercial product trying to fit every type of customer, on the other hand the whole point is having a centralized architecture for Ansible. This is something desirable as soon as you outgrow using it for ad-hoc jobs, and there is a payoff compared to just scripting in bash or dockerfiles. Of course the code has to be adapter for it, but it's not anything like a complete rewrite.

ratorx 5y ago

Ansible doesn’t have the right model for painless auto application, because it’s only idempotent in some cases and not others (e.g. removing config). It’s a good tool to set up a server from scratch to a desired state, but not to go between two versions of desired state. So a git-based deployment workflow is doomed for failure as soon as you start actually using the history (or removing config).

notabee 5y ago
We've been using the git module to pull down the desired version of the repo and then using synchronize to copy it over. That could be used to overwrite whatever was previously being copied from the git repo. So, can you explain where that doesn't work for a git based deployment? It's a pattern we use a lot. (A slight tangent though, systemd-tmpfiles can ruin your day if it partially deletes the repo, because that makes the git module crap out.)
- ratorx 5y ago
  If I understood your use case correctly, I think you’re deploying an application with it? I meant more about server configuration things with Ansible configuration which is version controlled and auto-applied on push. In that case, say you are using Ansible to set up a daemon A on a server. Now you want to remove the setup for daemon A and move to daemon B. Just changing the configuration is not enough (since nothing stops daemon A and cleans up it’s configuration). And after you’re in the new state (let’s say daemon B was just an experiment), if you decide to revert the commit that added daemon B and push, nothing will stop daemon B.
  - notabee 5y ago
    I'm still not seeing the difficulty with doing that. Basically any daemons that you're setting up using distro packages or systemd service units (which, even though I'm not strictly in love with systemd, is what you should do for any service daemons you set up), it's just a matter of telling the service/systemd module and probably the copy or template module that you used to created the config files that they should be "state: disabled" or "state: absent" respectively. It'll disable and remove config of whatever service you've set up. What I will also do some times is create an array variable of services to enable and services to disable, and all that's required to switch is moving a daemon's name from one list to another. So, unless you're doing something outside of the modules like copying a bunch of stuff with the shell module and launching these putative daemons without some supervisor like systemd, it's pretty trivial to tell Ansible to reverse whatever it did. And if you are just launching processes using a bash script or something, why? It can get more complicated if it's e.g. a dev box and people are using ssh to go in and tinker directly, but generally Ansible will still just remove/create whatever needs to be there if it needs to be changed, or do nothing. Depending on how robust you want to make Ansible against people doing unexpected things, you can use trap doors like "meta: end_host" to bail out on errors if people are causing problems with manual edits, but that's better solved as a people/process problem than trying to make Ansible handle all possible unexpected situations. In the end it's also usually just easier to use Ansible to bake a new VM or container configuration and deploy that instead of mutating an existing one over and over again.

nijave 5y ago

For small deploys you could set a cron that does `git pull && ansible-playbook`. You can setup your cron to email failures. This can be setup with the ansible playbook and bootstrapped by running over ssh instead of local. Cron can email failures For more than a few servers, other answers are probably better

mdaniel 5y ago
> set a cron that does `git pull && ansible-playbook` AKA ansible-pull, which can greatly simplify the auth story since the machine only needs outbound access and auth to the repo (if required): https://docs.ansible.com/ansible/2.10/cli/ansible-pull.html https://docs.ansible.com/ansible/2.10/cli/ansible-pull.html
- nijave 5y ago
  Didn't realize that was built in to Ansible. Depending on how ephemeral your hosts are, you could scale this to quite a bit of hosts and use host-reported health checks to determine if things are working correctly e.g. HTTP POST a "this went ok" to some web server or metrics/monitoring service