Ansible seems to be better at declaring a particular configuration at build time, which should be repeatable and tested, and then perhaps running limited update

Ansible seems to be better at declaring a particular configuration at build time, which should be repeatable and tested, and then perhaps running limited updates in a more controlled fashion later. The destination hosts should be disposable and recreated for any significant changes. If you write playbooks to change existing state frequently you have to be very careful to write in tasks that check the previous state. If you can version the roles and playbooks carefully then you can have some idea of the systems' state. For simple idempotency, like changing an app's config template and restarting a daemon, the modules have you covered, but for more complex dependencies between services it can get hairy. Some unpredictable things can happen. Example story: systemd-tmpfiles on CentOS 7 has a poorly documented behavior of skipping files in /tmp or /var/tmp that are both root-owned and read-only. It's not something in the options or config, just a default behavior (that was removed in later versions). We had a playbook that would clone a git repo into /var/tmp and then copy the required files into their appropriate place. The git module worked fine to update the repo each run, until several weeks passed and systemd-tmpfiles blew away most, but not all, of the repo files. Some files with just read permission remained. The module could handle either all the repo files existing or having the repo disappear just fine, but not a partially deleted repo. It would error out on that task. So now there's a new dependency on a changed systemd-tmpfiles config to skip that directory, and the partially deleted mess must be deleted for a run to work on existing hosts with the old config. There's lots of stuff like that that can accrue.