3 ms·
> Running a job against a single host will finish in 3 minutes... running that exact same job against thousands will take well over an hour and max out your mac
by movedx 11y ago
> Running a job against a single host will finish in 3 minutes... running that exact same job against thousands will take well over an hour and max out your machine.
Have you used the variety of options to throttle this down? Running against 3,000 hosts at once is, in my opinion, absolutely crazy. And almost an contradiction to:
> Slicing up our inventory into chunks and running them on different servers sucks big time and is pretty hacky. How do I combine the results? ...
How do YOU combine them all? All results from all 3,000 hosts.? And it's not "hacky" at all, it's what you should be doing as a systems engineer/architect in the first place. No one network should be 3,000 hosts in size in a flat network structure; that should be partitioned up into logical arrangements for easier management (which you currently don't have, as per your own points), not to mention security.
As for combining the results. Have you considered writing a simple module for Ansible which sits locally and it called after every task is complete? It's easy to work with and you can push the results into ElasticSearch.
Ansible isn't a magic bullet, it's a tool on which to build, so build on it :-)