3 ms·
I'd still argue that it's easier to scale Pulling than it is a distributed push. It's kind of why prometheus went that route in the first place. Back in the da
by llama052 3y ago
I'd still argue that it's easier to scale Pulling than it is a distributed push. It's kind of why prometheus went that route in the first place.
Back in the days of puppet or nagios, which would take requests and not pull. It was very common to hear about them cascading and causing huge denial of service issues and even causing massive outages because of it. For the simple fact that it's way harder to control thousands of servers sending out data on a timeframe versus a set of infrastructure designed to query them.
If I recall correctly facebook in the early days had a full on datacenter meltdown due to their puppet cluster pushing a bad release causing every host to check in, they were offline for a full day I think, they couldn't update the thousands of hosts because things were so saturated.
However in the case of polling, you'd dictate that from the monitoring servers themselves, you can control and dictate that without causing sprawls of outages and calls from everything.
Pull model can obviously scale(a lot of the shortcomings are now addressed in Thanos/Cortex): https://promcon.io/2016-berlin/talks/scaling-to-a-million-machines-with-prometheus/ https://promcon.io/2016-berlin/talks/scaling-to-a-million-ma...
- KaiserPro 3y ago> puppet cluster pushing a bad release causing every host to check in, It was probably chef, but yeah I totally can see that happening. in terms of scaling, nowadays everything either shards or can sit behind a load balancer, so partitioning is much more simple nowadays. for network layout though, having hosts that can get access to a large number of machines is something I really don't like. Traditional monitoring, where you have an agent running as root and can execute a bunch of functions is also a massive security risk, and has largely moved to other forms of monitoring.