3 ms·

I considered doing it the way you described, but then you need to deploy software on every single one of your machines and make sure it's running, that it's not

I considered doing it the way you described, but then you need to deploy software on every single one of your machines and make sure it's running, that it's not accidentally using up 99% of your CPU (I've had bad experiences with the monitoring agents for Splunk and Netdata misbehaving and slowing down the machines and causing problems), etc. Whereas with the "pull" approach I used in my tool, you don't need to deploy ANY software to the machines you are monitoring-- you just connect with SSH and grab the files you need and do all the work on your control node.

xorcist 3y ago
One way or the other, your hosts are running your application, and you are already deploying software on every single host. But I hear you with some of the agents. That's why I mentioned syslog. It's already there, it's supported by most logging packages, and it's dead simple. No additional software required. All text. What it doesn't do is structured logging, but analyzing on the log host is often enough. Agents aren't all that bad however, and you're likely already running some agent like icinga or zabbix for regular monitoring.
hjvalente 3y ago
having a Netdata agent taking your machine's CPU to 99% shouldn't happen, not sure when was the list time you tried it but a lot of recent improvements have been done on the Netdata Agent also, with Netdata you can achieve the same architecture design using a Netdata Parent that could be your "control node" and to where you stream the metrics of the nodes you want to keep running with as less load as possible - you can even offload the health engine and the machine learning take a look at https://learn.netdata.cloud/docs/streaming/ https://learn.netdata.cloud/docs/streaming/ and https://learn.netdata.cloud/docs/configuring/how-to-optimize-the-netdata-agent-s-performance https://learn.netdata.cloud/docs/configuring/how-to-optimize...
- eigenvalue 3y ago
  I hear you, but once something like that burns me, I am very loathe to risk it again if the cost/benefit ratio seems unfavorable to me. While it's nice to have the pretty dashboards for Netdata, it's not worth even a small risk of it breaking or degrading my systems.

xorcist 3y ago

One way or the other, your hosts are running your application, and you are already deploying software on every single host. But I hear you with some of the agents. That's why I mentioned syslog. It's already there, it's supported by most logging packages, and it's dead simple. No additional software required. All text. What it doesn't do is structured logging, but analyzing on the log host is often enough. Agents aren't all that bad however, and you're likely already running some agent like icinga or zabbix for regular monitoring.

hjvalente 3y ago

having a Netdata agent taking your machine's CPU to 99% shouldn't happen, not sure when was the list time you tried it but a lot of recent improvements have been done on the Netdata Agent also, with Netdata you can achieve the same architecture design using a Netdata Parent that could be your "control node" and to where you stream the metrics of the nodes you want to keep running with as less load as possible - you can even offload the health engine and the machine learning take a look at https://learn.netdata.cloud/docs/streaming/ https://learn.netdata.cloud/docs/streaming/ and https://learn.netdata.cloud/docs/configuring/how-to-optimize-the-netdata-agent-s-performance https://learn.netdata.cloud/docs/configuring/how-to-optimize...

eigenvalue 3y ago
I hear you, but once something like that burns me, I am very loathe to risk it again if the cost/benefit ratio seems unfavorable to me. While it's nice to have the pretty dashboards for Netdata, it's not worth even a small risk of it breaking or degrading my systems.