12 ms·

Netdata: Open-source real-time monitoring platform

256 points by dvfjsdhgfv 6y ago

Havoc 6y ago
It’s very neat for individual servers Doesn’t work well for monitoring multiple servers though from what I can tell.
- dddw 6y ago
  Indeed, very handy tool. I once used it to discover that a new deployment generated spike CPU load. Reason was a badly implemented javascript doing a call on the db when hovering over a product (preview stock). Fun to see the actual correlation in a GUI in realtime.
  - odyslam 6y ago
    Hey, That sounds super interesting. Would you be interested to share the use-case in our forums? https://community.netdata.cloud https://community.netdata.cloud Ping me (@odyslam in the forum) if you need any help. We are always looking for awesome stories like that!
    - dddw 6y ago
      sure I'll be in between jobs soon, so I got some time to write something
- szszrk 6y ago
  That is true. It has some features to allow to quickly jump between many machines, but indeed out of the box it is not for many servers at the same time. BUT it can be configured to push data to Prometheus or similar (it's called a "backend") and some other integrations like notifications can be done. Super neat project, very easy to set up. I highly recommend it to anyone who does performance troubleshooting. Netdata put into a standard Linux system will detect a lot of different things, like firewalls, containers, a lot of software like databases, queuing systems, mail systems and provide additional data every second. Works flawlessly on Proxmox clusters!
- PanosJee 6y ago
  Have you given a try to netdata.cloud? It's still early but solves this problem without any effort
- _joel 6y ago
  There's a netdata prometheus exporter, but it overlaps a lot with node-exporter. If you're already runnning netdata however then it could be a good choice
  - lima 6y ago
    node_exporter is a lot more robust. Had both running for a while and netdata would get stuck when there's I/O trouble, while node_exporter was carefully built not to do any I/O and kept working just fine.
    - _joel 6y ago
      Yes, I'd stick to node exporer where possible.
- underhood 6y ago
  Did you try Netdata Cloud?
- Abishek_Muthian 6y ago
  Neat project but I'm also confused with 'distributed'; It sounds like designed for monitoring multiple systems in a single dashbaord OOB with 'zero-config', But on further digging it seems like the distributed monitoring works 'only' with their cloud service[1]. Further clarity on this would be appreciated. [1] https://github.com/netdata/netdata/blob/master/docs/quickstart/infrastructure.md https://github.com/netdata/netdata/blob/master/docs/quicksta...
  - ohthehugemanate 6y ago
    It's OOTB configured to use their free cloud service, but with 2 lines of config you can run your own central collection point instead. That's what I do for my home install. BUT the UI for this is just a dropdown for each of your monitored servers. I've found I actually want to export data to a more robust system so I can view patterns across machines, too.
    - Abishek_Muthian 6y ago
      Thanks, So monitoring multiple machines is possible in a central console although not in a single dashboard. Hopefully it will be available soon and the project seems useful as it is as I like the idea of getting the system information of all the Servers, SBCs in my network.
      - cakrit 6y ago
        This is what we do with Netdata Cloud. We want to keep the FOSS agent as a powerful single node monitoring tool and use the cloud for free infrastructure monitoring. You can see ktsaou's comment above on how we intend to monetize.
- ktsaou 6y ago
  Hi. I am the founder of Netdata. We complement the Netdata agent with Netdata.Cloud, a free-forever SaaS offering that maintains all the principles of the Netdata agent, while providing infrastructure level monitoring and several additional convenience features. In Netdata.Cloud, infrastructure is organized in war-rooms. On each war-room you will find the "Overview" page, that provides a fully automated dashboard, very similar to the one provided by the agent, in which every chart presented aggregates data from all servers in the war-room! Magic! Zero configuration! Fully automated! Keep in mind that Netdata.Cloud is a thin convenience layer on top of the Netdata agent. We don't aggregate your data. Your data stay inside your servers. We only collect and store a few metadata (how many netdata agents you have, which metrics they collect, what alarms have been configured, when they triggered - but not the actual metric and log data of your systems). Try it! You will be surprised!
  - Havoc 6y ago
    >I am the founder of Netdata. Awesome! I see the free tier is indeed looking generous. Just hooked up a node and looks good - I like the calculate correlations on alerts thing in particular. >Keep in mind that Netdata.Cloud is a thin convenience layer on top of the Netdata agent. I see. Didn't know/understand that. On the claim node page - could you perhaps add the kickstart bash script code too? I find myself needing them one after the other yet they're on different pages
    - andrewm4894 6y ago
      Good to hear metrics correlation might be useful to you, just as background you can get more info here: https://www.netdata.cloud/blog/netdata-cloud-metric-correlations/ https://www.netdata.cloud/blog/netdata-cloud-metric-correlat... At the moment it's based on a short window of data so the focus is more for short term changes around an area of interest you have already found. Longer term it would be cool to be able to use an anomaly score on the metrics themselves (or if a lot of alarms happen to be going off) to automatically find such regions for you so its more like surfacing insights to you as opposed to you having to already know a window if time you are interested in.
  - 6y ago
dvfjsdhgfv 6y ago
The only gripe I have with it is the approach to security, i.e. the lack of user accounts (even one). So you have to either block the stats by IP (who is doing it these days?) or use other workarounds like proxying by Nginx etc.
- smarx007 6y ago
  I have it listen on a loopback interface and do SSH port forwarding when I want to look at the stats. Nginx proxying with basic auth is a perfectly reasonable approach and not a workaround in my humble opinion. I would trust these two approaches more than an unknown mechanism in Netdata.
- odyslam 6y ago
  Using Netdata Cloud is a great way not to spend any time with that and access the Agent's dashboard through Netdata Cloud. We use WSS and MQTT, so it's super secure and lightweight. The data are streamed from the Agent directly to your browser via the cloud. Relevant docs: https://learn.netdata.cloud/docs/configure/secure-nodes#disable-the-local-dashboard https://learn.netdata.cloud/docs/configure/secure-nodes#disa...
  - sammy2244 6y ago
    So the only convenient way to have security is to use the cloud version? Got it.
    - dvfjsdhgfv 6y ago
      > So the only convenient way to have security is to use the cloud version? Got it. I wouldn't formulate it that way, it's just a bit annoying for me to see this trend of not having even tiny bit of security built in and having to do extra work just to protect the dashboard. Just one admin account and a random generated password would be fine.
    - distantsounds 6y ago
      yes, because a 10 line nginx config with basic http auth is too difficult for a sysadmin to set up in conjunction with his systems monitoring tool stop being obtuse
mekster 6y ago
All these graphs are never really actionable and are only of interest for a short period of time and you won't be looking at it after a while because they don't mean anything unless you know where and when the problem is. A sever admin wants "Incident" panel that only shows anomaly components at the top coupled with adjustable alerting mechanism and not just a dump of all the data there is blindly. There are so many tools that does this and pretend it's impressive including ELK but whether it's Grafana or Kibana, you need a lot of manual tweaking to make the dashboards actually useful.
- xPaw 6y ago
  Netdata does have alarms/alerts, and comes with default ones. https://learn.netdata.cloud/guides/step-by-step/step-05 https://learn.netdata.cloud/guides/step-by-step/step-05
  - odyslam 6y ago
    Actually, we give a lot of thought in defining sane default alarms for most of the data sources that we have. We want our users to get 80% of the value with 20% of the effort. It's an opinionated approach that liberates a lot of users from having to setup and maintain everything.
    - mdip 6y ago
      I think you took a lot of flack in the original comment[0] The alarms in netdata resolved a long-standing network issue on one of my boxes, and have variously alerted me to problems I could resolve with storage which greatly improved performance on my largest volume. On my other box, one look at the graphs alerted me to the fact that the entire SSD for my bcache volume was going unused[1]. I then used them while altering configuration and working with the drive to ensure the cache was being filled in a manner consistent with what the volume stored/how it was used. The more I think about it, I might not have been as enthusiastic in my original comment as I should have been. It's been very helpful to me. I don't usually keep things like this running for very long (it wastes cycles on aging hardware...that isn't heavily used, but hey, it's the principal!) but I've kept this around because every time I've thought about removing it, I've visited the dashboard one last time and found something there that made me keep it. [0] Though, as I mentioned, I'm not a sysadmin; I have a lab that might indicate otherwise, but I don't get paid for it. [1] I had reloaded the machine/redone a previous configuration that included bcache and it screamed; I knew my new setup was much slower but I had forgotten about it until netdata made it obvious, again. I can't remember what I had to do to fix it, but it had something to do with the policy used to determine if a file should be put into the cache, and I think it was related to the fact that the cache was added to a volume with data present that rarely changed.
enz 6y ago
Is it any good? ;)
- odyslam 6y ago
  yes :)
- unixhero 6y ago
  Very good.
- papazach 6y ago
  It is great, I have claimed my VMs running Netdata to the Netdata Cloud and I am very happy with it! Took me only a few minutes to claim them all (11 VMs) and boom the dashboards were ready out of the box.
mprovost 6y ago
I write and maintain an open source monitoring tool and I looked into adding a mode to output metrics in Netdata format and ran away screaming. It's just an unstructured text format where you output commands to stdout, one per line. Each command consists of whitespace-separated fields. Which field is the units? Oh, the 4th. And some fields are optional, I'm not even sure how that works but I think you can't skip an optional field if you then want to use any field after that. It's like structured data formats like JSON or god forbid XML never happened.
- cakrit 6y ago
  Netdata can ingest prometheus metrics as well, so you can just use that format. Eventually everything will become Openmetrics/Opentelemetry
abbaselmas 6y ago
Why netdata is popular on HN now? There must be some big news or something..
- andrewm4894 6y ago
  i think it was just a random share by someone.
tifadg1 6y ago
Could someone enlighten me on the internals, how is netdata able to get realtime granularity, whereas prometheus defaults to 15s?
- distantsounds 6y ago
  it polls every second, to get metrics every second. genius, i know?
- wongarsu 6y ago
  I doesn't store much history from what I can tell. If you don't have years worth of data points then having 15 times as many isn't a big deal.
  - mekster 6y ago
    And people are somehow meant to only monitor a single server? There's a reason timeseries databases are trying to get downsampling right.
    - manigandham 6y ago
      It's a locally installed agent that monitors and serves metrics on the same host. If you want to monitor multiple hosts then you can either visit the dashboards individually, or scrape the APIs and put the metrics on a combined dashboard - which is what Netdata Cloud is.
- cakrit 6y ago
  It's because it was built with high granularity and unlimited metrics as a key differentiator from the beginning. The core is written in pure C, optimized to death. Even long-term retention was initially sacrificed, in order to be able achieve that high performance, with minimal resource needs. Long term retention is now possible, but with relatively high memory requirements, depending on how many metrics are collected. Again, it was a decision to never give up realtime granularity and speed, even at the cost of writing our own timeseries db in C and utilizing more memory.
- 6y ago
gregwebs 6y ago
Netdata is a great building block in a monitoring system. It now does a lot of monitoring via eBPF, connects to Prometheus, and integrates with k8s.
- odyslam 6y ago
  We do love ebpf. Guilty as charged ¯\_(ツ)_/¯ We have a whole bunch of metrics that we keep track and we are currently implementing a load more. Soonish, we will greatly increase the number of metrics that we gather with ebpf. That coupled with our per-second granularity, should give you a very detailed view of the system. Docs: https://learn.netdata.cloud/docs/agent/collectors/ebpf.plugin https://learn.netdata.cloud/docs/agent/collectors/ebpf.plugi... Community Forums discussion: https://community.netdata.cloud/t/linux-kernel-insights-with-ebpf/992 https://community.netdata.cloud/t/linux-kernel-insights-with...
petecooper 6y ago
Previous discussions: https://news.ycombinator.com/item?id=11388196 https://news.ycombinator.com/item?id=11388196 https://news.ycombinator.com/item?id=17773874 https://news.ycombinator.com/item?id=17773874 https://news.ycombinator.com/item?id=26886792 https://news.ycombinator.com/item?id=26886792 (For commentary, I'm not being snarky.)
saberience 6y ago
Why would I use it over DataDog?
- nine_k 6y ago
  Different scale? Different price point?
- PanosJee 6y ago
  More metrics, higher resolution, pre-configured alert. It cannot displace it as is but it can work in tandem very nicely.
crazypython 6y ago
Haven't been able to use its graphical interface to view historical data. At least it uses fewer resources than Grafana.
- distantsounds 6y ago
  netdata doesn't store metrics historically but you can funnel whatever ones you want out and ship them off to a log store like graphite or opentsdb.
  - PanosJee 6y ago
    It has the option now to set retention for up to a year.
    - distantsounds 6y ago
      ooh, even better!
sneak 6y ago
Note that netdata phones home without consent in the default configuration. For many, the whole point of doing system-administration is selfhosting and autonomy, and privacy is frequently a big component of that. Netdata blows a big hole in that by transmitting your usage information off of your box without getting permission.
- odyslam 6y ago
  That's a bit unfair. In the docs we are being very upfront that you can opt-out of anonymous telemetry: https://learn.netdata.cloud/docs/get https://learn.netdata.cloud/docs/get we use the data we gather in order to make smarter product decisions. We want to invest resources where it matters, so we need to know how our users use the product. We are also very detailed on what we gather: https://learn.netdata.cloud/docs/agent/netdata-security https://learn.netdata.cloud/docs/agent/netdata-security Lastly, we just changed our analytics engine, from google-analytics to a self-hosted posthog, which is an open-source product analytics platform
  - sneak 6y ago
    The telemetry isn't anonymous: it includes the client IP; the method you use to transmit the data cannot work anonymously. Additionally, what's actually unfair is that you proceed with this spying without the consent of the user. Being upfront about it is not obtaining consent: it's just informing the user you're about to violate their (lack of) consent. You must obtain consent from the user first, before transmitting their information. Otherwise, your software is spyware. (Disclosing that you're going to spy on the user doesn't make you not-spyware.) > we use the data we gather in order to make smarter product decisions. Yes, you transmit the private data of the user for the express purpose of enriching yourself. Opt-out is unethical: you must obtain opt-in consent first. The data you are transmitting does not belong to you.
    - yunohn 6y ago
      Dude, seriously? You choose to willfully install Netdata. You have to read the docs where the opt-out telemetry is clearly explained, before you can self-host it too. If you care, you can disable it. I honestly don’t understand HN. Multiple commenters deriding a free open-source project for having basic telemetry to understand feature usage.
hivacruz 6y ago
How does it compare to New Relic who also happens to monitor, if enabled, containers and system things?
mtmsr 6y ago
I have played around with netdata just yesterday on my home server. Great tool, but the defaults are overkill for my needs. After spending an hour trying to simplify (=disable most of the "collectors") using the documentation, I finally gave up. Settled on neofetch [1] instead: pure bash, wrote my own custom inputs including color coding for incident reporting in less time than it took me to strip down netdata. Highly recommended if you want to spend your time on other things than (setting up) server monitoring. [1] https://github.com/dylanaraps/neofetch https://github.com/dylanaraps/neofetch
- gregwebs 6y ago
  Thanks for the link: neofetch seems a good tool when you just want to manually see what is going on. Netdata is also designed to alert, forward data to other locations, monitor at 1 second granularity, and to store historical data efficiently if you want to see what went on in the recent past.
jitl 6y ago
I run Netdata on my home server using the official docker image. I mostly use it to detect run-away containers, and monitor system temprature. For these use-cases, it works great, and I like that it's self contained; way less headache than stringing together Graphviz stuff, or setting up Nagios or Prometheus. Beware - by default it'll send home telemetry, and the web UI will try to "register" your instance with some kind of cloud. I find this super annoying, but it's possible to turn it off; just not well documented. There's also a lot of plugins that scrape many kinds of logs, look at process data, etc. Again, might be useful, but for a home user it's much better to turn it all off. My notes here have a write-up of how to run Netdata via Docker with example config files that disable the unwanted features here: https://jake.tl/notes/2019-11-09-cozy-server#netdata https://jake.tl/notes/2019-11-09-cozy-server#netdata
linsomniac 6y ago
We have a central influxdb with telegraf metrics among others, and some grafana graphs. I still install netdata on every machine though. Almost never use it, but there have been some times where it was useful to look at netdata. It's light weight enough that it hasn't been a problem.

256 points by dvfjsdhgfv 6y ago

Havoc 6y ago

It’s very neat for individual servers Doesn’t work well for monitoring multiple servers though from what I can tell.

dddw 6y ago
Indeed, very handy tool. I once used it to discover that a new deployment generated spike CPU load. Reason was a badly implemented javascript doing a call on the db when hovering over a product (preview stock). Fun to see the actual correlation in a GUI in realtime.
- odyslam 6y ago
  Hey, That sounds super interesting. Would you be interested to share the use-case in our forums? https://community.netdata.cloud https://community.netdata.cloud Ping me (@odyslam in the forum) if you need any help. We are always looking for awesome stories like that!
  - dddw 6y ago
    sure I'll be in between jobs soon, so I got some time to write something
szszrk 6y ago
That is true. It has some features to allow to quickly jump between many machines, but indeed out of the box it is not for many servers at the same time. BUT it can be configured to push data to Prometheus or similar (it's called a "backend") and some other integrations like notifications can be done. Super neat project, very easy to set up. I highly recommend it to anyone who does performance troubleshooting. Netdata put into a standard Linux system will detect a lot of different things, like firewalls, containers, a lot of software like databases, queuing systems, mail systems and provide additional data every second. Works flawlessly on Proxmox clusters!
PanosJee 6y ago
Have you given a try to netdata.cloud? It's still early but solves this problem without any effort
_joel 6y ago
There's a netdata prometheus exporter, but it overlaps a lot with node-exporter. If you're already runnning netdata however then it could be a good choice
- lima 6y ago
  node_exporter is a lot more robust. Had both running for a while and netdata would get stuck when there's I/O trouble, while node_exporter was carefully built not to do any I/O and kept working just fine.
  - _joel 6y ago
    Yes, I'd stick to node exporer where possible.
underhood 6y ago
Did you try Netdata Cloud?
Abishek_Muthian 6y ago
Neat project but I'm also confused with 'distributed'; It sounds like designed for monitoring multiple systems in a single dashbaord OOB with 'zero-config', But on further digging it seems like the distributed monitoring works 'only' with their cloud service[1]. Further clarity on this would be appreciated. [1] https://github.com/netdata/netdata/blob/master/docs/quickstart/infrastructure.md https://github.com/netdata/netdata/blob/master/docs/quicksta...
- ohthehugemanate 6y ago
  It's OOTB configured to use their free cloud service, but with 2 lines of config you can run your own central collection point instead. That's what I do for my home install. BUT the UI for this is just a dropdown for each of your monitored servers. I've found I actually want to export data to a more robust system so I can view patterns across machines, too.
  - Abishek_Muthian 6y ago
    Thanks, So monitoring multiple machines is possible in a central console although not in a single dashboard. Hopefully it will be available soon and the project seems useful as it is as I like the idea of getting the system information of all the Servers, SBCs in my network.
    - cakrit 6y ago
      This is what we do with Netdata Cloud. We want to keep the FOSS agent as a powerful single node monitoring tool and use the cloud for free infrastructure monitoring. You can see ktsaou's comment above on how we intend to monetize.
ktsaou 6y ago
Hi. I am the founder of Netdata. We complement the Netdata agent with Netdata.Cloud, a free-forever SaaS offering that maintains all the principles of the Netdata agent, while providing infrastructure level monitoring and several additional convenience features. In Netdata.Cloud, infrastructure is organized in war-rooms. On each war-room you will find the "Overview" page, that provides a fully automated dashboard, very similar to the one provided by the agent, in which every chart presented aggregates data from all servers in the war-room! Magic! Zero configuration! Fully automated! Keep in mind that Netdata.Cloud is a thin convenience layer on top of the Netdata agent. We don't aggregate your data. Your data stay inside your servers. We only collect and store a few metadata (how many netdata agents you have, which metrics they collect, what alarms have been configured, when they triggered - but not the actual metric and log data of your systems). Try it! You will be surprised!
- Havoc 6y ago
  >I am the founder of Netdata. Awesome! I see the free tier is indeed looking generous. Just hooked up a node and looks good - I like the calculate correlations on alerts thing in particular. >Keep in mind that Netdata.Cloud is a thin convenience layer on top of the Netdata agent. I see. Didn't know/understand that. On the claim node page - could you perhaps add the kickstart bash script code too? I find myself needing them one after the other yet they're on different pages
  - andrewm4894 6y ago
    Good to hear metrics correlation might be useful to you, just as background you can get more info here: https://www.netdata.cloud/blog/netdata-cloud-metric-correlations/ https://www.netdata.cloud/blog/netdata-cloud-metric-correlat... At the moment it's based on a short window of data so the focus is more for short term changes around an area of interest you have already found. Longer term it would be cool to be able to use an anomaly score on the metrics themselves (or if a lot of alarms happen to be going off) to automatically find such regions for you so its more like surfacing insights to you as opposed to you having to already know a window if time you are interested in.
- 6y ago

dvfjsdhgfv 6y ago

The only gripe I have with it is the approach to security, i.e. the lack of user accounts (even one). So you have to either block the stats by IP (who is doing it these days?) or use other workarounds like proxying by Nginx etc.

smarx007 6y ago
I have it listen on a loopback interface and do SSH port forwarding when I want to look at the stats. Nginx proxying with basic auth is a perfectly reasonable approach and not a workaround in my humble opinion. I would trust these two approaches more than an unknown mechanism in Netdata.
odyslam 6y ago
Using Netdata Cloud is a great way not to spend any time with that and access the Agent's dashboard through Netdata Cloud. We use WSS and MQTT, so it's super secure and lightweight. The data are streamed from the Agent directly to your browser via the cloud. Relevant docs: https://learn.netdata.cloud/docs/configure/secure-nodes#disable-the-local-dashboard https://learn.netdata.cloud/docs/configure/secure-nodes#disa...
- sammy2244 6y ago
  So the only convenient way to have security is to use the cloud version? Got it.
  - dvfjsdhgfv 6y ago
    > So the only convenient way to have security is to use the cloud version? Got it. I wouldn't formulate it that way, it's just a bit annoying for me to see this trend of not having even tiny bit of security built in and having to do extra work just to protect the dashboard. Just one admin account and a random generated password would be fine.
  - distantsounds 6y ago
    yes, because a 10 line nginx config with basic http auth is too difficult for a sysadmin to set up in conjunction with his systems monitoring tool stop being obtuse

mekster 6y ago

All these graphs are never really actionable and are only of interest for a short period of time and you won't be looking at it after a while because they don't mean anything unless you know where and when the problem is. A sever admin wants "Incident" panel that only shows anomaly components at the top coupled with adjustable alerting mechanism and not just a dump of all the data there is blindly. There are so many tools that does this and pretend it's impressive including ELK but whether it's Grafana or Kibana, you need a lot of manual tweaking to make the dashboards actually useful.

xPaw 6y ago
Netdata does have alarms/alerts, and comes with default ones. https://learn.netdata.cloud/guides/step-by-step/step-05 https://learn.netdata.cloud/guides/step-by-step/step-05
- odyslam 6y ago
  Actually, we give a lot of thought in defining sane default alarms for most of the data sources that we have. We want our users to get 80% of the value with 20% of the effort. It's an opinionated approach that liberates a lot of users from having to setup and maintain everything.
  - mdip 6y ago
    I think you took a lot of flack in the original comment[0] The alarms in netdata resolved a long-standing network issue on one of my boxes, and have variously alerted me to problems I could resolve with storage which greatly improved performance on my largest volume. On my other box, one look at the graphs alerted me to the fact that the entire SSD for my bcache volume was going unused[1]. I then used them while altering configuration and working with the drive to ensure the cache was being filled in a manner consistent with what the volume stored/how it was used. The more I think about it, I might not have been as enthusiastic in my original comment as I should have been. It's been very helpful to me. I don't usually keep things like this running for very long (it wastes cycles on aging hardware...that isn't heavily used, but hey, it's the principal!) but I've kept this around because every time I've thought about removing it, I've visited the dashboard one last time and found something there that made me keep it. [0] Though, as I mentioned, I'm not a sysadmin; I have a lab that might indicate otherwise, but I don't get paid for it. [1] I had reloaded the machine/redone a previous configuration that included bcache and it screamed; I knew my new setup was much slower but I had forgotten about it until netdata made it obvious, again. I can't remember what I had to do to fix it, but it had something to do with the policy used to determine if a file should be put into the cache, and I think it was related to the fact that the cache was added to a volume with data present that rarely changed.

enz 6y ago

Is it any good? ;)

odyslam 6y ago
yes :)
unixhero 6y ago
Very good.
papazach 6y ago
It is great, I have claimed my VMs running Netdata to the Netdata Cloud and I am very happy with it! Took me only a few minutes to claim them all (11 VMs) and boom the dashboards were ready out of the box.

mprovost 6y ago

I write and maintain an open source monitoring tool and I looked into adding a mode to output metrics in Netdata format and ran away screaming. It's just an unstructured text format where you output commands to stdout, one per line. Each command consists of whitespace-separated fields. Which field is the units? Oh, the 4th. And some fields are optional, I'm not even sure how that works but I think you can't skip an optional field if you then want to use any field after that. It's like structured data formats like JSON or god forbid XML never happened.

cakrit 6y ago
Netdata can ingest prometheus metrics as well, so you can just use that format. Eventually everything will become Openmetrics/Opentelemetry

abbaselmas 6y ago

Why netdata is popular on HN now? There must be some big news or something..

andrewm4894 6y ago
i think it was just a random share by someone.

tifadg1 6y ago

Could someone enlighten me on the internals, how is netdata able to get realtime granularity, whereas prometheus defaults to 15s?

distantsounds 6y ago
it polls every second, to get metrics every second. genius, i know?
wongarsu 6y ago
I doesn't store much history from what I can tell. If you don't have years worth of data points then having 15 times as many isn't a big deal.
- mekster 6y ago
  And people are somehow meant to only monitor a single server? There's a reason timeseries databases are trying to get downsampling right.
  - manigandham 6y ago
    It's a locally installed agent that monitors and serves metrics on the same host. If you want to monitor multiple hosts then you can either visit the dashboards individually, or scrape the APIs and put the metrics on a combined dashboard - which is what Netdata Cloud is.
cakrit 6y ago
It's because it was built with high granularity and unlimited metrics as a key differentiator from the beginning. The core is written in pure C, optimized to death. Even long-term retention was initially sacrificed, in order to be able achieve that high performance, with minimal resource needs. Long term retention is now possible, but with relatively high memory requirements, depending on how many metrics are collected. Again, it was a decision to never give up realtime granularity and speed, even at the cost of writing our own timeseries db in C and utilizing more memory.
6y ago

gregwebs 6y ago

Netdata is a great building block in a monitoring system. It now does a lot of monitoring via eBPF, connects to Prometheus, and integrates with k8s.

odyslam 6y ago
We do love ebpf. Guilty as charged ¯\_(ツ)_/¯ We have a whole bunch of metrics that we keep track and we are currently implementing a load more. Soonish, we will greatly increase the number of metrics that we gather with ebpf. That coupled with our per-second granularity, should give you a very detailed view of the system. Docs: https://learn.netdata.cloud/docs/agent/collectors/ebpf.plugin https://learn.netdata.cloud/docs/agent/collectors/ebpf.plugi... Community Forums discussion: https://community.netdata.cloud/t/linux-kernel-insights-with-ebpf/992 https://community.netdata.cloud/t/linux-kernel-insights-with...

petecooper 6y ago

Previous discussions: https://news.ycombinator.com/item?id=11388196 https://news.ycombinator.com/item?id=11388196 https://news.ycombinator.com/item?id=17773874 https://news.ycombinator.com/item?id=17773874 https://news.ycombinator.com/item?id=26886792 https://news.ycombinator.com/item?id=26886792 (For commentary, I'm not being snarky.)

saberience 6y ago

Why would I use it over DataDog?

nine_k 6y ago
Different scale? Different price point?
PanosJee 6y ago
More metrics, higher resolution, pre-configured alert. It cannot displace it as is but it can work in tandem very nicely.

crazypython 6y ago

Haven't been able to use its graphical interface to view historical data. At least it uses fewer resources than Grafana.

distantsounds 6y ago
netdata doesn't store metrics historically but you can funnel whatever ones you want out and ship them off to a log store like graphite or opentsdb.
- PanosJee 6y ago
  It has the option now to set retention for up to a year.
  - distantsounds 6y ago
    ooh, even better!

sneak 6y ago

Note that netdata phones home without consent in the default configuration. For many, the whole point of doing system-administration is selfhosting and autonomy, and privacy is frequently a big component of that. Netdata blows a big hole in that by transmitting your usage information off of your box without getting permission.

odyslam 6y ago
That's a bit unfair. In the docs we are being very upfront that you can opt-out of anonymous telemetry: https://learn.netdata.cloud/docs/get https://learn.netdata.cloud/docs/get we use the data we gather in order to make smarter product decisions. We want to invest resources where it matters, so we need to know how our users use the product. We are also very detailed on what we gather: https://learn.netdata.cloud/docs/agent/netdata-security https://learn.netdata.cloud/docs/agent/netdata-security Lastly, we just changed our analytics engine, from google-analytics to a self-hosted posthog, which is an open-source product analytics platform
- sneak 6y ago
  The telemetry isn't anonymous: it includes the client IP; the method you use to transmit the data cannot work anonymously. Additionally, what's actually unfair is that you proceed with this spying without the consent of the user. Being upfront about it is not obtaining consent: it's just informing the user you're about to violate their (lack of) consent. You must obtain consent from the user first, before transmitting their information. Otherwise, your software is spyware. (Disclosing that you're going to spy on the user doesn't make you not-spyware.) > we use the data we gather in order to make smarter product decisions. Yes, you transmit the private data of the user for the express purpose of enriching yourself. Opt-out is unethical: you must obtain opt-in consent first. The data you are transmitting does not belong to you.
  - yunohn 6y ago
    Dude, seriously? You choose to willfully install Netdata. You have to read the docs where the opt-out telemetry is clearly explained, before you can self-host it too. If you care, you can disable it. I honestly don’t understand HN. Multiple commenters deriding a free open-source project for having basic telemetry to understand feature usage.

hivacruz 6y ago

How does it compare to New Relic who also happens to monitor, if enabled, containers and system things?

mtmsr 6y ago

I have played around with netdata just yesterday on my home server. Great tool, but the defaults are overkill for my needs. After spending an hour trying to simplify (=disable most of the "collectors") using the documentation, I finally gave up. Settled on neofetch [1] instead: pure bash, wrote my own custom inputs including color coding for incident reporting in less time than it took me to strip down netdata. Highly recommended if you want to spend your time on other things than (setting up) server monitoring. [1] https://github.com/dylanaraps/neofetch https://github.com/dylanaraps/neofetch

gregwebs 6y ago
Thanks for the link: neofetch seems a good tool when you just want to manually see what is going on. Netdata is also designed to alert, forward data to other locations, monitor at 1 second granularity, and to store historical data efficiently if you want to see what went on in the recent past.

jitl 6y ago

I run Netdata on my home server using the official docker image. I mostly use it to detect run-away containers, and monitor system temprature. For these use-cases, it works great, and I like that it's self contained; way less headache than stringing together Graphviz stuff, or setting up Nagios or Prometheus. Beware - by default it'll send home telemetry, and the web UI will try to "register" your instance with some kind of cloud. I find this super annoying, but it's possible to turn it off; just not well documented. There's also a lot of plugins that scrape many kinds of logs, look at process data, etc. Again, might be useful, but for a home user it's much better to turn it all off. My notes here have a write-up of how to run Netdata via Docker with example config files that disable the unwanted features here: https://jake.tl/notes/2019-11-09-cozy-server#netdata https://jake.tl/notes/2019-11-09-cozy-server#netdata

linsomniac 6y ago

We have a central influxdb with telegraf metrics among others, and some grafana graphs. I still install netdata on every machine though. Almost never use it, but there have been some times where it was useful to look at netdata. It's light weight enough that it hasn't been a problem.