7 ms·

μMon: Stupid simple monitoring (2022)

290 points by g0xA52A2A 3y ago

golem14 3y ago
Looks nice. I would like to use something like this to remotely monitor machines. Currently use Prometheus (but without Grafana), since the alerting and built-in graphing is sufficient. But agree with OP that Prometheus feels more complex than need be for simple use cases. But so does sendmail ;)
- ilyt 3y ago
  VictoriaMetrics have all-in-one binary that is pretty easy setup for simple one node install.
  - golem14 3y ago
    VictoriaMetrics is a layer on top of Prometheus from a quick read. I managed simple alerts just using Prometheus's alert manager and aws's simple mail system, I prefer this simpler approach.
    - dengolius 3y ago
      > VictoriaMetrics is a layer on top of Prometheus from a quick read. Sorry, but you're wrong! VictoriaMetrics was built from scratch with own ideas [1][2][3] and never was build on top of Prometheus. Yes, it use some libs which are used in Prometheus/InfluxDB/othe open source projects but thats all. More over VictoriaMetrics team has created own query language, named MetricsQL [4] which is inspired by PromQL [5]. > I managed simple alerts just using Prometheus's alert manager and aws's simple mail system, I prefer this simpler approach. In the VictoriaMetrics stack, alerting is placed in a separate utility - vmalert [6] that is responsible for alerts and works with alertmanager as well as Prometheus. [1] https://faun.pub/victoriametrics-creating-the-best-remote-storage-for-prometheus-5d92d66787ac https://faun.pub/victoriametrics-creating-the-best-remote-st... [2] https://valyala.medium.com/open-sourcing-victoriametrics-f31e34485c2b https://valyala.medium.com/open-sourcing-victoriametrics-f31... [3] https://www.youtube.com/watch?v=-DbbIZzFHIY https://www.youtube.com/watch?v=-DbbIZzFHIY [4] https://docs.victoriametrics.com/MetricsQL.html https://docs.victoriametrics.com/MetricsQL.html [5] https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e https://medium.com/@romanhavronenko/victoriametrics-promql-c... [6] https://docs.victoriametrics.com/vmalert.html#vmalert https://docs.victoriametrics.com/vmalert.html#vmalert
pphysch 3y ago
This doesn't seem much simpler than Prometheus+Grafana, if it all. Some pushback: - SNMP sucks. It's very limited, difficult to secure, etc. I've spent a lot of time with it, and it's more complex than Prometheus' simple HTTP metrics model. I use it where I have to (non-server devices), but I prefer dealing with Prometheus. - Grafana is not necessarily complex. It's powerful, and you can waste a lot of time overinvesting in dashboard design, but that's not required. It can be used quite elegantly. μMon does seem like "old school for the sake of old school". SNMP and RRDTool were designed when memory & bandwidth were much more limited. I will happily trade the overheads of HTTP and static Go binaries for the much superior UX they offer.
imiric 3y ago
I like the focus on simplicity of uMon, and agree with author's criticism of behemoths like Grafana. But looking at the installation instructions[1], I can't help but think that their reluctance to use Docker feels contrarian for no reason (and the quip about it being "out of fashion" completely misguided). This whole procedure could be automated in a Dockerfile, and actually running uMon would be vastly simplified. Docker itself is not much more than a wrapper around Linux primitives, and if they dislike it specifically for e.g. having to run a server and run containers as root, there are plenty of other lighterweight container alternatives. There's an argument to be made that the "Simple" Network Management Protocol they're a fan of is far from being simple either[2]. Configuring the security features of v3 is not a simple task, and entire books have been written about SNMP as well. They conveniently ignore this by using v2c and making access public, which might not be acceptable in real-world deployments. I'm all for choosing "simple" tools and stacks over "complex" ones, for whatever definition of those terms one chooses to use, and I strive to do that in my own projects whenever possible, but simplicity is not an inherent property of old and battle-tested technologies. We should be careful to not be biased for technology we happen to be familiar with, but be pragmatic about picking the right tool for the job that fits our requirements, regardless of its age or familiarity. [1]: https://tomscii.sig7.se/umon/#Installation%20and%20getting%20started https://tomscii.sig7.se/umon/#Installation%20and%20getting%2... [2]: I have a pet peeve about tools or protocols with "simple" or "trivial" in their name. They almost always end up being the opposite of that as they mature, and the name becomes an alluring mirage tricking you into its abyss of hidden complexity. I'm looking at you SMTP, TFTP...
- medellin 3y ago
  When i read that they thought docker was outdated I figured i would open the repo to find go code and pre built binaries… when i found c++ i was left thinking the same as you.
  - throwbadubadu 3y ago
    If it's simple doesn't need docker, and as someone else said you might even want to stuff it into your docker containers.. so a minimal thing doesn't need docker imo. The simple landscape is huge, same as for whether one believes that C++ is outdated or not ;) Guess figure what is easier to get up and run still on many platforms.. simple C/C++, or anything else?
toxik 3y ago
What about netdata?
amar0c 3y ago
Monitoring should be: "central" location with GUI/graphs + agents per bunch of servers. Let me chose from dropdown what I want to see. If I have to deploy this on each machine than it makes no sense. I know SNMP is able to be used like this, but is μMon ?
- rcarmo 3y ago
  You can easily put together something that will send UDP packets with stats at regular intervals. I’ve done that a number of times - https://github.com/rcarmo/raspi-cluster/blob/master/tools/servers/c/multicast_statusd.c https://github.com/rcarmo/raspi-cluster/blob/master/tools/se...
wtcactus 3y ago
Nice nifty project, he had me until the "no alerting" part. Anyway, I might still deploy this in a Proxmox homelab where I don't want to fight with the complexity of a grafana dashboard.
- rcarmo 3y ago
  What’s missing from the Proxmox charts that you would need a dashboard for?
  - wtcactus 3y ago
    From the top of my mind (I'm not near the deployment right now): - Temperatures - Fan speeds - Centralized metrics for all VMs
rcarmo 3y ago
Rrdtool is still my go-to for a lot of things. The only functionality I would like to have from it is an SVG version of the charts that allowed for panning and zooming into particular points in the past.
gjulianm 3y ago
I honestly don't get the criticisms of the Prometheus + Grafana stack. > A full-blown time-series database (with gigabytes of rolling on-disk data). Prometheus has a setting that allows you to limit the space used by the database. I'm not sure however how one can do monitoring without a time-series database. > Several Go binaries dozens of megabytes each, also consuming runtime resources. Compared to most monitoring tools I've tested, the Prometheus exporters are usually fairly lightweight in relation to the amount of metrics they generate. Also, "several dozens of megabytes" doesn't seem like too much when we're usually talking about disk spaces in the gigabytes... > Lengthy configuration files and lengthy argument lists to said binaries. Configuration files, yes if you want to change all the defaults. Argument lists, not really. In reality, a Docker deployment of Grafana + Prometheus is 20 lines in a docker-compose.yml file. Configuration files come with defaults if you install it to the system. By the way, I'm not sure that configuring a FastCGI server will be easier than configuring a Docker compose file... > Systems continuously talking to each other over the network (even when nobody is looking at any dashboard), periodically pulling metrics from nodes into Prometheus, which in turn runs all sorts of consolidation routines on that data. A constant source of noise in otherwise idling systems. Not necessarily. Systems talk to each other over the network if you configure them to do so. You can always install a Prometheus + Grafana on every node if you don't want to do central monitoring and you'll have no network noise. > A mind-boggingly complex web front-end (Grafana) with its own database, tons of JavaScript running in my browser, and role-based access control over multiple users. Grafana, complex? I think dragging and dropping panels with query builders that don't even require you to know the query language are far better than defining graphs in shell scripts. > A bespoke query language to pull metrics into dashboards, and lots of specialized knowledge in how to build useful dashboards. It is all meant to be intuitive, but man, is it complicated! Again, this is not a problem of the stack. Building useful dashboards is complicated no matter what tool you use. > maintenance: ongoing upgrades & migrations Not really. Both Prometheus and Grafana are usually very stable and you don't need to upgrade if you don't want to. I have a monitoring stack built with it in my homelab and I haven't updated it in two years, and it still works. Of course I don't have the new shiny features, but it works. To me, it seems that the author is conflating the complexity of the tool with the complexity of monitoring itself. Yes, monitoring is hard. Knowing which metrics to show, which to pull, how to retain them, it's hard. Knowing how to present those metrics to users is also hard. But this tool doesn't solve that. In the end, I don't know how useful it is to make a custom tool that collects very limited metrics based on other ancient, limited, buggy tools (SNMP, RRD, FastCGI...) that is missing even basic UX features like being able to zoom or pan on charts.
evilc00kie 3y ago
Nice work! I often think about the "reinventing the wheel" argument. Isn't open source about diversity? There are so many fork, clones, "Yet another..."'s (yacc, yaml,...). So many times I'm looking for suitable go libraries that solve a certain problem. There might be a few out there but every lib has its own pros and cons. Having the possibility to choose is great. Nothing sucks more than depending on a unmaintained clib nobody cares without alternatives. The only counter-example that comes in my mind is crypto. You don't want to do your own crypto.
- madsbuch 3y ago
  I think it mostly depends on purpose. Crypto is being reinvented several times a day by students needing to understand the mechanics of various algorithms. Personally, I find it rewarding to reimplement something known. There is always a solution when you are stuck, and who knows. Maybe one will develop a better API for the system of something else.
dobin 3y ago
I was also overwhelmed by Grafana and co. In the time required to install it, i coded a simple monitoring alternative DMSR "Does My Shit Run" in python. Each agent has plugins which basically just sends a data structure to the monitoring server, which will display it as yaml. No persistence, history, graphs or similar. uMon looks like a behemoth in comparison. Github: https://github.com/dobin/dmsr https://github.com/dobin/dmsr Live: https://mon.yookiterm.ch https://mon.yookiterm.ch
deleted 3y ago
[deleted]
ilyt 3y ago
https://collectd.org/ https://collectd.org/ does the gathering (and writing to RRDTool database, if you so desire) part very well. Many plugins, easy to add more (just return one line of text) Still need RRD viewere but that's not a huge stack And it scales all the way to hundreds of hosts, as on top of network send/receive of stats it supports few other write formats aside from just RRD files.
- Self-Perfection 3y ago
  I used it for years but somehow it went out of fashion. It is now missing in Ubuntu and Arch linux repos.
basemi 3y ago
Not to be confused with http://umonfw.com/ http://umonfw.com/
deleted 3y ago
[deleted]
Borg3 3y ago
Not bad. I like KISS concepts. I personally run old Cacti instance for monitoring here. Not that simple as uMon, but not very complicated either. And even wrote cacti_con CLI like graph viewer, to see specific port of that fat 100+ ports campus switches I had at work :)
ComputerGuru 3y ago
This really speaks to me - rrdtool is criminally underutilized. Great work! I did something different but in a similar vein for one server network. We had Seq already deployed for log monitoring so instead of setting up a separate network/node/app health monitoring interface I configured everything to regularly ping seq with a structured log message containing the needed data that could be extracted and graphed with seq’s limited oob charting abilities in the dashboard. Not perfect, but simpler.
kburman 3y ago
I really like the idea behind μMon. It reminds me of when software was simpler. I remember using a program called "Everything" by voidtools. It was small but could search a lot of files quickly. Nowadays, some projects use big tools like Elasticsearch just to search a few things. Some even use PostgreSQL, a big database, for small tasks. I wish more software would keep things simple.
- v3ss0n 3y ago
  Whats wrong is Postgres? It is very simple yet very powerful. Can run on very minimal resources , without hogging your CPU
  - xcdzvyn 3y ago
    I think GP meant more that people shouldn't use Postgres for small systems, over SQLite or something.
- dave8088 3y ago
  “Everything” is a must have for me. It’s shocking that Windows doesn’t come with local search that works.
  - sphars 3y ago
    Microsoft has added basically the same functionality to Power Toys, called Run[0], taking heavy inspiration from Everything. But yeah this should be a built-in utility. [0]: https://learn.microsoft.com/en-us/windows/powertoys/run https://learn.microsoft.com/en-us/windows/powertoys/run
    - zacmps 3y ago
      Does it read the ntfs journal? Because that's really what makes everything, everything. It makes me wish I used a journaling filesystem on Linux too.
      - 3y ago
vsviridov 3y ago
Looks more or less like munin...
ahofmann 3y ago
I like the concept of simple monitoring. Simple means it is simple to install, simple to maintain and simple to use. For me, this is netdata. Netdata could be much more, but I just install it on whatever machine and never think about it again. And when something is strange on that machine, I go to http://localhost:19999 http://localhost:19999 and look around.
- defanor 3y ago
  The article's complaints include complexity of JS web interfaces and "eye candy", while netdata's UI requires JS, is quite laggy and jerky, very interactive. I think munin fits better (uses the same RRDtool graphs, too), though possibly its configuration is too lengthy for the requirements.
- jerrac 3y ago
  Not sure I haven't run across it before, but this is the first time I've tried using Netdata. Looks like it is very good for metrics, at least in the 10 minutes I have spent installing it on my local desktop and poking around the ui there. I'm not seeing anything in it for logs, though. I'm guessing it doesn't aggregate or do anything with logs? What do you use for log aggregation and analysis? I'm very interested because I've been getting frustrated with the ELK Stack, and the Prometheus/Grafana/Loki stack has never worked for me. I'm really close to trying to reinvent the wheel...
  - thinkmassive 3y ago
    Prometheus has become ubiquitous for a reason. Exporting metrics on a basic http endpoint for scraping is as simple as you can get. Service discovery adds some complexity, but if you’re operating with any amount of scale that involves dynamically scaling machines then it’s also the simplest model available so far. What about it doesn’t work for you? Edit: I didn’t touch on logging because the post is about metrics. Personally I’ve enjoyed using Loki better than ELK/EFK, but it does have tradeoffs. I’d still be interested to hear why it doesn’t work, so I can keep that in mind when recommending solutions in the future.
louwrentius 3y ago
I do run Grafana + InfluxDB at home and I agree it’s not trivial to setup. Grafana in particular makes creating informative, easy to read graphs/dashboards a PhD worthy endeavor. Yet, I run some hobby projects that collect data and this setup is absolutely perfect for it. I even challenged myself to use SSL for the InfluxDB server (run small CA). Also, I use slack-based alerting through Grafana, for example if a disk would fill up, or something is down. So it’s really about what your needs are. And often, basic metrics about systems like CPU usage, load or network traffic doesn’t tell you anything useful or actionable.
random3 3y ago
it's 2023. Tracing has been around for 20 years(dtrace, x-trace https://cs.brown.edu/~rfonseca/pubs/xtr-nsdi07.pdf https://cs.brown.edu/~rfonseca/pubs/xtr-nsdi07.pdf). Very simple logging, if not structured, while not completely useless it's not very useful either. except for maybe showing some nice charts. Any serious monitoring tool is useful when it can explain things and only tracing gives you causal information.
nine_k 3y ago
I like the idea of simplicity and doing exactly what you need. And the single executable. The words "stupid simple" and "C++" together make me scratch my head though. C++ itself is not simple, and you have to recompile if you need to change something (and sometimes you inevitably do), which is slow. I'd likely go with a relatively simple C program that embeds the FFI for RRDtool and other stuff, and embeds Lua, or, better yet, Janet. Then most of the thing could be written in these languages, and would be easy to tweak when need be. That would still allow for a single executable + a single config file, on top of the logic already embedded. (But the author went and built the thing, and I did not, so their solution so far outperforms mine.)
- uxp8u61q 3y ago
  Simple means simple for the end users. Not necessarily simple for the developers. The end users do not care if the dev has to spend a couple of minutes recompiling, as long as the result is fast and simple to use.
  - nine_k 3y ago
    This is fair. But the developer and the user is one and the same here :)
- deleted 3y ago
  [deleted]
- lost_tourist 3y ago
  c++ can be relatively simple if you don't throw in all the bells and whistles and stay away from template metaprogramming. I use it all the time. Sure compiles can be slow, but with proper file partitioning and creating libraries 95% of that can be controlled for.
mayli 3y ago
fyi, here is a very similar project. https://github.com/pommi/CGP https://github.com/pommi/CGP
roger_ 3y ago
I was recently looking for an ultra minimal monitoring solution for OpenWrt and other lightweight systems (Pi’s, etc.) and was disappointed not to find one that met my needs (negligible CPU, disk space and RAM). I ended up hacking together a shell script to send data to Home Assistant (via MQTT) which runs on pretty much any system that has at least netcat: https://github.com/roger-/hass-sysmon https://github.com/roger-/hass-sysmon
- protoplancton 3y ago
  Did you have a look at https://www.monitorix.org/ https://www.monitorix.org/ ? I'm not sure if it works on OpenWRT, but otherwise checks all the boxes.
  - roger_ 3y ago
    It requires Perl, which ruled out OpenWrt devices with ~10 Kb of space.
- valyala 3y ago
  Did you try VictoriaMetrics [1] and vmagent [2]? It is a single self-contained binary without external dependencies. It requires relatively low amounts of CPU, RAM, disk space and disk IO, and it runs on ARM. [1] https://github.com/VictoriaMetrics/VictoriaMetrics/ https://github.com/VictoriaMetrics/VictoriaMetrics/ [2] https://docs.victoriametrics.com/vmagent.html https://docs.victoriametrics.com/vmagent.html
- synergy20 3y ago
  why not use https://collectd.org/ https://collectd.org/ which is in C and used by openwrt's luci already along with rrdtool, small in size, low on resource, and has so many plugins already?
  - roger_ 3y ago
    I wanted something that would run anywhere with no dependencies. This might work on OpenWrt, but would be trickier to setup on Debian.
new_user_final 3y ago
Is there any simple monitoring system for kubernetes that will monitor memory and CPU usage for each deployments and node? Prometheus and Grafana is good but too much configuration. I also like stats page of HAProxy. Something like that for per service?
posix86 3y ago
Why's everyone hating on Grafana? I find it fairly easy to use, it has a good balance between power & simplicity. And with docker you can make it run in seconds.
- SanderNL 3y ago
  Using Docker to run in seconds is like saying using a Windows VM in Qemu and you got Office running in seconds on Free BSD. It’s still a mess, but it’s hidden now.
- xrd 3y ago
  The difficult thing for me was understanding how to think about and then configure agents. I never could reason about getting the central server up and then connecting nodes to it, whether those nodes were applications or entire servers. The terminology didn't make sense to me and it made me question whether I had the right framing for the purpose of grafana.
  - dewey 3y ago
    In the authors case he used Prometheus and Grafana was just the "frontend" for it. Basically a way to build pretty dashboards our of your PromQL queries. There's no Grafana agent you have to configure.
- keyle 3y ago
  No one is "hating" on Grafana. But if you used tools of the past, you'd know how light they were compared to the Grafana/Kibana/Elastic mongrel. But I should mention that there is nothing "stupid simple" about the solution of the OP. Just look at the install procedure... https://tomscii.sig7.se/umon/ https://tomscii.sig7.se/umon/
  - posix86 3y ago
    what in the... what is the point?? I've used Grafana many times and it's always, always a simple docker start -v, ..., and modifying some configs to get started. This tool is completely useless.

290 points by g0xA52A2A 3y ago

Looks nice. I would like to use something like this to remotely monitor machines. Currently use Prometheus (but without Grafana), since the alerting and built-in graphing is sufficient. But agree with OP that Prometheus feels more complex than need be for simple use cases. But so does sendmail ;)

ilyt 3y ago
VictoriaMetrics have all-in-one binary that is pretty easy setup for simple one node install.
- golem14 3y ago
  VictoriaMetrics is a layer on top of Prometheus from a quick read. I managed simple alerts just using Prometheus's alert manager and aws's simple mail system, I prefer this simpler approach.
  - dengolius 3y ago
    > VictoriaMetrics is a layer on top of Prometheus from a quick read. Sorry, but you're wrong! VictoriaMetrics was built from scratch with own ideas [1][2][3] and never was build on top of Prometheus. Yes, it use some libs which are used in Prometheus/InfluxDB/othe open source projects but thats all. More over VictoriaMetrics team has created own query language, named MetricsQL [4] which is inspired by PromQL [5]. > I managed simple alerts just using Prometheus's alert manager and aws's simple mail system, I prefer this simpler approach. In the VictoriaMetrics stack, alerting is placed in a separate utility - vmalert [6] that is responsible for alerts and works with alertmanager as well as Prometheus. [1] https://faun.pub/victoriametrics-creating-the-best-remote-storage-for-prometheus-5d92d66787ac https://faun.pub/victoriametrics-creating-the-best-remote-st... [2] https://valyala.medium.com/open-sourcing-victoriametrics-f31e34485c2b https://valyala.medium.com/open-sourcing-victoriametrics-f31... [3] https://www.youtube.com/watch?v=-DbbIZzFHIY https://www.youtube.com/watch?v=-DbbIZzFHIY [4] https://docs.victoriametrics.com/MetricsQL.html https://docs.victoriametrics.com/MetricsQL.html [5] https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e https://medium.com/@romanhavronenko/victoriametrics-promql-c... [6] https://docs.victoriametrics.com/vmalert.html#vmalert https://docs.victoriametrics.com/vmalert.html#vmalert

pphysch 3y ago

This doesn't seem much simpler than Prometheus+Grafana, if it all. Some pushback: - SNMP sucks. It's very limited, difficult to secure, etc. I've spent a lot of time with it, and it's more complex than Prometheus' simple HTTP metrics model. I use it where I have to (non-server devices), but I prefer dealing with Prometheus. - Grafana is not necessarily complex. It's powerful, and you can waste a lot of time overinvesting in dashboard design, but that's not required. It can be used quite elegantly. μMon does seem like "old school for the sake of old school". SNMP and RRDTool were designed when memory & bandwidth were much more limited. I will happily trade the overheads of HTTP and static Go binaries for the much superior UX they offer.

imiric 3y ago

I like the focus on simplicity of uMon, and agree with author's criticism of behemoths like Grafana. But looking at the installation instructions[1], I can't help but think that their reluctance to use Docker feels contrarian for no reason (and the quip about it being "out of fashion" completely misguided). This whole procedure could be automated in a Dockerfile, and actually running uMon would be vastly simplified. Docker itself is not much more than a wrapper around Linux primitives, and if they dislike it specifically for e.g. having to run a server and run containers as root, there are plenty of other lighterweight container alternatives. There's an argument to be made that the "Simple" Network Management Protocol they're a fan of is far from being simple either[2]. Configuring the security features of v3 is not a simple task, and entire books have been written about SNMP as well. They conveniently ignore this by using v2c and making access public, which might not be acceptable in real-world deployments. I'm all for choosing "simple" tools and stacks over "complex" ones, for whatever definition of those terms one chooses to use, and I strive to do that in my own projects whenever possible, but simplicity is not an inherent property of old and battle-tested technologies. We should be careful to not be biased for technology we happen to be familiar with, but be pragmatic about picking the right tool for the job that fits our requirements, regardless of its age or familiarity. [1]: https://tomscii.sig7.se/umon/#Installation%20and%20getting%20started https://tomscii.sig7.se/umon/#Installation%20and%20getting%2... [2]: I have a pet peeve about tools or protocols with "simple" or "trivial" in their name. They almost always end up being the opposite of that as they mature, and the name becomes an alluring mirage tricking you into its abyss of hidden complexity. I'm looking at you SMTP, TFTP...

medellin 3y ago
When i read that they thought docker was outdated I figured i would open the repo to find go code and pre built binaries… when i found c++ i was left thinking the same as you.
- throwbadubadu 3y ago
  If it's simple doesn't need docker, and as someone else said you might even want to stuff it into your docker containers.. so a minimal thing doesn't need docker imo. The simple landscape is huge, same as for whether one believes that C++ is outdated or not ;) Guess figure what is easier to get up and run still on many platforms.. simple C/C++, or anything else?

toxik 3y ago

What about netdata?

amar0c 3y ago

Monitoring should be: "central" location with GUI/graphs + agents per bunch of servers. Let me chose from dropdown what I want to see. If I have to deploy this on each machine than it makes no sense. I know SNMP is able to be used like this, but is μMon ?

rcarmo 3y ago
You can easily put together something that will send UDP packets with stats at regular intervals. I’ve done that a number of times - https://github.com/rcarmo/raspi-cluster/blob/master/tools/servers/c/multicast_statusd.c https://github.com/rcarmo/raspi-cluster/blob/master/tools/se...

wtcactus 3y ago

Nice nifty project, he had me until the "no alerting" part. Anyway, I might still deploy this in a Proxmox homelab where I don't want to fight with the complexity of a grafana dashboard.

rcarmo 3y ago
What’s missing from the Proxmox charts that you would need a dashboard for?
- wtcactus 3y ago
  From the top of my mind (I'm not near the deployment right now): - Temperatures - Fan speeds - Centralized metrics for all VMs

rcarmo 3y ago

Rrdtool is still my go-to for a lot of things. The only functionality I would like to have from it is an SVG version of the charts that allowed for panning and zooming into particular points in the past.

gjulianm 3y ago

I honestly don't get the criticisms of the Prometheus + Grafana stack. > A full-blown time-series database (with gigabytes of rolling on-disk data). Prometheus has a setting that allows you to limit the space used by the database. I'm not sure however how one can do monitoring without a time-series database. > Several Go binaries dozens of megabytes each, also consuming runtime resources. Compared to most monitoring tools I've tested, the Prometheus exporters are usually fairly lightweight in relation to the amount of metrics they generate. Also, "several dozens of megabytes" doesn't seem like too much when we're usually talking about disk spaces in the gigabytes... > Lengthy configuration files and lengthy argument lists to said binaries. Configuration files, yes if you want to change all the defaults. Argument lists, not really. In reality, a Docker deployment of Grafana + Prometheus is 20 lines in a docker-compose.yml file. Configuration files come with defaults if you install it to the system. By the way, I'm not sure that configuring a FastCGI server will be easier than configuring a Docker compose file... > Systems continuously talking to each other over the network (even when nobody is looking at any dashboard), periodically pulling metrics from nodes into Prometheus, which in turn runs all sorts of consolidation routines on that data. A constant source of noise in otherwise idling systems. Not necessarily. Systems talk to each other over the network if you configure them to do so. You can always install a Prometheus + Grafana on every node if you don't want to do central monitoring and you'll have no network noise. > A mind-boggingly complex web front-end (Grafana) with its own database, tons of JavaScript running in my browser, and role-based access control over multiple users. Grafana, complex? I think dragging and dropping panels with query builders that don't even require you to know the query language are far better than defining graphs in shell scripts. > A bespoke query language to pull metrics into dashboards, and lots of specialized knowledge in how to build useful dashboards. It is all meant to be intuitive, but man, is it complicated! Again, this is not a problem of the stack. Building useful dashboards is complicated no matter what tool you use. > maintenance: ongoing upgrades & migrations Not really. Both Prometheus and Grafana are usually very stable and you don't need to upgrade if you don't want to. I have a monitoring stack built with it in my homelab and I haven't updated it in two years, and it still works. Of course I don't have the new shiny features, but it works. To me, it seems that the author is conflating the complexity of the tool with the complexity of monitoring itself. Yes, monitoring is hard. Knowing which metrics to show, which to pull, how to retain them, it's hard. Knowing how to present those metrics to users is also hard. But this tool doesn't solve that. In the end, I don't know how useful it is to make a custom tool that collects very limited metrics based on other ancient, limited, buggy tools (SNMP, RRD, FastCGI...) that is missing even basic UX features like being able to zoom or pan on charts.

evilc00kie 3y ago

Nice work! I often think about the "reinventing the wheel" argument. Isn't open source about diversity? There are so many fork, clones, "Yet another..."'s (yacc, yaml,...). So many times I'm looking for suitable go libraries that solve a certain problem. There might be a few out there but every lib has its own pros and cons. Having the possibility to choose is great. Nothing sucks more than depending on a unmaintained clib nobody cares without alternatives. The only counter-example that comes in my mind is crypto. You don't want to do your own crypto.

madsbuch 3y ago
I think it mostly depends on purpose. Crypto is being reinvented several times a day by students needing to understand the mechanics of various algorithms. Personally, I find it rewarding to reimplement something known. There is always a solution when you are stuck, and who knows. Maybe one will develop a better API for the system of something else.

dobin 3y ago

I was also overwhelmed by Grafana and co. In the time required to install it, i coded a simple monitoring alternative DMSR "Does My Shit Run" in python. Each agent has plugins which basically just sends a data structure to the monitoring server, which will display it as yaml. No persistence, history, graphs or similar. uMon looks like a behemoth in comparison. Github: https://github.com/dobin/dmsr https://github.com/dobin/dmsr Live: https://mon.yookiterm.ch https://mon.yookiterm.ch

deleted 3y ago

[deleted]

ilyt 3y ago

https://collectd.org/ https://collectd.org/ does the gathering (and writing to RRDTool database, if you so desire) part very well. Many plugins, easy to add more (just return one line of text) Still need RRD viewere but that's not a huge stack And it scales all the way to hundreds of hosts, as on top of network send/receive of stats it supports few other write formats aside from just RRD files.

Self-Perfection 3y ago
I used it for years but somehow it went out of fashion. It is now missing in Ubuntu and Arch linux repos.

basemi 3y ago

Not to be confused with http://umonfw.com/ http://umonfw.com/

deleted 3y ago

[deleted]

Borg3 3y ago

Not bad. I like KISS concepts. I personally run old Cacti instance for monitoring here. Not that simple as uMon, but not very complicated either. And even wrote cacti_con CLI like graph viewer, to see specific port of that fat 100+ ports campus switches I had at work :)

ComputerGuru 3y ago

This really speaks to me - rrdtool is criminally underutilized. Great work! I did something different but in a similar vein for one server network. We had Seq already deployed for log monitoring so instead of setting up a separate network/node/app health monitoring interface I configured everything to regularly ping seq with a structured log message containing the needed data that could be extracted and graphed with seq’s limited oob charting abilities in the dashboard. Not perfect, but simpler.

kburman 3y ago

I really like the idea behind μMon. It reminds me of when software was simpler. I remember using a program called "Everything" by voidtools. It was small but could search a lot of files quickly. Nowadays, some projects use big tools like Elasticsearch just to search a few things. Some even use PostgreSQL, a big database, for small tasks. I wish more software would keep things simple.

v3ss0n 3y ago
Whats wrong is Postgres? It is very simple yet very powerful. Can run on very minimal resources , without hogging your CPU
- xcdzvyn 3y ago
  I think GP meant more that people shouldn't use Postgres for small systems, over SQLite or something.
dave8088 3y ago
“Everything” is a must have for me. It’s shocking that Windows doesn’t come with local search that works.
- sphars 3y ago
  Microsoft has added basically the same functionality to Power Toys, called Run[0], taking heavy inspiration from Everything. But yeah this should be a built-in utility. [0]: https://learn.microsoft.com/en-us/windows/powertoys/run https://learn.microsoft.com/en-us/windows/powertoys/run
  - zacmps 3y ago
    Does it read the ntfs journal? Because that's really what makes everything, everything. It makes me wish I used a journaling filesystem on Linux too.
    - 3y ago

vsviridov 3y ago

Looks more or less like munin...

ahofmann 3y ago

I like the concept of simple monitoring. Simple means it is simple to install, simple to maintain and simple to use. For me, this is netdata. Netdata could be much more, but I just install it on whatever machine and never think about it again. And when something is strange on that machine, I go to http://localhost:19999 http://localhost:19999 and look around.

defanor 3y ago
The article's complaints include complexity of JS web interfaces and "eye candy", while netdata's UI requires JS, is quite laggy and jerky, very interactive. I think munin fits better (uses the same RRDtool graphs, too), though possibly its configuration is too lengthy for the requirements.
jerrac 3y ago
Not sure I haven't run across it before, but this is the first time I've tried using Netdata. Looks like it is very good for metrics, at least in the 10 minutes I have spent installing it on my local desktop and poking around the ui there. I'm not seeing anything in it for logs, though. I'm guessing it doesn't aggregate or do anything with logs? What do you use for log aggregation and analysis? I'm very interested because I've been getting frustrated with the ELK Stack, and the Prometheus/Grafana/Loki stack has never worked for me. I'm really close to trying to reinvent the wheel...
- thinkmassive 3y ago
  Prometheus has become ubiquitous for a reason. Exporting metrics on a basic http endpoint for scraping is as simple as you can get. Service discovery adds some complexity, but if you’re operating with any amount of scale that involves dynamically scaling machines then it’s also the simplest model available so far. What about it doesn’t work for you? Edit: I didn’t touch on logging because the post is about metrics. Personally I’ve enjoyed using Loki better than ELK/EFK, but it does have tradeoffs. I’d still be interested to hear why it doesn’t work, so I can keep that in mind when recommending solutions in the future.

louwrentius 3y ago

I do run Grafana + InfluxDB at home and I agree it’s not trivial to setup. Grafana in particular makes creating informative, easy to read graphs/dashboards a PhD worthy endeavor. Yet, I run some hobby projects that collect data and this setup is absolutely perfect for it. I even challenged myself to use SSL for the InfluxDB server (run small CA). Also, I use slack-based alerting through Grafana, for example if a disk would fill up, or something is down. So it’s really about what your needs are. And often, basic metrics about systems like CPU usage, load or network traffic doesn’t tell you anything useful or actionable.

random3 3y ago

it's 2023. Tracing has been around for 20 years(dtrace, x-trace https://cs.brown.edu/~rfonseca/pubs/xtr-nsdi07.pdf https://cs.brown.edu/~rfonseca/pubs/xtr-nsdi07.pdf). Very simple logging, if not structured, while not completely useless it's not very useful either. except for maybe showing some nice charts. Any serious monitoring tool is useful when it can explain things and only tracing gives you causal information.

nine_k 3y ago

I like the idea of simplicity and doing exactly what you need. And the single executable. The words "stupid simple" and "C++" together make me scratch my head though. C++ itself is not simple, and you have to recompile if you need to change something (and sometimes you inevitably do), which is slow. I'd likely go with a relatively simple C program that embeds the FFI for RRDtool and other stuff, and embeds Lua, or, better yet, Janet. Then most of the thing could be written in these languages, and would be easy to tweak when need be. That would still allow for a single executable + a single config file, on top of the logic already embedded. (But the author went and built the thing, and I did not, so their solution so far outperforms mine.)

uxp8u61q 3y ago
Simple means simple for the end users. Not necessarily simple for the developers. The end users do not care if the dev has to spend a couple of minutes recompiling, as long as the result is fast and simple to use.
- nine_k 3y ago
  This is fair. But the developer and the user is one and the same here :)
deleted 3y ago
[deleted]
lost_tourist 3y ago
c++ can be relatively simple if you don't throw in all the bells and whistles and stay away from template metaprogramming. I use it all the time. Sure compiles can be slow, but with proper file partitioning and creating libraries 95% of that can be controlled for.

mayli 3y ago

fyi, here is a very similar project. https://github.com/pommi/CGP https://github.com/pommi/CGP

roger_ 3y ago

I was recently looking for an ultra minimal monitoring solution for OpenWrt and other lightweight systems (Pi’s, etc.) and was disappointed not to find one that met my needs (negligible CPU, disk space and RAM). I ended up hacking together a shell script to send data to Home Assistant (via MQTT) which runs on pretty much any system that has at least netcat: https://github.com/roger-/hass-sysmon https://github.com/roger-/hass-sysmon

protoplancton 3y ago
Did you have a look at https://www.monitorix.org/ https://www.monitorix.org/ ? I'm not sure if it works on OpenWRT, but otherwise checks all the boxes.
- roger_ 3y ago
  It requires Perl, which ruled out OpenWrt devices with ~10 Kb of space.
valyala 3y ago
Did you try VictoriaMetrics [1] and vmagent [2]? It is a single self-contained binary without external dependencies. It requires relatively low amounts of CPU, RAM, disk space and disk IO, and it runs on ARM. [1] https://github.com/VictoriaMetrics/VictoriaMetrics/ https://github.com/VictoriaMetrics/VictoriaMetrics/ [2] https://docs.victoriametrics.com/vmagent.html https://docs.victoriametrics.com/vmagent.html
synergy20 3y ago
why not use https://collectd.org/ https://collectd.org/ which is in C and used by openwrt's luci already along with rrdtool, small in size, low on resource, and has so many plugins already?
- roger_ 3y ago
  I wanted something that would run anywhere with no dependencies. This might work on OpenWrt, but would be trickier to setup on Debian.

new_user_final 3y ago

Is there any simple monitoring system for kubernetes that will monitor memory and CPU usage for each deployments and node? Prometheus and Grafana is good but too much configuration. I also like stats page of HAProxy. Something like that for per service?

posix86 3y ago

Why's everyone hating on Grafana? I find it fairly easy to use, it has a good balance between power & simplicity. And with docker you can make it run in seconds.

SanderNL 3y ago
Using Docker to run in seconds is like saying using a Windows VM in Qemu and you got Office running in seconds on Free BSD. It’s still a mess, but it’s hidden now.
xrd 3y ago
The difficult thing for me was understanding how to think about and then configure agents. I never could reason about getting the central server up and then connecting nodes to it, whether those nodes were applications or entire servers. The terminology didn't make sense to me and it made me question whether I had the right framing for the purpose of grafana.
- dewey 3y ago
  In the authors case he used Prometheus and Grafana was just the "frontend" for it. Basically a way to build pretty dashboards our of your PromQL queries. There's no Grafana agent you have to configure.
keyle 3y ago
No one is "hating" on Grafana. But if you used tools of the past, you'd know how light they were compared to the Grafana/Kibana/Elastic mongrel. But I should mention that there is nothing "stupid simple" about the solution of the OP. Just look at the install procedure... https://tomscii.sig7.se/umon/ https://tomscii.sig7.se/umon/
- posix86 3y ago
  what in the... what is the point?? I've used Grafana many times and it's always, always a simple docker start -v, ..., and modifying some configs to get started. This tool is completely useless.