10 ms·

Grafana 7.0

345 points by pjf 6y ago

telaelit 6y ago
I love Grafana so much!! Keep up the great work!!
torkelo 6y ago
Hi, my name is Torkel Ödegaard. Grafana project creator & lead. Here to answer any questions.
- bauerd 6y ago
  Does the new plugin architecture allow me to provision plugins from a config file on startup? Like installing the Kubernetes App without `grafana-cli plugins install kubernetes-app`
  - ryantxu 6y ago
    You can provision plugins on startup with something like: https://grafana.com/docs/grafana/latest/installation/docker/#install-official-and-community-grafana-plugins https://grafana.com/docs/grafana/latest/installation/docker/...
    - mleonhard 6y ago
      Does that method support removing plugins?
  - deleted 6y ago
    [deleted]
  - ulzeraj 6y ago
    Someone already replied but I should add that most if not all config parameters can be passed to the container through env variables.
- thunfischbrot 6y ago
  Thank you for this amazing, helpful project! Are there plans for letting power-users style their own custom panels using html, css and js using just the web interface? Meaning without having to develop a plugin.
  - danlimerick 6y ago
    This has been possible for ages. The built-in text panel has an HTML option (though you have to disable this setting to allow script tags: https://grafana.com/docs/grafana/latest/installation/configuration/#disable-sanitize-html https://grafana.com/docs/grafana/latest/installation/configu...) There are some external plugins that offer more advanced support: - https://grafana.com/grafana/plugins/aidanmountford-html-panel https://grafana.com/grafana/plugins/aidanmountford-html-pane... - https://grafana.com/grafana/plugins/ryantxu-ajax-panel https://grafana.com/grafana/plugins/ryantxu-ajax-panel
  - ryantxu 6y ago
    There are a few plugins that let you enter css/html/js directly -- see https://grafana.com/grafana/plugins/ryantxu-ajax-panel https://grafana.com/grafana/plugins/ryantxu-ajax-panel https://github.com/srclosson/grafana-div-panel https://github.com/srclosson/grafana-div-panel and the text panel (if the server is configured with disable_sanitize_html = true) this is powerful, but long term tends to lead to a maintenance nightmare
- iampims 6y ago
  Do you know if performance has improved for large dashboards?
  - ryantxu 6y ago
    since when? :) A few things have been added that may help: 1. only visible data is queried and loaded 2. If you have multiple panels that use the same data, you can now just reuse the results (or a subset of the results) in a new panel. This will cut down on additional queries that may not be necessary
    - leeoniya 6y ago
      i see you still have plans to continue with uPlot testing :) https://github.com/grafana/grafana/pull/21835 https://github.com/grafana/grafana/pull/21835
      - ryantxu 6y ago
        absolutely -- with everything else for 7.0 that did not make the cut. But would love to work more with your frankly amazing library :)
- Systemic33 6y ago
  Hi Torkel, First off, Grafana is amazing! So keep up the good work. Question: Currently alerts are fairly limited both in applicable panels and how they export/share relevant data to the chosen notification channel, as well as more advanced conditions. Is there any work going on with improving alerts?
  - dig1 6y ago
    Also to mention frequently asked feature: alerts using queries and templated variables [1][2]. Creating even simple generic dashboards with alerts is not possible, unless variables are hardcoded in hidden queries. [1] https://community.grafana.com/t/template-variables-are-not-supported-in-alert-queries-while-setting-up-alert/2514 https://community.grafana.com/t/template-variables-are-not-s... [2] https://github.com/grafana/grafana/issues/6557 https://github.com/grafana/grafana/issues/6557
    - atrandom 6y ago
      yes, this is a huge pain point for us as well
    - hobochili 6y ago
      Agreed. If Grafana alerts supported templated variables I would migrate a decent chunk of my monitoring stack. But without templated variables the alert system feels half baked and I almost prefer it not even be a feature.
- kentosi 6y ago
  Hi, I used Grafana in my last company and absolutely loved it. How do I convince my current company to get the hell off Datadog?
  - mefraimsson 6y ago
    The cost aspect is one thing. Vendor lock-in another. Probably need to have a team that can lead the transition towards Grafana, educate and speak for it. Grafana has an enterprise plugin for datadog that potentially could ease the migration
  - stitched2gethr 6y ago
    We just moved everything from datadog to Grafana, Prometheus and Loki, mainly due to cost. Honestly though, I feel like we get more out of the Grafana stack. Its powerful but simple enough to keep it pleasant to work with.
  - bsaul 6y ago
    Haha, my team is going the opposite direction: we're leaving prometheus + grafana to datadog, and we're loving it. having every AWS monitoring default dashboard work right out of the bat is a bliss..
  - user5994461 6y ago
    How about don't? Datadog offers everything that grafana does plus alerting, as well as more and better integrations (databases, AWS, middleware, etc...).
- OldFatCactus 6y ago
  have you seen any implementations of internal tracing like APM being used with Grafana? I think lack of a good open source alternative to New Relic or Datadog APM kinda killed OSS monitoring for my company. The Jaeger integration is a good step in the right direction though, kudos!
  - akulkarni 6y ago
    What more would you like to see than what Jaeger provides?
    - OldFatCactus 6y ago
      Jaeger is used for traces between distributed systems (external) Traditional APM products are internal traces
      - malkia 6y ago
        AFAIK, You can have internal trace markers, not just between apps. I've used that (for Jaegger) in one of our Windows Application to see the call-graph (it's a level editor for our game), and it was tracking the loading of various resources across multiple threads, then it was visualized with jaeger and even zipkin.
- dharmab 6y ago
  Thanks for leading the Grafana project! As a color blind user, I find it difficult to read the dashboards created by others. How can I help any efforts to increase Grafana's accessibility?
  - VectorLock 6y ago
    I'm also colorblind and a heavy Grafana user but I haven't had much difficulty reading dashboards. What are you having trouble with? The most trouble I've had is colors of multiple lines on a chart being too similar but being able to use the mouse to select a line and see what the line is and its value helps there.
    - dharmab 6y ago
      The default palette is definitely a challenge, since it has pretty low contrast and Grafana's defaults use a lot of thin lines. I especially have trouble matching colors in a legend with the drawn objects in the graph. The best help has been the hover panel, but it's unsorted by default and the highlight contrast on the panel for hovering over a line is pretty dim.
  - ryantxu 6y ago
    Upvote: https://github.com/grafana/grafana/issues/18041 https://github.com/grafana/grafana/issues/18041 and add any additional comments and context there. In addition to making the light/dark themes more accessible, I think we should add an additional high contrast one.
- imwill 6y ago
  Hi Torkel, Just finished watching the GrafanaCon 7.0 Release talk. There are some great Usability improvements and new functionality which will be definitely useful for my upcoming project. Thanks for the great work!
  - torkelo 6y ago
    Thanks!
- dreamer_soul 6y ago
  Amazing work we have been experimenting with grafana at work and the experience is awesome! Thank you for the awesome project
- veesahni 6y ago
  Are we any closer to having some kind of anomaly detection in grafana (when using prometheus)?
  - cranekam 6y ago
    Just curious: what kind of anomaly detection are you looking for? Can you describe the kind of thing you'd want this system to detect?
    - veesahni 6y ago
      We have all sorts of data flowing into Prometheus and charted in Grafana. Grafana lets me alert based on thresholds and that works fine for many things. EXAMPLE 1: Imagine a service. Load avg of 2 is unacceptable and means things will be slowing down. However, a heavy job runs every night and pushes load averages between 1.5 and 2.5 during a 30 minute window.. I wanted to set a load avg alert threshold @ 2.0 but can't since it would fire false alarms too often. So i end up setting it at 3, and end up getting a late alert when things get really bad. Instead, an anomaly detection would look for a number of standard deviations away from a mean, and ideally account for seasonality (i.e. compare the data with same data last few weeks). Now, things will behave better. If loads hit 2 during the day, it'll fire. But during that 30 minute window at night, loads would need to exceed 3 before it fires. EXAMPLE 2: Let's say I'm tracking 30 metrics for a service. I understand the service well enough to set thresholds in 5 metrics. However, I'm sure a failure won't happen alone and the other 25 metrics may show some oddities and give me an early warning of "something wrong happening" .. Here, some kind of loose "anomaly alerts" could help as an early warning. If a certain metric throws too many false alerts, we find a better way to monitor it. EXAMPLE 3: Lets say you have a service that auto scales. There's no limit to how many requests the service handles. So I can't really thresholds on requests/5min. Here, some kind of "anomaly alert" would be useful to let the team know that something is going on that's causing a high level of requests. Again, accounting for seasonality because things tend to change on weekends. At the end of the day, an anomaly alert is not some "magic solution" but just and extra tool in the toolkit to use when it makes sense. Datadog and AWS both have built in easy to use anomaly alerting. Would be nice for grafana/prometheus to have something similar.
- elagost 6y ago
  At work we have to compile everything from source because compliance, and Grafana is one of the harder ones to get right. There used to be good documentation for it a few versions back but I can't seem to find it lately. Is there any good documentation on how to get Grafana compiled from source?
  - danlimerick 6y ago
    Checkout the contribute folder on GitHub. Here is the developer guide: https://github.com/grafana/grafana/blob/master/contribute/developer-guide.md https://github.com/grafana/grafana/blob/master/contribute/de...
  - imwill 6y ago
    I just built it from sources because of the same reason. What are the issues on your side?
    - elagost 6y ago
      I've got a complicated proxy setup to contend with and have to build via Jenkins on some older Linux boxes, so it's just trial and error. You have a build script you'd be willing to share?
      - ooOOoo 6y ago
        We are building from source with Docker. Without using Docker, you could try to just run the commands in https://github.com/grafana/grafana/blob/master/Dockerfile https://github.com/grafana/grafana/blob/master/Dockerfile
  - mefraimsson 6y ago
    Please let us know and/or open an GitHub issue describing if there are any certain part/step making it hard to build from source. Thanks
- heipei 6y ago
  Just wanted to say thanks for an awesome piece of software. It's one of those "I can't believe it's doing it and doing it this well, and it's free" kind of projects. For me, Grafana together with ElasticSearch and InfluxDB+Telegraf is pure monitoring/analysis/alerting bliss. Stay true to your roots and don't get tempted to turn into Kibana ;)
- qwertox 6y ago
  Is it now possible to create a horizontal lined based on a single value (either a fixed or a computed number)? For example when I'm tracking the power grid frequency, I would like to add two lines at 50.05 and 49.95 Hz, as well as one at 50 Hz (with different color). Others have voiced their desire to display a horizontal line at the 95th percentile in a bandwidth usage graph.
  - danlimerick 6y ago
    Maybe the thresholds feature in Grafana would work for you? Otherwise depending on the database, they have functions for drawing lines or you can do a hack like this one for Prometheus: https://play.grafana.org/d/h1lglb7ik/threshold-hack?editPanel=2&orgId=1 https://play.grafana.org/d/h1lglb7ik/threshold-hack?editPane...
- thruhiker 6y ago
  I just wanted to say how much I appreciate your work on Grafana. It's an indispensable tool that I couldn't do my job without!
  - torkelo 6y ago
    Thank you!
- deleted 6y ago
  [deleted]
- AustinBGibbons 6y ago
  Hey! Thanks for grafana, and especially a HUGE thank you for the user usage metrics! Now if only you could persuade kibana to do the same :-)
- em-bee 6y ago
  when you post release announcements that possibly get posted snd forwarded to other places, please include a short description about what grafana is. and maybe also add one to your home page. (make it link to the features) when i see any release anouncement about a product i am unfamiliar with, the first question i have is what kind of product it is, so i can decide if i want to check it out
- guanzo 6y ago
  Zero experience with Grafana here. I have a mongodb database with several collections. All collections share a set of fields: "createdAt", "channelId", "status", etc. How easy would it be to dump all this data into Grafana, and then be able to slice and dice these collections by any combination of the shared fields. Basically, I'm tired of writing mongo queries whenever I want to inspect my data. I want to point and click around a beautiful UI, with data visualizations and easy filtering/aggregation. Is this easy to do with Grafana?
  - agacera 6y ago
    I don't believe grafana will help you here. If you have mongodb enterprise you can use Mongo bi connector and connect it with any bi tool. I recommend metabase since it is open source and very good for slice and dice and reports. The mongodb bi connector is a postgres with foreign data wrapper to mongodb. So even if you don't have a mongodb subscription, you may be able to pull this off using the OS version https://github.com/EnterpriseDB/mongo_fdw https://github.com/EnterpriseDB/mongo_fdw
  - redis_mlc 6y ago
    Grafana is a UI, not a database (it doesn't store any data.) Plugins are used for showing pie charts, etc.
  - viraptor 6y ago
    I've never tried this one, but the mongo datasource plugin exists: https://github.com/JamesOsgood/mongodb-grafana https://github.com/JamesOsgood/mongodb-grafana Maybe you can use it directly this way? (if you don't mind the BI-equivalent hammering your primary data source)
- btmiller 6y ago
  Hi Torkel, this issue has been open since 2016[1]. There is incredible demand for it yet complete silence from Grafana on committing to the effort. Can I get official word on this please? [1] https://github.com/grafana/grafana/issues/6983 https://github.com/grafana/grafana/issues/6983
  - gempir 6y ago
    Have there been no efforts to add this with a PR? I mean Grafana is open source and if the project leads wants to focus on other things that's his choice.
    - btmiller 6y ago
      Attempts were made: https://github.com/grafana/grafana/pull/14075 https://github.com/grafana/grafana/pull/14075
  - torkelo 6y ago
    The reason is we want to move alert rules out of the dashboard. To be their own thing, have their own UI & workflow for editing and updating. Once that is done, we plan to add the ability to link an alert rule to 1 or more dashboard panels.
- morb 6y ago
  Will Grafana ever add direct support for rrd datasources?
  - wazoox 6y ago
    Does this help? https://github.com/doublemarket/grafana-rrd-server https://github.com/doublemarket/grafana-rrd-server
    - morb 6y ago
      Kinda, but not really. It has no support for rrdcached flush, AVERAGE CF is hardcoded , and I cannot seem to get it to not connect lines through gaps where there were no measurements. Example not related to rrd, but same problem different db: https://github.com/grafana/grafana/issues/2125 https://github.com/grafana/grafana/issues/2125.
- mleonhard 6y ago
  How about adding support for configuring Grafana with files? This would allow us to deploy immutable Grafana instances using infrastructure-as-code tools like Terraform and Docker. Specifically, I want a way to eliminate post-install steps such as adding users and organizations and setting user passwords.
  - dharmab 6y ago
    Wouldn't a better way to configure users and passwords be to use an external auth provider such as OAuth or LDAP? If you don't already have a auth provider, you could deploy Keycloak as your user DB.
    - mleonhard 6y ago
      OAuth/LDAP are great for large organizations. Small organizations don't need such complexity. Even if you use OAuth/LDAP for accounts and passwords, Grafana still requires manual steps to create the organization, install plugins, and set the admin password. I wrote a Python script to do this on Docker container startup. That script is a workaround for Grafana server's poor ops design. It's extra code that we must maintain.
      - dharmab 6y ago
        The admin password can be set with an environment variable. That just leaves the org and plugins.
- mleonhard 6y ago
  Do you plan to make debug logging useful? I had difficulty troubleshooting Grafana & InfluxDB auth problems and found that debug logging did not provide any extra detail.
- mleonhard 6y ago
  Thanks for adding more support for logs in Grafana. I'm not using AWS CloudWatch Logs because of the financial risk of a runaway server running up a huge AWS bill. I had difficulty getting log display and log search working well in Grafana 6.6.2 and InfluxDB. Do you plan to improve log support for other databases?
- hashhar 6y ago
  Thanks a lot for creating it and pushing it forward. It's my go-to tool for monitoring dashboards and also some simple interactive data dashboards. Never had any issues with it (and the ones I had were promptly fixed by the devs and the community).
deleted 6y ago
[deleted]
nepotism2018 6y ago
Awesome tool. Supports all the datasources I need.
imwill 6y ago
Custom Themes seems to be one of the most wanted features of the Community. Is there any plan implementing it in the near future? Recently I digged into the source code to find out how easy it is to create my own custom theme. Unfortunately I found the dark and light theme variables hard coded all over the place. :(
hokkos 6y ago
It would be nice to have yearly, monthly, weekly and other recurring graph, I tried to hack it with sql window function, but it gives lot of problems. https://community.grafana.com/t/yearly-monthly-weekly-and-other-recurring-graph/24998 https://community.grafana.com/t/yearly-monthly-weekly-and-ot...
- errantspark 6y ago
  I have this set up to compare my last three days of solar output. I don't remember off the top of my head how I did it but I might be able to help out. What DB are you running on top of? https://imgur.com/a/UXQ4bfY https://imgur.com/a/UXQ4bfY
  - hokkos 6y ago
    I'm using Postgres, I managed to hack something to see yearly data, it works if it look at the full year, but it fails if I the zoom in graphana.
- danlimerick 6y ago
  There is the timeshift function in the query editor but that is per panel and not per query. So you would have to have a panel per year. It should be possible with SQL but it is trickier. TSDB's like Prometheus, Graphite and Influxdb (Flux) have the timeshift function built into their query languages. (https://community.grafana.com/t/advanced-graphing-part2-visualize-timeshift/365/11 https://community.grafana.com/t/advanced-graphing-part2-visu...)
  - danlimerick 6y ago
    We are also working on an expression language to backfill other data sources. You can see the frontend version of this - transformations in 7.0. The backend version which will support functions like timeshift is in alpha for now. Hopefully it will be officially released in the near future.
- deleted 6y ago
  [deleted]
rossmohax 6y ago
Company I am working at has moved from dozens of graphanas across multiple teams to Datadog and I miss it dearly.
- nfadili 6y ago
  What do you find yourself missing?
- PaulWaldman 6y ago
  Why did they move away from Grafana?
  - jesterson 6y ago
    I'd suggest they got a director/manager who was BSed into "fascinating" presentation from Datadog salesmen. A lot of great opensource projects are killed pretty much the same way. Disclaimer: datadog may (or may not) be a good product, it's irrelevant in the case.
    - user5994461 6y ago
      Datadog is a great product that's likely to have been pushed by the developers or the ops team. Grafana has equally "fascinating" presentations for management. Management doesn't take much but a few gauges chart to be impressed.
      - jesterson 6y ago
        Once again, it's not the matter of how great or crappy product is. It's matter of marketing and tenacity of sales team. Try to register your email for Grafana and Datadog. Last one will bombard you with endless pitching emails until you ask them to stop doing it.
- h91wka 6y ago
  Kinda the same situation here, but we managed to keep our dear Grafanas, by telling: "ok, give us budget to migrate all the dashboards". I can confirm that DD marketing is aggressive and full of BS and they will try to sell you missing features, since I attended a DD "workshop" (aka sales presentation) once.
errantspark 6y ago
I have a Grafana dashboard set up for all of my power/hvac/internet infrastructure (I live off grid) and it's an amazing tool. I'm excited to try out some of the new features. Without Grafana I wouldn't have been able to diagnose a charge issue my solar controller has on partially cloudy days. Very grateful to the devs, thank you all for the hard work.
- adamcharnock 6y ago
  I do exactly the same! I love it, it is so useful. I keep my solar grafana dashboard as a pinned tab and it is amazing. I actually developed a python library for reading data from my Outback Mate3[1]. I have ambitions to manage and automate a lot more this way, but I need to improve my electronics skills first. https://github.com/adamcharnock/mate3/ https://github.com/adamcharnock/mate3/
micw 6y ago
I like grafana for graphing but it still lacks essential features for alerting. Most important would be per-series state tracking (https://github.com/grafana/grafana/issues/6041 https://github.com/grafana/grafana/issues/6041) - the issue is open since 2016 and it does not look like it will ever be fixed :-(
- thruhiker 6y ago
  What datasource are you using? I've only used Grafana as a visualizer and use other alerting/monitoring against that datasources. At my current employer, Prometheus is the data source and we use Alert Manager which allows us to write alerts in PromQL which is very flexible.
dkresge 6y ago
"Thank you" doesn't seem to suffice. Where might I donate?
decko 6y ago
I really hope the new plugin architecture does not change for a while. The biggest reason we are still on an older grafana release at work is because we don't have resources to update our plugins to the later versions. And now we have to migrate to yet another new architecture?
nerdbaggy 6y ago
My biggest thing I wish was fixed was being able to use template variables in alerts. https://github.com/grafana/grafana/issues/6557 https://github.com/grafana/grafana/issues/6557 Right now you have to make a chart that has every server on it or a graph per server. I would love to not have to do that
XCSme 6y ago
Congrats! I used Grafana for two projects and I really liked it. I am thinking of providing an easy way to use Grafana as a "custom UI" for by analytics platform, so you could either view the normal charts in the analytics app itself or create your own dashboards in Grafana using the data already collected. I assume this could be easily done my connecting the MySQL database as a datasource and providing the users with some example queries to get started, right?
est 6y ago
Anyone remembers the old grafana? Just few htmls and css/js, upload to static web server and bam! you got a dashboard!
deleted 6y ago
[deleted]
ruizander 6y ago
Congratulations for this great Project! Talking about application metrics (like business transaction timings) are there any plans to release this kind of feature in the roadmap? What about user frontend to backends metrics?

Grafana 7.0

345 points by pjf 6y ago

telaelit 6y ago

I love Grafana so much!! Keep up the great work!!

torkelo 6y ago

Hi, my name is Torkel Ödegaard. Grafana project creator & lead. Here to answer any questions.

bauerd 6y ago
Does the new plugin architecture allow me to provision plugins from a config file on startup? Like installing the Kubernetes App without `grafana-cli plugins install kubernetes-app`
- ryantxu 6y ago
  You can provision plugins on startup with something like: https://grafana.com/docs/grafana/latest/installation/docker/#install-official-and-community-grafana-plugins https://grafana.com/docs/grafana/latest/installation/docker/...
  - mleonhard 6y ago
    Does that method support removing plugins?
- deleted 6y ago
  [deleted]
- ulzeraj 6y ago
  Someone already replied but I should add that most if not all config parameters can be passed to the container through env variables.
thunfischbrot 6y ago
Thank you for this amazing, helpful project! Are there plans for letting power-users style their own custom panels using html, css and js using just the web interface? Meaning without having to develop a plugin.
- danlimerick 6y ago
  This has been possible for ages. The built-in text panel has an HTML option (though you have to disable this setting to allow script tags: https://grafana.com/docs/grafana/latest/installation/configuration/#disable-sanitize-html https://grafana.com/docs/grafana/latest/installation/configu...) There are some external plugins that offer more advanced support: - https://grafana.com/grafana/plugins/aidanmountford-html-panel https://grafana.com/grafana/plugins/aidanmountford-html-pane... - https://grafana.com/grafana/plugins/ryantxu-ajax-panel https://grafana.com/grafana/plugins/ryantxu-ajax-panel
- ryantxu 6y ago
  There are a few plugins that let you enter css/html/js directly -- see https://grafana.com/grafana/plugins/ryantxu-ajax-panel https://grafana.com/grafana/plugins/ryantxu-ajax-panel https://github.com/srclosson/grafana-div-panel https://github.com/srclosson/grafana-div-panel and the text panel (if the server is configured with disable_sanitize_html = true) this is powerful, but long term tends to lead to a maintenance nightmare
iampims 6y ago
Do you know if performance has improved for large dashboards?
- ryantxu 6y ago
  since when? :) A few things have been added that may help: 1. only visible data is queried and loaded 2. If you have multiple panels that use the same data, you can now just reuse the results (or a subset of the results) in a new panel. This will cut down on additional queries that may not be necessary
  - leeoniya 6y ago
    i see you still have plans to continue with uPlot testing :) https://github.com/grafana/grafana/pull/21835 https://github.com/grafana/grafana/pull/21835
    - ryantxu 6y ago
      absolutely -- with everything else for 7.0 that did not make the cut. But would love to work more with your frankly amazing library :)
Systemic33 6y ago
Hi Torkel, First off, Grafana is amazing! So keep up the good work. Question: Currently alerts are fairly limited both in applicable panels and how they export/share relevant data to the chosen notification channel, as well as more advanced conditions. Is there any work going on with improving alerts?
- dig1 6y ago
  Also to mention frequently asked feature: alerts using queries and templated variables [1][2]. Creating even simple generic dashboards with alerts is not possible, unless variables are hardcoded in hidden queries. [1] https://community.grafana.com/t/template-variables-are-not-supported-in-alert-queries-while-setting-up-alert/2514 https://community.grafana.com/t/template-variables-are-not-s... [2] https://github.com/grafana/grafana/issues/6557 https://github.com/grafana/grafana/issues/6557
  - atrandom 6y ago
    yes, this is a huge pain point for us as well
  - hobochili 6y ago
    Agreed. If Grafana alerts supported templated variables I would migrate a decent chunk of my monitoring stack. But without templated variables the alert system feels half baked and I almost prefer it not even be a feature.
kentosi 6y ago
Hi, I used Grafana in my last company and absolutely loved it. How do I convince my current company to get the hell off Datadog?
- mefraimsson 6y ago
  The cost aspect is one thing. Vendor lock-in another. Probably need to have a team that can lead the transition towards Grafana, educate and speak for it. Grafana has an enterprise plugin for datadog that potentially could ease the migration
- stitched2gethr 6y ago
  We just moved everything from datadog to Grafana, Prometheus and Loki, mainly due to cost. Honestly though, I feel like we get more out of the Grafana stack. Its powerful but simple enough to keep it pleasant to work with.
- bsaul 6y ago
  Haha, my team is going the opposite direction: we're leaving prometheus + grafana to datadog, and we're loving it. having every AWS monitoring default dashboard work right out of the bat is a bliss..
- user5994461 6y ago
  How about don't? Datadog offers everything that grafana does plus alerting, as well as more and better integrations (databases, AWS, middleware, etc...).
OldFatCactus 6y ago
have you seen any implementations of internal tracing like APM being used with Grafana? I think lack of a good open source alternative to New Relic or Datadog APM kinda killed OSS monitoring for my company. The Jaeger integration is a good step in the right direction though, kudos!
- akulkarni 6y ago
  What more would you like to see than what Jaeger provides?
  - OldFatCactus 6y ago
    Jaeger is used for traces between distributed systems (external) Traditional APM products are internal traces
    - malkia 6y ago
      AFAIK, You can have internal trace markers, not just between apps. I've used that (for Jaegger) in one of our Windows Application to see the call-graph (it's a level editor for our game), and it was tracking the loading of various resources across multiple threads, then it was visualized with jaeger and even zipkin.
dharmab 6y ago
Thanks for leading the Grafana project! As a color blind user, I find it difficult to read the dashboards created by others. How can I help any efforts to increase Grafana's accessibility?
- VectorLock 6y ago
  I'm also colorblind and a heavy Grafana user but I haven't had much difficulty reading dashboards. What are you having trouble with? The most trouble I've had is colors of multiple lines on a chart being too similar but being able to use the mouse to select a line and see what the line is and its value helps there.
  - dharmab 6y ago
    The default palette is definitely a challenge, since it has pretty low contrast and Grafana's defaults use a lot of thin lines. I especially have trouble matching colors in a legend with the drawn objects in the graph. The best help has been the hover panel, but it's unsorted by default and the highlight contrast on the panel for hovering over a line is pretty dim.
- ryantxu 6y ago
  Upvote: https://github.com/grafana/grafana/issues/18041 https://github.com/grafana/grafana/issues/18041 and add any additional comments and context there. In addition to making the light/dark themes more accessible, I think we should add an additional high contrast one.
imwill 6y ago
Hi Torkel, Just finished watching the GrafanaCon 7.0 Release talk. There are some great Usability improvements and new functionality which will be definitely useful for my upcoming project. Thanks for the great work!
- torkelo 6y ago
  Thanks!
dreamer_soul 6y ago
Amazing work we have been experimenting with grafana at work and the experience is awesome! Thank you for the awesome project
veesahni 6y ago
Are we any closer to having some kind of anomaly detection in grafana (when using prometheus)?
- cranekam 6y ago
  Just curious: what kind of anomaly detection are you looking for? Can you describe the kind of thing you'd want this system to detect?
  - veesahni 6y ago
    We have all sorts of data flowing into Prometheus and charted in Grafana. Grafana lets me alert based on thresholds and that works fine for many things. EXAMPLE 1: Imagine a service. Load avg of 2 is unacceptable and means things will be slowing down. However, a heavy job runs every night and pushes load averages between 1.5 and 2.5 during a 30 minute window.. I wanted to set a load avg alert threshold @ 2.0 but can't since it would fire false alarms too often. So i end up setting it at 3, and end up getting a late alert when things get really bad. Instead, an anomaly detection would look for a number of standard deviations away from a mean, and ideally account for seasonality (i.e. compare the data with same data last few weeks). Now, things will behave better. If loads hit 2 during the day, it'll fire. But during that 30 minute window at night, loads would need to exceed 3 before it fires. EXAMPLE 2: Let's say I'm tracking 30 metrics for a service. I understand the service well enough to set thresholds in 5 metrics. However, I'm sure a failure won't happen alone and the other 25 metrics may show some oddities and give me an early warning of "something wrong happening" .. Here, some kind of loose "anomaly alerts" could help as an early warning. If a certain metric throws too many false alerts, we find a better way to monitor it. EXAMPLE 3: Lets say you have a service that auto scales. There's no limit to how many requests the service handles. So I can't really thresholds on requests/5min. Here, some kind of "anomaly alert" would be useful to let the team know that something is going on that's causing a high level of requests. Again, accounting for seasonality because things tend to change on weekends. At the end of the day, an anomaly alert is not some "magic solution" but just and extra tool in the toolkit to use when it makes sense. Datadog and AWS both have built in easy to use anomaly alerting. Would be nice for grafana/prometheus to have something similar.
elagost 6y ago
At work we have to compile everything from source because compliance, and Grafana is one of the harder ones to get right. There used to be good documentation for it a few versions back but I can't seem to find it lately. Is there any good documentation on how to get Grafana compiled from source?
- danlimerick 6y ago
  Checkout the contribute folder on GitHub. Here is the developer guide: https://github.com/grafana/grafana/blob/master/contribute/developer-guide.md https://github.com/grafana/grafana/blob/master/contribute/de...
- imwill 6y ago
  I just built it from sources because of the same reason. What are the issues on your side?
  - elagost 6y ago
    I've got a complicated proxy setup to contend with and have to build via Jenkins on some older Linux boxes, so it's just trial and error. You have a build script you'd be willing to share?
    - ooOOoo 6y ago
      We are building from source with Docker. Without using Docker, you could try to just run the commands in https://github.com/grafana/grafana/blob/master/Dockerfile https://github.com/grafana/grafana/blob/master/Dockerfile
- mefraimsson 6y ago
  Please let us know and/or open an GitHub issue describing if there are any certain part/step making it hard to build from source. Thanks
heipei 6y ago
Just wanted to say thanks for an awesome piece of software. It's one of those "I can't believe it's doing it and doing it this well, and it's free" kind of projects. For me, Grafana together with ElasticSearch and InfluxDB+Telegraf is pure monitoring/analysis/alerting bliss. Stay true to your roots and don't get tempted to turn into Kibana ;)
qwertox 6y ago
Is it now possible to create a horizontal lined based on a single value (either a fixed or a computed number)? For example when I'm tracking the power grid frequency, I would like to add two lines at 50.05 and 49.95 Hz, as well as one at 50 Hz (with different color). Others have voiced their desire to display a horizontal line at the 95th percentile in a bandwidth usage graph.
- danlimerick 6y ago
  Maybe the thresholds feature in Grafana would work for you? Otherwise depending on the database, they have functions for drawing lines or you can do a hack like this one for Prometheus: https://play.grafana.org/d/h1lglb7ik/threshold-hack?editPanel=2&orgId=1 https://play.grafana.org/d/h1lglb7ik/threshold-hack?editPane...
thruhiker 6y ago
I just wanted to say how much I appreciate your work on Grafana. It's an indispensable tool that I couldn't do my job without!
- torkelo 6y ago
  Thank you!
deleted 6y ago
[deleted]
AustinBGibbons 6y ago
Hey! Thanks for grafana, and especially a HUGE thank you for the user usage metrics! Now if only you could persuade kibana to do the same :-)
em-bee 6y ago
when you post release announcements that possibly get posted snd forwarded to other places, please include a short description about what grafana is. and maybe also add one to your home page. (make it link to the features) when i see any release anouncement about a product i am unfamiliar with, the first question i have is what kind of product it is, so i can decide if i want to check it out
guanzo 6y ago
Zero experience with Grafana here. I have a mongodb database with several collections. All collections share a set of fields: "createdAt", "channelId", "status", etc. How easy would it be to dump all this data into Grafana, and then be able to slice and dice these collections by any combination of the shared fields. Basically, I'm tired of writing mongo queries whenever I want to inspect my data. I want to point and click around a beautiful UI, with data visualizations and easy filtering/aggregation. Is this easy to do with Grafana?
- agacera 6y ago
  I don't believe grafana will help you here. If you have mongodb enterprise you can use Mongo bi connector and connect it with any bi tool. I recommend metabase since it is open source and very good for slice and dice and reports. The mongodb bi connector is a postgres with foreign data wrapper to mongodb. So even if you don't have a mongodb subscription, you may be able to pull this off using the OS version https://github.com/EnterpriseDB/mongo_fdw https://github.com/EnterpriseDB/mongo_fdw
- redis_mlc 6y ago
  Grafana is a UI, not a database (it doesn't store any data.) Plugins are used for showing pie charts, etc.
- viraptor 6y ago
  I've never tried this one, but the mongo datasource plugin exists: https://github.com/JamesOsgood/mongodb-grafana https://github.com/JamesOsgood/mongodb-grafana Maybe you can use it directly this way? (if you don't mind the BI-equivalent hammering your primary data source)
btmiller 6y ago
Hi Torkel, this issue has been open since 2016[1]. There is incredible demand for it yet complete silence from Grafana on committing to the effort. Can I get official word on this please? [1] https://github.com/grafana/grafana/issues/6983 https://github.com/grafana/grafana/issues/6983
- gempir 6y ago
  Have there been no efforts to add this with a PR? I mean Grafana is open source and if the project leads wants to focus on other things that's his choice.
  - btmiller 6y ago
    Attempts were made: https://github.com/grafana/grafana/pull/14075 https://github.com/grafana/grafana/pull/14075
- torkelo 6y ago
  The reason is we want to move alert rules out of the dashboard. To be their own thing, have their own UI & workflow for editing and updating. Once that is done, we plan to add the ability to link an alert rule to 1 or more dashboard panels.
morb 6y ago
Will Grafana ever add direct support for rrd datasources?
- wazoox 6y ago
  Does this help? https://github.com/doublemarket/grafana-rrd-server https://github.com/doublemarket/grafana-rrd-server
  - morb 6y ago
    Kinda, but not really. It has no support for rrdcached flush, AVERAGE CF is hardcoded , and I cannot seem to get it to not connect lines through gaps where there were no measurements. Example not related to rrd, but same problem different db: https://github.com/grafana/grafana/issues/2125 https://github.com/grafana/grafana/issues/2125.
mleonhard 6y ago
How about adding support for configuring Grafana with files? This would allow us to deploy immutable Grafana instances using infrastructure-as-code tools like Terraform and Docker. Specifically, I want a way to eliminate post-install steps such as adding users and organizations and setting user passwords.
- dharmab 6y ago
  Wouldn't a better way to configure users and passwords be to use an external auth provider such as OAuth or LDAP? If you don't already have a auth provider, you could deploy Keycloak as your user DB.
  - mleonhard 6y ago
    OAuth/LDAP are great for large organizations. Small organizations don't need such complexity. Even if you use OAuth/LDAP for accounts and passwords, Grafana still requires manual steps to create the organization, install plugins, and set the admin password. I wrote a Python script to do this on Docker container startup. That script is a workaround for Grafana server's poor ops design. It's extra code that we must maintain.
    - dharmab 6y ago
      The admin password can be set with an environment variable. That just leaves the org and plugins.
mleonhard 6y ago
Do you plan to make debug logging useful? I had difficulty troubleshooting Grafana & InfluxDB auth problems and found that debug logging did not provide any extra detail.
mleonhard 6y ago
Thanks for adding more support for logs in Grafana. I'm not using AWS CloudWatch Logs because of the financial risk of a runaway server running up a huge AWS bill. I had difficulty getting log display and log search working well in Grafana 6.6.2 and InfluxDB. Do you plan to improve log support for other databases?
hashhar 6y ago
Thanks a lot for creating it and pushing it forward. It's my go-to tool for monitoring dashboards and also some simple interactive data dashboards. Never had any issues with it (and the ones I had were promptly fixed by the devs and the community).

deleted 6y ago

[deleted]

nepotism2018 6y ago

Awesome tool. Supports all the datasources I need.

imwill 6y ago

Custom Themes seems to be one of the most wanted features of the Community. Is there any plan implementing it in the near future? Recently I digged into the source code to find out how easy it is to create my own custom theme. Unfortunately I found the dark and light theme variables hard coded all over the place. :(

hokkos 6y ago

It would be nice to have yearly, monthly, weekly and other recurring graph, I tried to hack it with sql window function, but it gives lot of problems. https://community.grafana.com/t/yearly-monthly-weekly-and-other-recurring-graph/24998 https://community.grafana.com/t/yearly-monthly-weekly-and-ot...

errantspark 6y ago
I have this set up to compare my last three days of solar output. I don't remember off the top of my head how I did it but I might be able to help out. What DB are you running on top of? https://imgur.com/a/UXQ4bfY https://imgur.com/a/UXQ4bfY
- hokkos 6y ago
  I'm using Postgres, I managed to hack something to see yearly data, it works if it look at the full year, but it fails if I the zoom in graphana.
danlimerick 6y ago
There is the timeshift function in the query editor but that is per panel and not per query. So you would have to have a panel per year. It should be possible with SQL but it is trickier. TSDB's like Prometheus, Graphite and Influxdb (Flux) have the timeshift function built into their query languages. (https://community.grafana.com/t/advanced-graphing-part2-visualize-timeshift/365/11 https://community.grafana.com/t/advanced-graphing-part2-visu...)
- danlimerick 6y ago
  We are also working on an expression language to backfill other data sources. You can see the frontend version of this - transformations in 7.0. The backend version which will support functions like timeshift is in alpha for now. Hopefully it will be officially released in the near future.
deleted 6y ago
[deleted]

rossmohax 6y ago

Company I am working at has moved from dozens of graphanas across multiple teams to Datadog and I miss it dearly.

nfadili 6y ago
What do you find yourself missing?
PaulWaldman 6y ago
Why did they move away from Grafana?
- jesterson 6y ago
  I'd suggest they got a director/manager who was BSed into "fascinating" presentation from Datadog salesmen. A lot of great opensource projects are killed pretty much the same way. Disclaimer: datadog may (or may not) be a good product, it's irrelevant in the case.
  - user5994461 6y ago
    Datadog is a great product that's likely to have been pushed by the developers or the ops team. Grafana has equally "fascinating" presentations for management. Management doesn't take much but a few gauges chart to be impressed.
    - jesterson 6y ago
      Once again, it's not the matter of how great or crappy product is. It's matter of marketing and tenacity of sales team. Try to register your email for Grafana and Datadog. Last one will bombard you with endless pitching emails until you ask them to stop doing it.
h91wka 6y ago
Kinda the same situation here, but we managed to keep our dear Grafanas, by telling: "ok, give us budget to migrate all the dashboards". I can confirm that DD marketing is aggressive and full of BS and they will try to sell you missing features, since I attended a DD "workshop" (aka sales presentation) once.

errantspark 6y ago

I have a Grafana dashboard set up for all of my power/hvac/internet infrastructure (I live off grid) and it's an amazing tool. I'm excited to try out some of the new features. Without Grafana I wouldn't have been able to diagnose a charge issue my solar controller has on partially cloudy days. Very grateful to the devs, thank you all for the hard work.

adamcharnock 6y ago
I do exactly the same! I love it, it is so useful. I keep my solar grafana dashboard as a pinned tab and it is amazing. I actually developed a python library for reading data from my Outback Mate3[1]. I have ambitions to manage and automate a lot more this way, but I need to improve my electronics skills first. https://github.com/adamcharnock/mate3/ https://github.com/adamcharnock/mate3/

micw 6y ago

I like grafana for graphing but it still lacks essential features for alerting. Most important would be per-series state tracking (https://github.com/grafana/grafana/issues/6041 https://github.com/grafana/grafana/issues/6041) - the issue is open since 2016 and it does not look like it will ever be fixed :-(

thruhiker 6y ago
What datasource are you using? I've only used Grafana as a visualizer and use other alerting/monitoring against that datasources. At my current employer, Prometheus is the data source and we use Alert Manager which allows us to write alerts in PromQL which is very flexible.

dkresge 6y ago

"Thank you" doesn't seem to suffice. Where might I donate?

decko 6y ago

I really hope the new plugin architecture does not change for a while. The biggest reason we are still on an older grafana release at work is because we don't have resources to update our plugins to the later versions. And now we have to migrate to yet another new architecture?

nerdbaggy 6y ago

My biggest thing I wish was fixed was being able to use template variables in alerts. https://github.com/grafana/grafana/issues/6557 https://github.com/grafana/grafana/issues/6557 Right now you have to make a chart that has every server on it or a graph per server. I would love to not have to do that

XCSme 6y ago

Congrats! I used Grafana for two projects and I really liked it. I am thinking of providing an easy way to use Grafana as a "custom UI" for by analytics platform, so you could either view the normal charts in the analytics app itself or create your own dashboards in Grafana using the data already collected. I assume this could be easily done my connecting the MySQL database as a datasource and providing the users with some example queries to get started, right?

est 6y ago

Anyone remembers the old grafana? Just few htmls and css/js, upload to static web server and bam! you got a dashboard!

deleted 6y ago

[deleted]

ruizander 6y ago

Congratulations for this great Project! Talking about application metrics (like business transaction timings) are there any plans to release this kind of feature in the roadmap? What about user frontend to backends metrics?