5 ms·

How does this better than grafana? Loki -> logs Tempo -> traces Prometheus/mimir -> metrics I know personally of several companies which use this stack with te

How does this better than grafana? Loki -> logs Tempo -> traces Prometheus/mimir -> metrics I know personally of several companies which use this stack with tens of thousands of nodes. Everything runs on object storage which means it is easy to scale up, and you can move between storage providers as long as they implement the S3 API. The grafana ecosystem is very sticky as well. If I download some random helm chart it most likely will come with a grafana dashboard which I can easily import and instantly have dashboards and alerts for the helm chart.

__turbobrew__ 2y ago
I see you address this in a blog post: “You will also hear of LGTM stack (Loki, Grafana, Tempo, Mimir) which is a pretty good stack for observability. Each of these components are separate open source tools built by grafana labs.” Not very convincing why I wouldn’t go for the LGTM stack which has been proven to be effective.
- prabhatsharma 2y ago
  By all means, if LGTM works for you stay with it. For those looking at much more simplicity, and much higher performance OpenObserve is the way to go. Many folks have moved from Loki to OpenObserve due to performance issues with Loki. Many have moved from LGTM stack completely to OpenObserve. Many have chosen to use Grafana as a front end for OpenObserve too. Take a look at how easy it can be to build dashboards in OpenObserve. It takes time for community and ecosystem to build for great products. Grafana started in 2014. OpenObserve started in 2022.
  - __turbobrew__ 2y ago
    What would convince me is if you show benchmarks between large LTGM deployments and OpenObserve. Show me OpenObserve ingesting a few billion metrics streams and show me how query times are faster than mimir.
    - prabhatsharma 2y ago
      In all likelihood you are not going to get convinced by that. You did not switch to using LGTM stack because grafana gave you a benchmark of LGTM against what you were using previously. We run benchmarks internally and will publish them once we are ready for it, but are unlikely to benchmark someone else's product. Benchmarking someone else's product always leads to a conversation similar to - You ran your benchmark in the most optimized way but used our non optimized settings or a version of that. People who use OpenObserve like it for it's ease of setup, ease of management, high performance and rich feature set. Especially for logs, grafana and loki is no match in terms of features and performance when it comes to OpenObserve. I will let you test it if you are curious and have some spare time. OpenObserve is used by people ingesting MBs of data in their basement to PBs of data in large clusters in AWS, Azure, GCP and other cloud environments. BTW, here is a story of a large EV company who moved to OpenObserve for traces and increased performance by a factor of 10x and reduced their cost at the same time - https://openobserve.ai/blog/jidu-journey-to-100-tracing-fidelity https://openobserve.ai/blog/jidu-journey-to-100-tracing-fide...
      - __turbobrew__ 2y ago
        Grafana has always been the lingua-franca of telemetry so I think if you want real traction with people like me you are going to need to publish a publicly verifiable benchmark than OpenObserve can ingest a billion metrics streams, which mimir could do several years ago: https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-prometheus-tsdb-grafana-mimir-to-1-billion-active-series/ https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-pr... Like I said, the industry is small and I know several colleagues running LGTM at large scales (10k-100k nodes) so I know the system works and it is a safe bet.

__turbobrew__ 2y ago

I see you address this in a blog post: “You will also hear of LGTM stack (Loki, Grafana, Tempo, Mimir) which is a pretty good stack for observability. Each of these components are separate open source tools built by grafana labs.” Not very convincing why I wouldn’t go for the LGTM stack which has been proven to be effective.

prabhatsharma 2y ago
By all means, if LGTM works for you stay with it. For those looking at much more simplicity, and much higher performance OpenObserve is the way to go. Many folks have moved from Loki to OpenObserve due to performance issues with Loki. Many have moved from LGTM stack completely to OpenObserve. Many have chosen to use Grafana as a front end for OpenObserve too. Take a look at how easy it can be to build dashboards in OpenObserve. It takes time for community and ecosystem to build for great products. Grafana started in 2014. OpenObserve started in 2022.
- __turbobrew__ 2y ago
  What would convince me is if you show benchmarks between large LTGM deployments and OpenObserve. Show me OpenObserve ingesting a few billion metrics streams and show me how query times are faster than mimir.
  - prabhatsharma 2y ago
    In all likelihood you are not going to get convinced by that. You did not switch to using LGTM stack because grafana gave you a benchmark of LGTM against what you were using previously. We run benchmarks internally and will publish them once we are ready for it, but are unlikely to benchmark someone else's product. Benchmarking someone else's product always leads to a conversation similar to - You ran your benchmark in the most optimized way but used our non optimized settings or a version of that. People who use OpenObserve like it for it's ease of setup, ease of management, high performance and rich feature set. Especially for logs, grafana and loki is no match in terms of features and performance when it comes to OpenObserve. I will let you test it if you are curious and have some spare time. OpenObserve is used by people ingesting MBs of data in their basement to PBs of data in large clusters in AWS, Azure, GCP and other cloud environments. BTW, here is a story of a large EV company who moved to OpenObserve for traces and increased performance by a factor of 10x and reduced their cost at the same time - https://openobserve.ai/blog/jidu-journey-to-100-tracing-fidelity https://openobserve.ai/blog/jidu-journey-to-100-tracing-fide...
    - __turbobrew__ 2y ago
      Grafana has always been the lingua-franca of telemetry so I think if you want real traction with people like me you are going to need to publish a publicly verifiable benchmark than OpenObserve can ingest a billion metrics streams, which mimir could do several years ago: https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-prometheus-tsdb-grafana-mimir-to-1-billion-active-series/ https://grafana.com/blog/2022/04/08/how-we-scaled-our-new-pr... Like I said, the industry is small and I know several colleagues running LGTM at large scales (10k-100k nodes) so I know the system works and it is a safe bet.