17 ms·

Are Prometheus and Thanos developers against fair competition?

• valyala 7y ago
  An additional case of unfair competition was at PromCon 2019. There was a talk named `Remote storage wars` from Adidas at the conference [1]. Various remote storage solutions for Prometheus were compared in the talk for use cases specific to Adidas - Cortex, M3DB, VictoriaMetrics and Thanos receiver. VictoriaMetrics showed the best performance, the lowest RAM usage and the lowest disk space usage. The was a video recording, but unfortunately slides were broken during the talk [2]. I was hoping that slides will be published in the next months after the conference. Unfortunately slides for this talk weren't published yet, while slides from other talks are already published - see `slides` link at the bottom of every talk page from [3]. This doesn't look very fair :) [1] https://promcon.io/2019-munich/talks/remote-write-storage-wars/ https://promcon.io/2019-munich/talks/remote-write-storage-wa... [2] https://youtu.be/4vZ1PqZ_Foc?t=16025 https://youtu.be/4vZ1PqZ_Foc?t=16025 [3] https://promcon.io/2019-munich/schedule/ https://promcon.io/2019-munich/schedule/
  • SuperQue 7y ago
    The videos will be uploaded soon. Don't worry, none of the slides or other content was lost. We're just waiting for our video company to finish the final renders with all the correct color and lighting quality adjustments made. Each talk will be individually published on YouTube just like all previous PromCon events.
    • valyala 7y ago
      Great news! I hope this isn't a sarcasm :)
• valyala 7y ago
  Another case of unfair competition is the article [1] published by Prometheus author, where he claims VictoriaMetrics doesn't compress random data well. I hope Prometheus author knows information theory [2] and was aware that random data cannot be compressed. Then it is unclear why he decided to publish the comparison :) I published response to this article [3], but unfortunately it isn't referenced from the original article spreading the FUD. [1] https://www.robustperception.io/evaluating-performance-and-correctness https://www.robustperception.io/evaluating-performance-and-c... [2] https://en.wikipedia.org/wiki/Information_theory#Entropy_of_an_information_source https://en.wikipedia.org/wiki/Information_theory#Entropy_of_... [3] https://medium.com/@valyala/evaluating-performance-and-correctness-victoriametrics-response-e27315627e87 https://medium.com/@valyala/evaluating-performance-and-corre...
• FrankiFrank 7y ago
  Sometimes when I read your comments at Prometheus issues or at twitter, it sounds like you want to put prometheus into bad light. It often sounds like you're subliminally insulting prometheus.
  • valyala 7y ago
    I'm trying hard to write polite comments, which could help users finding the solution for their issues. My comments could be harsh only in response to misleading posts about VictoriaMetrics [1]. [1] https://www.robustperception.io/evaluating-performance-and-correctness https://www.robustperception.io/evaluating-performance-and-c...
    • deleted 7y ago
      [deleted]
  • nanopeni 7y ago
    Would you mind to share few particular cases? I'm wondering if that's a matter of your personal perception or something more general.
• valyala 7y ago
  Yet another case of "fair" competition from Prometheus and Thanos devs [1]: * Prometheus users proposed extending `/api/v1/label/.../values` API [2], which has been already implemented in VictoriaMetrics. * I mentioned about this and described high-level architecture for this feature, so it can be implemented by Prometheus devs. * They say "you cannot point Prometheus users to the solution in VictoriaMetrics, since this is self-promotion" [1]. The same issue already contains Thanos promotion in the first comments, but nobody complains about this. [1] https://github.com/prometheus/prometheus/issues/6178#issuecomment-565694157 https://github.com/prometheus/prometheus/issues/6178#issueco... [2] https://github.com/prometheus/prometheus/issues/6178 https://github.com/prometheus/prometheus/issues/6178
• jrv 7y ago
  You continue to claim that VictoriaMetrics is backwards-compatible with Prometheus and only include a side-note that it actually is not in all cases. Such knowingly misleading statements are dangerous for users and the ecosystem. As for the more general point, my comment at https://news.ycombinator.com/item?id=21772791 https://news.ycombinator.com/item?id=21772791 stands. It's not about competition (we welcome and love that from all sides), it's about overly marketing a product on Prometheus OSS community channels, while at the same time not being clear about or ignoring compatibility issues that will for sure cause problems for the ecosystem. EDIT: One more point: The people seeing a problem around VM behavior here are not just Prometheus team members. Multiple external community members / users have pointed this out as well to us.
  • valyala 7y ago
    > You continue to claim that VictoriaMetrics is backwards-compatible with Prometheus and only include a side-note that it actually is not in all cases * There are a few controversial cases where VictoriaMetrics' implementation for PromQL intentionally diverges from the original implementation in Prometheus. This discrepancy is needed because VictoriaMetrics tries hard to make PromQL more user-friendly at edge cases, which aren't solved for a long time on Prometheus side [1]. * Of course, there may be non-intentional discrepancies in PromQL implementation for VictoriaMetrics, because it is implemented from scratch without using Prometheus data model and source code. Such discrepancies are usually fixed as soon as users report about them. > It's not about competition (we welcome and love that from all sides) I see. Facts mentioned in this topic shows the real love :) [1] https://github.com/prometheus/prometheus/issues/3746 https://github.com/prometheus/prometheus/issues/3746
    • deleted 7y ago
      [deleted]
      • deleted 7y ago
        [deleted]
  • shitloadofbooks 7y ago
• hagen1778 7y ago
  This is my response to ongoing discussion at https://github.com/prometheus/prometheus/issues/6178 https://github.com/prometheus/prometheus/issues/6178 because github issues is truly the wrong place for doing this. >> @hagen1778 My comment at #5706 (comment) goes into this a bit more. It can indeed be helpful to cross-link to other products/projects at times, and I don't see this individual comment as a big problem - but it fits into an overall pattern that is less ok (see details in my linked comment). @juliusv I am familiar with the comment you linked. I am also sharing your opinion about being `a bit more sensitive about using Prometheus channels`. However, I find it a bit unfair comparing to other OSS mentions in Prometheus issues. Anyone can verify this easily by searching for `thanos`, `cortex` or `victoriametrics` on https://github.com/prometheus/prometheus/issues https://github.com/prometheus/prometheus/issues I disagree with second point about `own implementation of PromQL`. The reason why it was re-implemented(AFAIK) with so much efforts instead of just reusing the existing Prometheus code is the unwillingness to extend existing implementation. I'm pretty sure you are familiar with that, because this discussion appeared in your own proposal ticket from Jun 25. And I understand this, authors should protect their projects from being overwhelmed by other people proposals and needs to keep it solid. But I don't understand why it is not allowed to mention the alternative way for users on public platforms like github. I am very grateful for Prometheus and the ecosystem around it and strongly believe it becomes a standard for the industry, so we all should work on it. That's why such confrontations make me sad. I hope that such situations will never occur in future and opensource will remain the place for creating cool software.
  • roidelapluie 7y ago
    To be fair, members of Thanos and Cortex projects do contribute code to Prometheus, which might explain why they are named more often here.
    • valyala 7y ago
      To be fair, VictoriaMetrics also contributes to Prometheus ecosystem by providing open source cost-effective remote storage for Prometheus, which is easy to setup and operate. Additional contributions to Prometheus ecosystem include: * Filing issues and feature requests for Prometheus [1], [2], [3]. * Publishing introduction materials for Prometheus users [4], [5]. * Helping Prometheus users at prometheus-users Google group [6]. [1] https://github.com/prometheus/prometheus/issues/4456 https://github.com/prometheus/prometheus/issues/4456 [2] https://github.com/prometheus/prometheus/issues/4769 https://github.com/prometheus/prometheus/issues/4769 [3] https://github.com/prometheus/prometheus/issues/5166 https://github.com/prometheus/prometheus/issues/5166 [4] https://medium.com/@valyala/promql-tutorial-for-beginners-9ab455142085 https://medium.com/@valyala/promql-tutorial-for-beginners-9a... [5] https://medium.com/@valyala/prometheus-storage-technical-terms-for-humans-4ab4de6c3d48 https://medium.com/@valyala/prometheus-storage-technical-ter... [6] https://groups.google.com/forum/#!searchin/prometheus-users/valyala%7Csort:date https://groups.google.com/forum/#!searchin/prometheus-users/...
• shitloadofbooks 7y ago
  I took offence with @valyala posting about Victoria Metrics in the Cortex Slack channel in the past, but I'm so glad he did because we had _endless_ issues with Cortex and we found Victoria Metrics to be _incredibly_ fast and reliable after trying it when he suggested that it could do something Cortex couldn't. This github response wasn't cross promotion though -- this was @vayala explaining how it could be implemented, as someone who has done it. The fact that no one complained when Thanos was mentioned, but complained when VM was mentioned shows how hypocritical some Prometheus contributors are. Honestly, @valyala should just create a metrics collector and then I wouldn't have to deal with the "we know better" attitude from some Prometheus contributors. The Victoria Metrics contributors are absolutely amazing. Every issue or suggestion has been implemented within days (sometimes hours) and every question answered in incredible detail, with warmth and friendliness.
  • valyala 7y ago
    Thanks for the warm response! > Honestly, @valyala should just create a metrics collector and then I wouldn't have to deal with the "we know better" attitude from some Prometheus contributors We didn't want going this route, since this could hurt Prometheus community in the long run. But we constantly keep hearing this feature request from VictoriaMetrics users, who deal with high ingestion rates. Probably such a collector will be implemented in the future unless Prometheus devs address the following issues in Prometheus: * Increased RAM and CPU usage when enabling remote_write. * Increased disk space usage on high ingestion rate. * High network bandwidth usage for pushing data to remote storage.