10 ms·

This makes me think: an open-source project can be better off if it's not controlled by any one company. While in Prometheus (http://prometheus.io http://promet

by jrv 11y ago

This makes me think: an open-source project can be better off if it's not controlled by any one company. While in Prometheus (http://prometheus.io http://prometheus.io), we might still take a while before we have a clustered remote long-term storage, we'd never prevent it because the project is independent of any company and we'd want the open-source project to be as good as it can be. Also, there were some tentative thoughts about using InfluxDB as the main long-term storage backend for Prometheus, but that has become pretty much uninteresting now that clustering support (needed for LTS and durability) is basically cancelled for the open-source version. Still, I guess I can understand that when you're a company, you need to focus on making money.

mindcrime 11y ago
This is concerning to me, not because I use InfluxDB, but because my company is also a "pure play" OSS vendor. So it's always concerning to see a company fail to achieve success with the pure-play model, and wind up having to go "open core". Also, there were some tentative thoughts about using InfluxDB as the main long-term storage backend for Prometheus, but that has become pretty much uninteresting now that clustering support (needed for LTS and durability) is basically cancelled for the open-source version. True, but consider this: Given that InfluxDB (core) is Open Source, there's nothing to stop somebody else from coming along and building a version with clustering support. Whether that will happen or not is, of course, an open question.
- jrv 11y ago
  True, but a fork is a fork is a fork. It would have to be done very competently (clustering is hard) and maintained long-term, by people who are not even part of InfluxData and thus have a big starting disadvantage in experience and user base. That makes it much less viable, unless there's a huge critical momentum around it.
- bbrazil 11y ago
  There's nothing to stop somebody else from coming along and building a version with clustering support. Whether that will happen or not is, of course, an open question. Clustering is really really hard. My personal experience with that class of problem is that it took 3 years to get it production ready, and that was for a simpler problem than what needs to be solved here. The chances of a 3rd party implementing this given that InfluxData plan to have a commercial version seem very low. I'd expect that most companies who want to stay open source will treat this as a standard database that needs client-side sharding and HA (which is the recommend approach - see https://influxdata.com/high-availability/ https://influxdata.com/high-availability/).
bogomipz 11y ago
+1 to Prometheus. I'm a new user but have found the project to be well thought out and have found the community on IRC channel to incredibly helpful and responsive to the community. I'm glad you didn't hitch your wagon to Influx. Is there a reason to not continue with your LevelDB backend long term?
- jrv 11y ago
  Thanks! We only use LevelDB for Prometheus's time series indexes, while using a custom-built storage engine for the actual series data. But Prometheus's local storage has always been built as single-node storage without clustering - very much on purpose actually, because we want monitoring and alerting to be decoupled and autonomous enough to not have any hard distributed storage dependencies (as that has a high risk of breaking down when shit hits the fan). We definitely want to support full remote, distributed, and durable storage for Prometheus, but we still want local nodes to have enough autonomous storage for alerting and short-term inspection of data. Whether some of the technology of the local storage engine will be used for building a clustered storage remains to be seen, but it's likely.

by jrv 11y ago

mindcrime 11y ago

This is concerning to me, not because I use InfluxDB, but because my company is also a "pure play" OSS vendor. So it's always concerning to see a company fail to achieve success with the pure-play model, and wind up having to go "open core". Also, there were some tentative thoughts about using InfluxDB as the main long-term storage backend for Prometheus, but that has become pretty much uninteresting now that clustering support (needed for LTS and durability) is basically cancelled for the open-source version. True, but consider this: Given that InfluxDB (core) is Open Source, there's nothing to stop somebody else from coming along and building a version with clustering support. Whether that will happen or not is, of course, an open question.

jrv 11y ago
True, but a fork is a fork is a fork. It would have to be done very competently (clustering is hard) and maintained long-term, by people who are not even part of InfluxData and thus have a big starting disadvantage in experience and user base. That makes it much less viable, unless there's a huge critical momentum around it.
bbrazil 11y ago
There's nothing to stop somebody else from coming along and building a version with clustering support. Whether that will happen or not is, of course, an open question. Clustering is really really hard. My personal experience with that class of problem is that it took 3 years to get it production ready, and that was for a simpler problem than what needs to be solved here. The chances of a 3rd party implementing this given that InfluxData plan to have a commercial version seem very low. I'd expect that most companies who want to stay open source will treat this as a standard database that needs client-side sharding and HA (which is the recommend approach - see https://influxdata.com/high-availability/ https://influxdata.com/high-availability/).

bogomipz 11y ago

+1 to Prometheus. I'm a new user but have found the project to be well thought out and have found the community on IRC channel to incredibly helpful and responsive to the community. I'm glad you didn't hitch your wagon to Influx. Is there a reason to not continue with your LevelDB backend long term?

jrv 11y ago
Thanks! We only use LevelDB for Prometheus's time series indexes, while using a custom-built storage engine for the actual series data. But Prometheus's local storage has always been built as single-node storage without clustering - very much on purpose actually, because we want monitoring and alerting to be decoupled and autonomous enough to not have any hard distributed storage dependencies (as that has a high risk of breaking down when shit hits the fan). We definitely want to support full remote, distributed, and durable storage for Prometheus, but we still want local nodes to have enough autonomous storage for alerting and short-term inspection of data. Whether some of the technology of the local storage engine will be used for building a clustered storage remains to be seen, but it's likely.