9 ms·

InfluxDB 1.0 GA Released: A Retrospective and What’s Next

136 points by pauldix 10y ago

dozzie 10y ago
Packages built in the same shitty way as always. I expected nothing less from you guys.
- pauldix 10y ago
  We're happy to take PRs. Seems everyone has opinions on packaging and you know what it's like when everyone has opinions, right? :)
  - dozzie 10y ago
    > Seems everyone has opinions on packaging and you know what it's like when everyone has opinions, right? :) Don't try to push this into opinion realm, because it's not, it's a standards realm. Your package doesn't stand lintian test, and it's not a doesn't-really-matter type of errors that there's no README or license file. You don't even put an initscript in packages in a proper way. It wouldn't matter that much, but your build system doesn't have an easy, clean build from local source tree with a method of putting files in right places in $DESTDIR, so the software could be properly debianized (or equivalent) by somebody else.
    - kfitzpatrick 10y ago
      dozzie, do you have a PR or issue open where you could help guide us to a good resolution?
      - dozzie 10y ago
        No. Why would I do so if I don't use your software, given that your product feels more like a paid thing than open source? You have enough money to hire competent sysadmin to teach you how to package software.
- gyre007 10y ago
  Well, if that's the only complaint you have, then I think Influx have done a stellar job :-)
  - dozzie 10y ago
    It's the only complaint because I didn't bother looking further. It's not a sign of a good engineering when the build system and packaging are mess.
- gtirloni 10y ago
  At least the RPM one seems okay. You should elaborate.
- pauldix 10y ago
  Or maybe you were sad because you only saw package downloads from S3 and you were looking for an actual Debian repository? Well, that actually already exists. See: https://docs.influxdata.com/influxdb/v1.0/introduction/installation/#ubuntu-debian https://docs.influxdata.com/influxdb/v1.0/introduction/insta...
  - dozzie 10y ago
    I'm glad you answered your own question, but it was you who asked that. I couldn't care less if they provided APT repository; using random repositories from various companies is asking for trouble with managing your servers (hint: package retention policies).
    - pauldix 10y ago
      I answered my own question because your original comment left no clue as to why you're so upset. Neither does this one that I'm now responding to.
      - linsomniac 10y ago
        Agreed, them providing .debs is far from "packaging being a mess". I'm quite happy that they provided .debs, worked with them to make some changes to the debs via pull-requests, and after that have been super happy with the packaging. I grab and import their packages into my own private repo for historic reasons, but it does mean I have to chase upstream changes. Personally, I think this is a straw-man argument. With many places not even providing packages, I'm glad that the Influx folks went through that process for me.
        dozzie 10y ago
        > Agreed, them providing .debs is far from "packaging being a mess". I'm quite happy that they provided .debs, worked with them to make some changes to the debs via pull-requests, and after that have been super happy with the packaging. Clearly you haven't checked how they build their DEBs and RPMs. Some opaque, overcomplicated script that eventually calls fpm instead of proper debianization or spec file for RPM. This results, among the others, in some configs not being marked as configs. > I grab and import their packages into my own private repo for historic reasons, but it does mean I have to chase upstream changes. Personally, I think this is a straw-man argument. It's not. You can't rebuild a server if you suddenly don't have access to packages this server has installed, especially if you need them to be in specific versions. BTDT, several times.
otoolep 10y ago
Great stuff. Congrats to everyone on the InfluxDB team on this big milestone.
- pauldix 10y ago
  Thanks, Philip! You helped us get here :)
hamax 10y ago
A very happy InfluxDB user here, but did you really had to title your release email "InfluxDB 1.0 GA is Here! Plus 27x Faster than MongoDB Benchmarks"?
joekrie 10y ago
Cool to see a database specializing in time series data.
LogicX 10y ago
We've been using influx since 0.9 in production. Had a few bumps with cardinality growing out of control, but now working around those limits, it's going well. Looking forward to that being something tackled with upcoming releases.
- pauldix 10y ago
  Yep, we're very focused on solving the cardinality problem now. See these for some details about how we're thinking about it: https://github.com/influxdata/influxdb/pull/7175 https://github.com/influxdata/influxdb/pull/7175 https://github.com/influxdata/influxdb/pull/7174 https://github.com/influxdata/influxdb/pull/7174 https://github.com/influxdata/influxdb/pull/7186 https://github.com/influxdata/influxdb/pull/7186 https://github.com/influxdata/influxdb/pull/7264 https://github.com/influxdata/influxdb/pull/7264
linsomniac 10y ago
I've been using InfluxDB for almost a year now. At one point around 9 months ago I had given up on it because it was a bit crashy, and the database just was growing too fast. But the promise behind it was too compelling and I started experimenting with newer versions around 6 months ago and it has been just great! Much easier to deal with than Graphite/collectd/carbon, telegraf has not been eating our servers like collectd was, CPU usage is way down... Loving InfluxDB. Still need to implement annotations and SNMP polling in telegraf, but it is awesome. We are even pushing some application stats into it.
- pauldix 10y ago
  We updated the SNMP plugin a few weeks ago. Was thanks to a contributor who has been super helpful. Does the updated one do what you need?
  - imaginenore 10y ago
    Please pay the guy. He is helping your product.
    - rrhyne 10y ago
      So, super obvious, but thought I'd just mention that influxdb is open source. It's likely that free version helped the contributor with his issue so he contributed back.
      - imaginenore 10y ago
        You don't think the enterprise users can use his work?
        tinco 10y ago
        Even if he contributed, I think it's likely that InfluxDB helped they contributor more than he helped InfluxDB.
        10y ago
scrollaway 10y ago
Influx is absolutely amazing. We're using it along with grafana to store and display our desktop and web apps' analytics (it completely replaced GA), store and display HTTP health analytics (piping custom uwsgi request logger into UDP input), and do continuous analysis of Hearthstone games. It's incredibly fast and the grafana/influx/telegraf stack is really cool to play with. Highly recommended.
- chrissnell 10y ago
  Want to hear more about your HTTP analytics. Particularly interested in collecting selenium/sitespeed.io-style page timing metrics. There are a million commercial solutions for this but so few open source options.
  - scrollaway 10y ago
    Glad you asked! I'm pretty proud of this hack :) uWSGI has a very flexible logging system. You can create a log with just requests, and completely customize the format of the line. On top of that, uWSGI supports creating multiple log targets, and supports logging directly into a UDP socket. If you make it match InfluxDB's line format (https://docs.influxdata.com/influxdb/v1.0/write_protocols/line/ https://docs.influxdata.com/influxdb/v1.0/write_protocols/li...), and you set up InfluxDB to accept connections from a matching UDP socket, then uWSGI can essentially log every request directly into influx. Like this for example: logformat = uwsgirequest,host="%(host)",status=%(status) msecs=%(msecs),size=%(size) %(epoch)000000000 logger = file:/var/logs/uwsgi/uwsgi.log req-logger = file:/var/logs/uwsgi/requests.log req-logger = socket:127.0.0.1:8083 You can also add 'path="%(var.PATH_INFO)"' to that but there is a potential data injection vulnerability if you don't re-parse the output. With that, you can create nice graphs with Grafana to analyze your loading times, error rates, page sizes etc: https://i.imgur.com/fzbXgUd.png https://i.imgur.com/fzbXgUd.png You can also set up Kapacitor to identify when your page load times are abnormally increasing. We had the idea after a database connection leak increased our average loading times by 300%. Do note that this doesn't batch inputs, so 1 request = 1 connection to influx. But it's pretty easy to put a middleware agent in between those to do validation and batching. The nice thing about all this is it's application level timing metrics. There's not a lot of ways to get that, except with application middleware which can't always catch everything you want.
fabian2k 10y ago
I played around with InfluxDB, Telegraf and Grafana a while ago, and it worked very nicely for the basic stuff I tried. One thing in Telegraf where I didn't figure out a good solution was a way to parse arbitrary log files and generate data points and/or annotations from them. There is a particularly annoying log file format from a proprietary application containing data I like to monitor which contains time series values in a multiline format as well as error messages. What I'd like to do is to have Telegraf tail the log file and pass it through a script that generates actual influxdb data from that. So something similar like the telegraf tail plugin, but with a data transformation in between.
- pauldix 10y ago
  We added log parsing to Telegraf a month or so ago. See here for more details: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/logparser https://github.com/influxdata/telegraf/tree/master/plugins/i...
  - joshpadnick 10y ago
    I'm wondering if Telegraf is a legit replacement for logstash or fluentd for shipping logs. I couldn't find this info from docs, so do you know if: - it supports multi-line logs (e.g. java stack traces) - it can output to elasticsearch (didn't see an output plugin) - there's any solution for reading docker logs (looks like docker metrics are supported) - any other critical logstash functionality missing? If it doesn't support the classic Elasticsearch output, where are telegraf log users typically outputting logs to?
    - pauldix 10y ago
      Telegraf users are generally parsing the logs into structured metrics and events that go into InfluxDB. Not sure that it supports multi-line logs or docker logs. I'll have to look into it, but both should probably be done if they're not already there. For ES output, we're happy to take PRs for other output plugins. There are people using Telegraf that aren't using InfluxDB. That's fine and we're happy to have an open source collector that others use and contribute to.
zenlot 10y ago
While talking about InfluxDB people should not forget about their beta release of clustering to attract more users and then making it just enterprise. And no, it's not an argument that this is the only way to get paid in open source. Users should be careful while adopting InfluxDB as Influxdata does not clearly elaborate their plans on the product.
- deadbunny 10y ago
  Yeah, that's kinda a dick move. I was waiting for 1.0 before giving it another real go (after their decision to rewrite everything for 0.9). Looks like I'll be checking out something else now, maybe Prometheus[1] or DalmatinerDB[2] will fit the bill. 1. https://prometheus.io/ https://prometheus.io/ 2. https://dalmatiner.io/ https://dalmatiner.io/
  - pauldix 10y ago
    It's worth noting that Prometheus isn't distributed either. So OSS InfluxDB or that are the same in terms of distributed capabilities.
    - deleted 10y ago
      [deleted]
    - alexandre_m 10y ago
      Prometheus can be scaled with sharding and there are different solutions coming out to address this, like a proxy layer for queries. See the PromCon 2016 videos for latest news on this.
      - caniszczyk 10y ago
        The video is here: https://www.youtube.com/watch?list=PLoz-W_CUquUlCq-Q0hy53TolAhaED9vm&v=likpVWB5Lvo https://www.youtube.com/watch?list=PLoz-W_CUquUlCq-Q0hy53Tol...
    - chrissnell 10y ago
gorodetsky 10y ago
I still don't quite understand Chronograf: I know that you want to own the stack but are there any major advantages over Grafana? Sorry if I'm being ignorant but I couldn't find anything that would've made me think one way or another.
- pauldix 10y ago
  We're working on a re-envisioned Chronograf. The goal is to have something that's complementary to Grafana. Most of our users love Grafana and that's good. The next version of Chronograf, coming later this year, will be a re-envisioned and fully open source version. It won’t be about dashboards, it’ll be about an out of the box user experience for monitoring containers, Kubernetes, and Docker Swarm. We're actually looking for early testers that want to walk through wireframes and work with us on making a great out of the box experience for what will be a fully open source monitoring stack.
  - zenlot 10y ago
    Until you'll decide to make some of the features just for enterprise, as you did with InfluxDB?
    - dominotw 10y ago
      Just exploring this for devops at work. This comment makes me a little worried. Is there a risk that some features will go 'enterprise only' in future?
      - pauldix 10y ago
        Our enterprise offering is for HA and scale out clusters of InfluxDB. InfluxDB single server, Telegraf, Kapacitor single server, and soon Chronograf are all open source. We'll continue to heavily develop our open source projects in addition to developing closed source software that we can license to customers. Basically, to be able to continue open source development, we need to have paying customers.
tbarbugli 10y ago
We evaluated InfluxDB 6 months ago and did not move forward because cluster mode was still in beta. Is there anyone running a cluster in production with some decent traffic?
- iddqd 10y ago
  They yanked the clustering from the beta and put it into their Enterprise offering, so I guess you'll have to contact a sales rep for that kind of information.
aleksi 10y ago
> We had been grappling with what we should do for most of August 2013 and had another idea that I planned to debut at Monitorama in Berlin in late September. The conference was all about monitoring and I thought it would be a good place to find a receptive audience for a new monitoring product. What was it?
- pauldix 10y ago
  It was going to be called Anomalous. The idea behind it was that it would be an agent that gets deployed everywhere in your infrastructure that has collection, time series storage (either in memory, on-disk or both), alerting, and a basic web UI. The agents would all call to a central service, which would have bidirectional communication. The central service would then act as a distributed alerting/query aggregator. So you could hit that and query a specific agent, aggregate across multiple ones, etc. Kind of like a fully distributed time series, monitoring, and anomaly detection platform. I still like the idea, but we didn't have the runway or resources to go down that path. Open source got us much more traction, combined with the fact that I personally like to work on OSS projects :)
nwmcsween 10y ago
As per the article the issue influx tries to solve does so by trying to jam non-ts data into a ts database. You gain much more by simplifying things with say a constant increment mirror mmap file with another simple KV that maps offsets for tags and such. I would bet this would be immensely faster except for contrived update-tag-on-every-entry.
Rapzid 10y ago
We are using influxDB to calculate SLO performance. Currently, among other things, we process about 30M ELB logs entries into influxdb per day; it handles this easily of course. Here are some musings for those interested based on 0.9: Wins: * The new storage engine is very, very cool. Would love to work on this thing. It's fast and space efficient. * Built in support for time bucketing GROUP's * Grafana integration is pretty good * Writes come back after the data is stored; makes it easy to create durable, idempotent processing pipelines. Woahs * Unable to combine measurements in the same query; needs ETL with continuous queries or external tools * No support for subqueries; more ETL * Stream processing is a little lacking -> can't group on values and high cardinality tags make the DB explode; high cardinality is being worked on but IDK how high it will go, plus I mean the storage engine serves up streams of time-sorted data so samza that stuff up. * Random crashes but the DB gets along fine when it comes back up * Compactions use LOTS of RAM. Supposedly this can be tweaked and has been improved for 1.0 * Backfill queries with lots of points seem to use a crazy amount of RAM when bucketing on narrow time windows Overall it's chugging along quite well. Most of the query limitations we are able to solve with a combination of continuous queries and AWS lambda functions kicked off by CloudWatch Events.
Rapzid 10y ago
We are using influxDB to calculate SLO performance. Currently, among other things, we process about 30M ELB logs entries into influxdb per day; it handles this easily of course. Here are some musings for those interested based on 0.9: Wins: * The new storage engine is very, very cool. Would love to work on this thing. It's fast and space efficient. * Built in support for time bucketing GROUP's * Grafana integration is pretty good * Writes come back after the data is stored; makes it easy to create durable, idempotent processing pipelines. Woahs * Unable to combine measurements in the same query; needs ETL with continuous queries or external tools * No support for subqueries; more ETL * Stream processing is a little lacking -> can't group on values and high cardinality tags make the DB explode; high cardinality is being worked on but IDK how high it will go, plus I mean the storage engine serves up streams of time-sorted data so samza that stuff up. * Random crashes but the DB gets along fine when it comes back up * Compactions use LOTS of RAM. Supposedly this can be tweaked and has been improved for 1.0 * Backfill queries with lots of points seem to use a crazy amount of RAM when bucketing on narrow time windows Overall it's chugging along quite well. Most of the query limitations we are able to solve with a combination of continuous queries and AWS lambda functions kicked off by CloudWatch Events.

136 points by pauldix 10y ago

dozzie 10y ago

Packages built in the same shitty way as always. I expected nothing less from you guys.

pauldix 10y ago
We're happy to take PRs. Seems everyone has opinions on packaging and you know what it's like when everyone has opinions, right? :)
- dozzie 10y ago
  > Seems everyone has opinions on packaging and you know what it's like when everyone has opinions, right? :) Don't try to push this into opinion realm, because it's not, it's a standards realm. Your package doesn't stand lintian test, and it's not a doesn't-really-matter type of errors that there's no README or license file. You don't even put an initscript in packages in a proper way. It wouldn't matter that much, but your build system doesn't have an easy, clean build from local source tree with a method of putting files in right places in $DESTDIR, so the software could be properly debianized (or equivalent) by somebody else.
  - kfitzpatrick 10y ago
    dozzie, do you have a PR or issue open where you could help guide us to a good resolution?
    - dozzie 10y ago
      No. Why would I do so if I don't use your software, given that your product feels more like a paid thing than open source? You have enough money to hire competent sysadmin to teach you how to package software.
gyre007 10y ago
Well, if that's the only complaint you have, then I think Influx have done a stellar job :-)
- dozzie 10y ago
  It's the only complaint because I didn't bother looking further. It's not a sign of a good engineering when the build system and packaging are mess.
gtirloni 10y ago
At least the RPM one seems okay. You should elaborate.
pauldix 10y ago
Or maybe you were sad because you only saw package downloads from S3 and you were looking for an actual Debian repository? Well, that actually already exists. See: https://docs.influxdata.com/influxdb/v1.0/introduction/installation/#ubuntu-debian https://docs.influxdata.com/influxdb/v1.0/introduction/insta...
- dozzie 10y ago
  I'm glad you answered your own question, but it was you who asked that. I couldn't care less if they provided APT repository; using random repositories from various companies is asking for trouble with managing your servers (hint: package retention policies).
  - pauldix 10y ago
    I answered my own question because your original comment left no clue as to why you're so upset. Neither does this one that I'm now responding to.
    - linsomniac 10y ago
      Agreed, them providing .debs is far from "packaging being a mess". I'm quite happy that they provided .debs, worked with them to make some changes to the debs via pull-requests, and after that have been super happy with the packaging. I grab and import their packages into my own private repo for historic reasons, but it does mean I have to chase upstream changes. Personally, I think this is a straw-man argument. With many places not even providing packages, I'm glad that the Influx folks went through that process for me.
      - dozzie 10y ago
        > Agreed, them providing .debs is far from "packaging being a mess". I'm quite happy that they provided .debs, worked with them to make some changes to the debs via pull-requests, and after that have been super happy with the packaging. Clearly you haven't checked how they build their DEBs and RPMs. Some opaque, overcomplicated script that eventually calls fpm instead of proper debianization or spec file for RPM. This results, among the others, in some configs not being marked as configs. > I grab and import their packages into my own private repo for historic reasons, but it does mean I have to chase upstream changes. Personally, I think this is a straw-man argument. It's not. You can't rebuild a server if you suddenly don't have access to packages this server has installed, especially if you need them to be in specific versions. BTDT, several times.

otoolep 10y ago

Great stuff. Congrats to everyone on the InfluxDB team on this big milestone.

pauldix 10y ago
Thanks, Philip! You helped us get here :)

hamax 10y ago

A very happy InfluxDB user here, but did you really had to title your release email "InfluxDB 1.0 GA is Here! Plus 27x Faster than MongoDB Benchmarks"?

joekrie 10y ago

Cool to see a database specializing in time series data.

LogicX 10y ago

We've been using influx since 0.9 in production. Had a few bumps with cardinality growing out of control, but now working around those limits, it's going well. Looking forward to that being something tackled with upcoming releases.

pauldix 10y ago
Yep, we're very focused on solving the cardinality problem now. See these for some details about how we're thinking about it: https://github.com/influxdata/influxdb/pull/7175 https://github.com/influxdata/influxdb/pull/7175 https://github.com/influxdata/influxdb/pull/7174 https://github.com/influxdata/influxdb/pull/7174 https://github.com/influxdata/influxdb/pull/7186 https://github.com/influxdata/influxdb/pull/7186 https://github.com/influxdata/influxdb/pull/7264 https://github.com/influxdata/influxdb/pull/7264

linsomniac 10y ago

I've been using InfluxDB for almost a year now. At one point around 9 months ago I had given up on it because it was a bit crashy, and the database just was growing too fast. But the promise behind it was too compelling and I started experimenting with newer versions around 6 months ago and it has been just great! Much easier to deal with than Graphite/collectd/carbon, telegraf has not been eating our servers like collectd was, CPU usage is way down... Loving InfluxDB. Still need to implement annotations and SNMP polling in telegraf, but it is awesome. We are even pushing some application stats into it.

pauldix 10y ago
We updated the SNMP plugin a few weeks ago. Was thanks to a contributor who has been super helpful. Does the updated one do what you need?
- imaginenore 10y ago
  Please pay the guy. He is helping your product.
  - rrhyne 10y ago
    So, super obvious, but thought I'd just mention that influxdb is open source. It's likely that free version helped the contributor with his issue so he contributed back.
    - imaginenore 10y ago
      You don't think the enterprise users can use his work?
      - tinco 10y ago
        Even if he contributed, I think it's likely that InfluxDB helped they contributor more than he helped InfluxDB.
      - 10y ago

scrollaway 10y ago

Influx is absolutely amazing. We're using it along with grafana to store and display our desktop and web apps' analytics (it completely replaced GA), store and display HTTP health analytics (piping custom uwsgi request logger into UDP input), and do continuous analysis of Hearthstone games. It's incredibly fast and the grafana/influx/telegraf stack is really cool to play with. Highly recommended.

chrissnell 10y ago
Want to hear more about your HTTP analytics. Particularly interested in collecting selenium/sitespeed.io-style page timing metrics. There are a million commercial solutions for this but so few open source options.
- scrollaway 10y ago
  Glad you asked! I'm pretty proud of this hack :) uWSGI has a very flexible logging system. You can create a log with just requests, and completely customize the format of the line. On top of that, uWSGI supports creating multiple log targets, and supports logging directly into a UDP socket. If you make it match InfluxDB's line format (https://docs.influxdata.com/influxdb/v1.0/write_protocols/line/ https://docs.influxdata.com/influxdb/v1.0/write_protocols/li...), and you set up InfluxDB to accept connections from a matching UDP socket, then uWSGI can essentially log every request directly into influx. Like this for example: logformat = uwsgirequest,host="%(host)",status=%(status) msecs=%(msecs),size=%(size) %(epoch)000000000 logger = file:/var/logs/uwsgi/uwsgi.log req-logger = file:/var/logs/uwsgi/requests.log req-logger = socket:127.0.0.1:8083 You can also add 'path="%(var.PATH_INFO)"' to that but there is a potential data injection vulnerability if you don't re-parse the output. With that, you can create nice graphs with Grafana to analyze your loading times, error rates, page sizes etc: https://i.imgur.com/fzbXgUd.png https://i.imgur.com/fzbXgUd.png You can also set up Kapacitor to identify when your page load times are abnormally increasing. We had the idea after a database connection leak increased our average loading times by 300%. Do note that this doesn't batch inputs, so 1 request = 1 connection to influx. But it's pretty easy to put a middleware agent in between those to do validation and batching. The nice thing about all this is it's application level timing metrics. There's not a lot of ways to get that, except with application middleware which can't always catch everything you want.

fabian2k 10y ago

I played around with InfluxDB, Telegraf and Grafana a while ago, and it worked very nicely for the basic stuff I tried. One thing in Telegraf where I didn't figure out a good solution was a way to parse arbitrary log files and generate data points and/or annotations from them. There is a particularly annoying log file format from a proprietary application containing data I like to monitor which contains time series values in a multiline format as well as error messages. What I'd like to do is to have Telegraf tail the log file and pass it through a script that generates actual influxdb data from that. So something similar like the telegraf tail plugin, but with a data transformation in between.

pauldix 10y ago
We added log parsing to Telegraf a month or so ago. See here for more details: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/logparser https://github.com/influxdata/telegraf/tree/master/plugins/i...
- joshpadnick 10y ago
  I'm wondering if Telegraf is a legit replacement for logstash or fluentd for shipping logs. I couldn't find this info from docs, so do you know if: - it supports multi-line logs (e.g. java stack traces) - it can output to elasticsearch (didn't see an output plugin) - there's any solution for reading docker logs (looks like docker metrics are supported) - any other critical logstash functionality missing? If it doesn't support the classic Elasticsearch output, where are telegraf log users typically outputting logs to?
  - pauldix 10y ago
    Telegraf users are generally parsing the logs into structured metrics and events that go into InfluxDB. Not sure that it supports multi-line logs or docker logs. I'll have to look into it, but both should probably be done if they're not already there. For ES output, we're happy to take PRs for other output plugins. There are people using Telegraf that aren't using InfluxDB. That's fine and we're happy to have an open source collector that others use and contribute to.

zenlot 10y ago

While talking about InfluxDB people should not forget about their beta release of clustering to attract more users and then making it just enterprise. And no, it's not an argument that this is the only way to get paid in open source. Users should be careful while adopting InfluxDB as Influxdata does not clearly elaborate their plans on the product.

deadbunny 10y ago
Yeah, that's kinda a dick move. I was waiting for 1.0 before giving it another real go (after their decision to rewrite everything for 0.9). Looks like I'll be checking out something else now, maybe Prometheus[1] or DalmatinerDB[2] will fit the bill. 1. https://prometheus.io/ https://prometheus.io/ 2. https://dalmatiner.io/ https://dalmatiner.io/
- pauldix 10y ago
  It's worth noting that Prometheus isn't distributed either. So OSS InfluxDB or that are the same in terms of distributed capabilities.
  - deleted 10y ago
    [deleted]
  - alexandre_m 10y ago
    Prometheus can be scaled with sharding and there are different solutions coming out to address this, like a proxy layer for queries. See the PromCon 2016 videos for latest news on this.
    - caniszczyk 10y ago
      The video is here: https://www.youtube.com/watch?list=PLoz-W_CUquUlCq-Q0hy53TolAhaED9vm&v=likpVWB5Lvo https://www.youtube.com/watch?list=PLoz-W_CUquUlCq-Q0hy53Tol...
  - chrissnell 10y ago

gorodetsky 10y ago

I still don't quite understand Chronograf: I know that you want to own the stack but are there any major advantages over Grafana? Sorry if I'm being ignorant but I couldn't find anything that would've made me think one way or another.

pauldix 10y ago
We're working on a re-envisioned Chronograf. The goal is to have something that's complementary to Grafana. Most of our users love Grafana and that's good. The next version of Chronograf, coming later this year, will be a re-envisioned and fully open source version. It won’t be about dashboards, it’ll be about an out of the box user experience for monitoring containers, Kubernetes, and Docker Swarm. We're actually looking for early testers that want to walk through wireframes and work with us on making a great out of the box experience for what will be a fully open source monitoring stack.
- zenlot 10y ago
  Until you'll decide to make some of the features just for enterprise, as you did with InfluxDB?
  - dominotw 10y ago
    Just exploring this for devops at work. This comment makes me a little worried. Is there a risk that some features will go 'enterprise only' in future?
    - pauldix 10y ago
      Our enterprise offering is for HA and scale out clusters of InfluxDB. InfluxDB single server, Telegraf, Kapacitor single server, and soon Chronograf are all open source. We'll continue to heavily develop our open source projects in addition to developing closed source software that we can license to customers. Basically, to be able to continue open source development, we need to have paying customers.

tbarbugli 10y ago

We evaluated InfluxDB 6 months ago and did not move forward because cluster mode was still in beta. Is there anyone running a cluster in production with some decent traffic?

iddqd 10y ago
They yanked the clustering from the beta and put it into their Enterprise offering, so I guess you'll have to contact a sales rep for that kind of information.

aleksi 10y ago

> We had been grappling with what we should do for most of August 2013 and had another idea that I planned to debut at Monitorama in Berlin in late September. The conference was all about monitoring and I thought it would be a good place to find a receptive audience for a new monitoring product. What was it?

pauldix 10y ago
It was going to be called Anomalous. The idea behind it was that it would be an agent that gets deployed everywhere in your infrastructure that has collection, time series storage (either in memory, on-disk or both), alerting, and a basic web UI. The agents would all call to a central service, which would have bidirectional communication. The central service would then act as a distributed alerting/query aggregator. So you could hit that and query a specific agent, aggregate across multiple ones, etc. Kind of like a fully distributed time series, monitoring, and anomaly detection platform. I still like the idea, but we didn't have the runway or resources to go down that path. Open source got us much more traction, combined with the fact that I personally like to work on OSS projects :)

nwmcsween 10y ago

As per the article the issue influx tries to solve does so by trying to jam non-ts data into a ts database. You gain much more by simplifying things with say a constant increment mirror mmap file with another simple KV that maps offsets for tags and such. I would bet this would be immensely faster except for contrived update-tag-on-every-entry.

Rapzid 10y ago

We are using influxDB to calculate SLO performance. Currently, among other things, we process about 30M ELB logs entries into influxdb per day; it handles this easily of course. Here are some musings for those interested based on 0.9: Wins: * The new storage engine is very, very cool. Would love to work on this thing. It's fast and space efficient. * Built in support for time bucketing GROUP's * Grafana integration is pretty good * Writes come back after the data is stored; makes it easy to create durable, idempotent processing pipelines. Woahs * Unable to combine measurements in the same query; needs ETL with continuous queries or external tools * No support for subqueries; more ETL * Stream processing is a little lacking -> can't group on values and high cardinality tags make the DB explode; high cardinality is being worked on but IDK how high it will go, plus I mean the storage engine serves up streams of time-sorted data so samza that stuff up. * Random crashes but the DB gets along fine when it comes back up * Compactions use LOTS of RAM. Supposedly this can be tweaked and has been improved for 1.0 * Backfill queries with lots of points seem to use a crazy amount of RAM when bucketing on narrow time windows Overall it's chugging along quite well. Most of the query limitations we are able to solve with a combination of continuous queries and AWS lambda functions kicked off by CloudWatch Events.

Rapzid 10y ago