3 ms·

In addition to this helpful guide, note that statsd / graphite both spring some unfortunate surprises on new users, e.g., graphite changing your data across ret

by another 13y ago

In addition to this helpful guide, note that statsd / graphite both spring some unfortunate surprises on new users, e.g., graphite changing your data across retention rates and time scales [0], graphite changing your data at different plot widths (?!) [1], statsd believing that only count and time data deserve to be aggregated [2], etc. I have no alternative to suggest, however. Perhaps Cube [3], but unclear if it has any user community. [0] http://stackoverflow.com/questions/10820119/graphite-is-not-graphing-anything-for-ranges-bigger-than-7-hours http://stackoverflow.com/questions/10820119/graphite-is-not-... [1] http://graphite.readthedocs.org/en/1.0/functions.html#graphite.render.functions.cumulative http://graphite.readthedocs.org/en/1.0/functions.html#graphi... [2] https://github.com/etsy/statsd/issues/98 https://github.com/etsy/statsd/issues/98 [3] https://github.com/square/cube https://github.com/square/cube

pestaa 13y ago
Changing data in undesired ways is something I'd call a bug, not a surprise.
- sciurus 13y ago
  It's not a bug; carbon was behaving exactly the way it was configured to behave. This wouldn't be surprising to anyone who is familiar with RRDTool. However, since one of the reasons graphite uses its own file format (whisper) instead of RRD is to better handle intermittent values, I could see the argument that the default xFilesFactor should be higher. "xFilesFactor should be a floating point number between 0 and 1, and specifies what fraction of the previous retention level’s slots must have non-null values in order to aggregate to a non-null value. The default is 0.5." - http://graphite.readthedocs.org/en/1.0/config-carbon.html http://graphite.readthedocs.org/en/1.0/config-carbon.html
jlgreco 13y ago
Fixing statsd is easy at least, that program is trivial.
- another 13y ago
  Oh, yeah, it's weird but not even a serious limitation: it's being fixed, other statsd clones have more features, and you can always pretend your data are time intervals. statsd is limited but nicely simple.
fourk 13y ago
Re [0]: If you never want your data downsampled, keep data at a single resolution which is equal to the flush interval used to push data to Graphite. Carbon will never "change your data" under such a configuration. Re [1]: How would you expect the presentation layer to present >n data points using n pixels? Graphite doesn't "change your data". Presentation of data != the data itself, just as a map of a city != the city itself.
- another 13y ago
  > If you never want your data downsampled, keep data at a single resolution... Sure, and many people do exactly that. The point is that a new user to graphite is likely to be surprised by this behavior. (I would further bet that a reasonable fraction of statsd+graphite users end up viewing incorrect data without realizing it, especially given the statsd focus on count data, for which the default aggregationMethod setting is exactly the wrong choice.) (And even awareness of this behavior isn't quite enough, since every user needs to also remember their server's exact storage configuration, lest they inadvertently expand their plot across a retention boundary.) > How would you expect the presentation layer to present n data points using n pixels? The same way that most plotting tools do so: by overdrawing. Yes, one ends up with a solid block of pixels if the data are noisy and the plot is small, but that outcome is easily understood and has the easily understood solution of explicitly aggregating appropriately. Graphite instead takes the approach of implicitly aggregating based on how wide the plot is rendered in a given interface. That behavior is, at the very least, surprising.

by another 13y ago

pestaa 13y ago

Changing data in undesired ways is something I'd call a bug, not a surprise.

sciurus 13y ago
It's not a bug; carbon was behaving exactly the way it was configured to behave. This wouldn't be surprising to anyone who is familiar with RRDTool. However, since one of the reasons graphite uses its own file format (whisper) instead of RRD is to better handle intermittent values, I could see the argument that the default xFilesFactor should be higher. "xFilesFactor should be a floating point number between 0 and 1, and specifies what fraction of the previous retention level’s slots must have non-null values in order to aggregate to a non-null value. The default is 0.5." - http://graphite.readthedocs.org/en/1.0/config-carbon.html http://graphite.readthedocs.org/en/1.0/config-carbon.html

jlgreco 13y ago

Fixing statsd is easy at least, that program is trivial.

another 13y ago
Oh, yeah, it's weird but not even a serious limitation: it's being fixed, other statsd clones have more features, and you can always pretend your data are time intervals. statsd is limited but nicely simple.

fourk 13y ago

Re [0]: If you never want your data downsampled, keep data at a single resolution which is equal to the flush interval used to push data to Graphite. Carbon will never "change your data" under such a configuration. Re [1]: How would you expect the presentation layer to present >n data points using n pixels? Graphite doesn't "change your data". Presentation of data != the data itself, just as a map of a city != the city itself.

another 13y ago
> If you never want your data downsampled, keep data at a single resolution... Sure, and many people do exactly that. The point is that a new user to graphite is likely to be surprised by this behavior. (I would further bet that a reasonable fraction of statsd+graphite users end up viewing incorrect data without realizing it, especially given the statsd focus on count data, for which the default aggregationMethod setting is exactly the wrong choice.) (And even awareness of this behavior isn't quite enough, since every user needs to also remember their server's exact storage configuration, lest they inadvertently expand their plot across a retention boundary.) > How would you expect the presentation layer to present n data points using n pixels? The same way that most plotting tools do so: by overdrawing. Yes, one ends up with a solid block of pixels if the data are noisy and the plot is small, but that outcome is easily understood and has the easily understood solution of explicitly aggregating appropriately. Graphite instead takes the approach of implicitly aggregating based on how wide the plot is rendered in a given interface. That behavior is, at the very least, surprising.