3 ms·

Cloud is also (more so hopefully) about, dynamically ramping up based on actual usage. Vs making guesses about future capacity needs. Cloud is to capacity pla

Cloud is also (more so hopefully) about, dynamically ramping up based on actual usage. Vs making guesses about future capacity needs. Cloud is to capacity planning as agile is to waterfall. Re RRD, have you read about graphite? "Graphite originally did use RRD for storage until fundamental limitations arose that required a new storage engine."

peterwwillis 15y ago
Dynamically ramping is nice until your site explodes and you need 20 minutes to get more capacity. Versus just pre-allocating it and not going down. Call me crazy, some people don't like to be down for 20 minutes. Hmm, didn't know that. Too bad they didn't just extend RRD. Did they say what the limitations were? I see a note about high volume causing lots of writes and implementing caching to deal with it, but that can be dealt with via tuned filesystem parameters... Ah, I found the page: http://graphite.wikidot.com/whisper http://graphite.wikidot.com/whisper RRD can be tuned to ignore 'irregular' data points, or include them all. The timestamp issue can be a problem but there are methods to deal with order of updates (like take them via tcp, or rrd merge tools). If you have a lot of RAM to spare, an excellent hack is putting your RRDs with the highest amount of writes in a tmpfs volume and rsync'ing them regularly (it's insanely fast, trust me). More on tuning RRD: http://oss.oetiker.ch/rrdtool-trac/wiki/TuningRRD http://oss.oetiker.ch/rrdtool-trac/wiki/TuningRRD http://oss.oetiker.ch/rrdtool/doc/rrdcached.en.html http://oss.oetiker.ch/rrdtool/doc/rrdcached.en.html http://sourceforge.net/apps/trac/ganglia/wiki/rrdcached_integration http://sourceforge.net/apps/trac/ganglia/wiki/rrdcached_inte... http://community.zenoss.org/docs/DOC-4696 http://community.zenoss.org/docs/DOC-4696
- mikeyk 15y ago
  To clarify: we did do a fair amount of capacity planning and elastic ramp-up/pre-allocating of our infrastructure, but no prediction is perfect, so the blog was about diagnosing and addressing issues as they crop up under unprecedented load.
- sciurus 15y ago
  Some more useful RRD info: http://collectd.org/wiki/index.php/Inside_the_RRDtool_plugin http://collectd.org/wiki/index.php/Inside_the_RRDtool_plugin

peterwwillis 15y ago

Dynamically ramping is nice until your site explodes and you need 20 minutes to get more capacity. Versus just pre-allocating it and not going down. Call me crazy, some people don't like to be down for 20 minutes. Hmm, didn't know that. Too bad they didn't just extend RRD. Did they say what the limitations were? I see a note about high volume causing lots of writes and implementing caching to deal with it, but that can be dealt with via tuned filesystem parameters... Ah, I found the page: http://graphite.wikidot.com/whisper http://graphite.wikidot.com/whisper RRD can be tuned to ignore 'irregular' data points, or include them all. The timestamp issue can be a problem but there are methods to deal with order of updates (like take them via tcp, or rrd merge tools). If you have a lot of RAM to spare, an excellent hack is putting your RRDs with the highest amount of writes in a tmpfs volume and rsync'ing them regularly (it's insanely fast, trust me). More on tuning RRD: http://oss.oetiker.ch/rrdtool-trac/wiki/TuningRRD http://oss.oetiker.ch/rrdtool-trac/wiki/TuningRRD http://oss.oetiker.ch/rrdtool/doc/rrdcached.en.html http://oss.oetiker.ch/rrdtool/doc/rrdcached.en.html http://sourceforge.net/apps/trac/ganglia/wiki/rrdcached_integration http://sourceforge.net/apps/trac/ganglia/wiki/rrdcached_inte... http://community.zenoss.org/docs/DOC-4696 http://community.zenoss.org/docs/DOC-4696

mikeyk 15y ago
To clarify: we did do a fair amount of capacity planning and elastic ramp-up/pre-allocating of our infrastructure, but no prediction is perfect, so the blog was about diagnosing and addressing issues as they crop up under unprecedented load.
sciurus 15y ago
Some more useful RRD info: http://collectd.org/wiki/index.php/Inside_the_RRDtool_plugin http://collectd.org/wiki/index.php/Inside_the_RRDtool_plugin