4 ms·

But then I have to maintain a Riak cluster.

by saryant 12y ago

johansch 12y ago
For all means, if you have LOTS more money than time, you should pay someone else to do this. Outside of certain sheltered silicon valley startups, this is often not the case though. :)
- eropple 12y ago
  You know, I'm pretty sure that almost everyone is better off spending a few cents per gigabyte per month on S3 for inert file storage, a functionality that's performance-inelastic except at the far end of the curve, than maintaining a Riak cluster. Source: I've maintained a Riak cluster. There are lots of AWS services I avoid (I won't use SQS, I won't use Dynamo except during prototyping, etc.), but S3 is so utterly portable across any cloud you go to (and latency is already not something you are worrying about if you're using S3) that it's laughable to suggest that you're going to roll your own with anywhere near the reliability at anywhere near the cost.
  - saryant 12y ago
    Yeah, we use S3 at the company I work for a little under a terabyte and we pay less than $25 a month.
  - johansch 12y ago
    We rolled our own proprietary version of what became known as S3 and scaled it up to several thousand nodes. At any given time, less than half a FTE spent any time on this, and I think we've had roughly equal (probably better) availability than that of S3, at a fraction of the cost.
    - eropple 12y ago
      I spent zero FTEs on S3 and average around twenty-five bucks, or under a third of an FTE's hourly amortized rate, per terabyte. You might not be in "almost everyone", but almost everyone else is.
    - nucleardog 12y ago
      Assuming you pay your engineers a paltry $50k/yr, half of a FTE is $25k/yr. Quick calculation based on the public S3 pricing says that's 847TB of data you could be storing just with their salary. That doesn't include all the actual costs associated with running thousands of nodes - acquisition, hardware replacements, utilities and other operating costs, space, etc. (To be honest, with that many nodes I have a hard time believing that you're not spending half a FTE just replacing dead drives.) Not to mention the additional costs if you're actually interested in the same level of durability (target eleven 9s (99.999999999%), but last time I talked to an engineer they're still at 100% since leaving beta). Multiple DCs, transit between them... I'm sure there's a scale at which it's worth it, but it's pretty far up there and for almost everyone S3 is a cheaper option.
- untog 12y ago
  I think you're overestimating how expensive S3 is. You'd have to be storing a lot of data before running your own infrastructure becomes more affordable (especially factoring in employee costs) Not to say it never happens, but 90% of people are better off just using S3.
- deleted 12y ago
  [deleted]

But then I have to maintain a Riak cluster.

by saryant 12y ago

But then I have to maintain a Riak cluster.

For all means, if you have LOTS more money than time, you should pay someone else to do this. Outside of certain sheltered silicon valley startups, this is often not the case though. :)

eropple 12y ago
You know, I'm pretty sure that almost everyone is better off spending a few cents per gigabyte per month on S3 for inert file storage, a functionality that's performance-inelastic except at the far end of the curve, than maintaining a Riak cluster. Source: I've maintained a Riak cluster. There are lots of AWS services I avoid (I won't use SQS, I won't use Dynamo except during prototyping, etc.), but S3 is so utterly portable across any cloud you go to (and latency is already not something you are worrying about if you're using S3) that it's laughable to suggest that you're going to roll your own with anywhere near the reliability at anywhere near the cost.
- saryant 12y ago
  Yeah, we use S3 at the company I work for a little under a terabyte and we pay less than $25 a month.
- johansch 12y ago
  We rolled our own proprietary version of what became known as S3 and scaled it up to several thousand nodes. At any given time, less than half a FTE spent any time on this, and I think we've had roughly equal (probably better) availability than that of S3, at a fraction of the cost.
  - eropple 12y ago
    I spent zero FTEs on S3 and average around twenty-five bucks, or under a third of an FTE's hourly amortized rate, per terabyte. You might not be in "almost everyone", but almost everyone else is.
  - nucleardog 12y ago
    Assuming you pay your engineers a paltry $50k/yr, half of a FTE is $25k/yr. Quick calculation based on the public S3 pricing says that's 847TB of data you could be storing just with their salary. That doesn't include all the actual costs associated with running thousands of nodes - acquisition, hardware replacements, utilities and other operating costs, space, etc. (To be honest, with that many nodes I have a hard time believing that you're not spending half a FTE just replacing dead drives.) Not to mention the additional costs if you're actually interested in the same level of durability (target eleven 9s (99.999999999%), but last time I talked to an engineer they're still at 100% since leaving beta). Multiple DCs, transit between them... I'm sure there's a scale at which it's worth it, but it's pretty far up there and for almost everyone S3 is a cheaper option.
untog 12y ago
I think you're overestimating how expensive S3 is. You'd have to be storing a lot of data before running your own infrastructure becomes more affordable (especially factoring in employee costs) Not to say it never happens, but 90% of people are better off just using S3.
deleted 12y ago
[deleted]