3 ms·

FYI, I'm one of the founders of SpiderOak, it case that was not clear (it's in my profile.) For the storage cost calculations: electricity, cooling, and bandwi

FYI, I'm one of the founders of SpiderOak, it case that was not clear (it's in my profile.) For the storage cost calculations: electricity, cooling, and bandwidth end up costing more than the drives, even once you include all additional hardware to keep them running. Note of caution: I would recommend that shops build bulk storage hardware themselves only if someone on their team has an intensive storage background. There are a number of gotchas that are distinctly non obvious; they are capital intensive lessons to learn, and endanger data integrity. You saw what Backblaze went through to design their storage pods: all of the self hosted offsite backup companies do likewise. It's a core competency. Typically it's much better to outsource or buy the more expensive business storage products with hardware RAID, enterprise class drives, cache batteries, and all that. SpiderOak's DIY is designed for high reliability, but not high performance. In price comparisons to S3, there's no option for reduced-reliability storage, but also no charge for bandwidth in our out (up to a reasonable point.) Having said all that, if you are interested in building 100+ TB of storage hardware in house, feel free to send me a mail; I may be able to save you some difficulty. If you'd rather us host it, we do discounts for startups. :)

e1ven 15y ago
Oh, Man, that'll teach me to look more carefully ;) Oops. Thanks for the awesome service. I've used SO for backups on and off, and I appreciate the tech work you do. I particularly like that you do client-side encryption. I just wish your UI was a bit better ;) I must have been mixing up your service with diomede; IIRC, their techcrunch article was about letting you choose the levels of redundancy you want to keep. I'll grant you that the bandwidth is certainly a price-factor, but your page talks a LOT about how you're higher latency than S3, and not everyone needs that speed for that price, etc, etc, and then your price comes out to be about 80% of theirs. Honestly, after reading your product positioning, I was envisioning it coming out to like 10%. You're absolutely right that building a huge in-house storage network is a huge ordeal.. If you're worried about making it fast and scalable. What I want to do is to treat it as the equivalent of a huge tape drive- Just throw stuff onto it, and hope for the best. I've been toying around with the idea of dealing with it entirely in application logic. Rather than RAIDing the disks, mount each one separately, and keep a hashtable of my files in memory with their full path, including redundant copies. Then, if a file fails, I have application logic try the backup copy. Anyway, you know a LOT more about this than I do, so I'll shut up before I embarrass myself further. My point is just that there's a lot of people in the market who have huge datasets currently, for various reasons. Lots of science departments, analytics groups, etc. I'd love to be able to keep an nearline copy of work. I don't mind if it takes a good 15-20 minutes to be able to access the first byte, I just want to know that I can if I need to, faster than waiting for Iron Mountain to ship me something. Side note- I considered trying to justify it under BackBlaze's "$5, unlimited space" offer, but I didn't think they'd go for it. ;)

e1ven 15y ago

Oh, Man, that'll teach me to look more carefully ;) Oops. Thanks for the awesome service. I've used SO for backups on and off, and I appreciate the tech work you do. I particularly like that you do client-side encryption. I just wish your UI was a bit better ;) I must have been mixing up your service with diomede; IIRC, their techcrunch article was about letting you choose the levels of redundancy you want to keep. I'll grant you that the bandwidth is certainly a price-factor, but your page talks a LOT about how you're higher latency than S3, and not everyone needs that speed for that price, etc, etc, and then your price comes out to be about 80% of theirs. Honestly, after reading your product positioning, I was envisioning it coming out to like 10%. You're absolutely right that building a huge in-house storage network is a huge ordeal.. If you're worried about making it fast and scalable. What I want to do is to treat it as the equivalent of a huge tape drive- Just throw stuff onto it, and hope for the best. I've been toying around with the idea of dealing with it entirely in application logic. Rather than RAIDing the disks, mount each one separately, and keep a hashtable of my files in memory with their full path, including redundant copies. Then, if a file fails, I have application logic try the backup copy. Anyway, you know a LOT more about this than I do, so I'll shut up before I embarrass myself further. My point is just that there's a lot of people in the market who have huge datasets currently, for various reasons. Lots of science departments, analytics groups, etc. I'd love to be able to keep an nearline copy of work. I don't mind if it takes a good 15-20 minutes to be able to access the first byte, I just want to know that I can if I need to, faster than waiting for Iron Mountain to ship me something. Side note- I considered trying to justify it under BackBlaze's "$5, unlimited space" offer, but I didn't think they'd go for it. ;)