4 ms·
> 750,000 GB = 750 TB, and you can buy an 8 TB hard drive for $150. So for $15,000 you can store 500,000 human genomes. As others have mentioned [1], the raw s
by mmt 8y ago
> 750,000 GB = 750 TB, and you can buy an 8 TB hard drive for $150. So for $15,000 you can store 500,000 human genomes.
As others have mentioned [1], the raw sequencing data (still) has to be stored, so 1.5GB is off by about 30x. Even at 6:1 compression, that's still 5x.
More importantly, though, the cost of storage, especially at scale, is significantly higher than just the price of bare drives. Backblaze famously complained about this [2] in 2009.
Even their particularly cheap, particularly low-performance solution claimed to add almost 45% on top of the cost of the bare drives. A higher-performancd storage system (using, say, standard SAS expander backplanes instead of their non-standard SATA multipliers) has, in my experience, added 70% on top of the bare drives (and that's being frugal, which seems remarkably rare, even among startups).
Your $15k is now $128k, minimum. That's just purchase cost, and, while operating cost isn't huge for that little storage, it's not zero. The cost gets much, much worse, if the orgnization can't/won't hire someone like me or won't allow that someone to implement such a frugal DIY assembly (which is the vast majority of organizations today).
That leaves "enterprise" storage or cloud, which are both approximately as expensive. I recently heard an estimate of half a million dollars per petabyte for NetApp, which would translate to $1,875,000 here.
[1] Well detailed by https://news.ycombinator.com/item?id=17821523 https://news.ycombinator.com/item?id=17821523
[2] famous on HN, anyway. https://www.backblaze.com/blog/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ https://www.backblaze.com/blog/petabytes-on-a-budget-how-to-...
- wenc 8y agoCloud: just did a quick calculation for AWS. 75 PB (= 150GB * 500,000) of storage costs $300,000 on AWS Glacier, and $1.57mil on S3 Standard. And that's before accounting for any read/write costs. Wow.
- mmt 8y agoI was using 3.75PB (assuming 45GB compressed 6:1 so 7.5GB per person). I'm not certain, but my intuition is that, even paying AWS for the CPU time for compression, that would be cheaper than paying for the uncompressed transfer charges. But, yes, even for my lower storage numbers, for "EU (London)", even for just 3 years (a conservative enough lifespan for an HDD), S3-Standard would be almost $2.97M, S3-Standard-IA would be $1.77M, S3-OneZone-IA $1.4M, and Glacier would be merely $608k. Of course, that excludes transfer charges.