3 ms·

"The storage requirements are just too high" Yeah? Can you elaborate on how big we're talking here?

by VikingCoder 11y ago

adenadel 11y ago
100GB per person for the raw data from 30X WGS
- cinquemb 11y ago
  So right now, one can buy 5 TB hard-rive for about $130 (lets ignore bulk buying because I have no knowledge of such). If you theoretically could get 30 Million people at 30X WGS, that would be about 3M terabytes (600k drives) and you would need at least $78M, which seems a bit expensive, but it seems feasible if you were planning on capitalizing for at least a order of magnitude more, no?
  - adenadel 11y ago
    Yes, in fact the UK is doing the first real population sequencing project right now http://www.genomicsengland.co.uk/ http://www.genomicsengland.co.uk/ This all isn't impossible. HLI is also sequencing well over 10000 genomes per year at this point. The service requested (having your data delivered and then deleted) already exists, but is not the most consumer friendly option.
  - a_bonobo 11y ago
    BGI can theoretically sequence 100 human genomes per day now, with that rate only increasing: http://www.genomics.cn/en/news/show_news?nid=98952 http://www.genomics.cn/en/news/show_news?nid=98952 So 100GB*100=10000GB=10TB per day, that's quite a lot of data but not unheard of in the age of Google. And from there you'd get the additional data from read-alignments, SNP-calling, genome re-annotation etc. pp.
  - tetron 11y ago
    Buying those 600,000 drives is probably only 10% of your costs, you need to run a data center if you want to actually use that data. Also analysis pipelines generate intermediate results that are also large, and data is stored with 2x or 3x redundancy. So that 100GB might be more like 300GB once you factor in robust storage and bioinformatics.
    - cinquemb 11y ago
      Still, multiply that by 3, then 1.5 for the overhead… that's still about 5 months of QE :P A post doc at who worked at ATLAS came to visit our lab a couple weeks a go, and just listening to the workflow they have for the events coming off the detectors… and all the BS they have to deal with working in academia… yeah… I'll take that money, erect a CSP system up in Spain and ask them to come do this with me, just to start off with a fun side project :P

"The storage requirements are just too high" Yeah? Can you elaborate on how big we're talking here?

adenadel 11y ago

100GB per person for the raw data from 30X WGS

cinquemb 11y ago
So right now, one can buy 5 TB hard-rive for about $130 (lets ignore bulk buying because I have no knowledge of such). If you theoretically could get 30 Million people at 30X WGS, that would be about 3M terabytes (600k drives) and you would need at least $78M, which seems a bit expensive, but it seems feasible if you were planning on capitalizing for at least a order of magnitude more, no?
- adenadel 11y ago
  Yes, in fact the UK is doing the first real population sequencing project right now http://www.genomicsengland.co.uk/ http://www.genomicsengland.co.uk/ This all isn't impossible. HLI is also sequencing well over 10000 genomes per year at this point. The service requested (having your data delivered and then deleted) already exists, but is not the most consumer friendly option.
- a_bonobo 11y ago
  BGI can theoretically sequence 100 human genomes per day now, with that rate only increasing: http://www.genomics.cn/en/news/show_news?nid=98952 http://www.genomics.cn/en/news/show_news?nid=98952 So 100GB*100=10000GB=10TB per day, that's quite a lot of data but not unheard of in the age of Google. And from there you'd get the additional data from read-alignments, SNP-calling, genome re-annotation etc. pp.
- tetron 11y ago
  Buying those 600,000 drives is probably only 10% of your costs, you need to run a data center if you want to actually use that data. Also analysis pipelines generate intermediate results that are also large, and data is stored with 2x or 3x redundancy. So that 100GB might be more like 300GB once you factor in robust storage and bioinformatics.
  - cinquemb 11y ago
    Still, multiply that by 3, then 1.5 for the overhead… that's still about 5 months of QE :P A post doc at who worked at ATLAS came to visit our lab a couple weeks a go, and just listening to the workflow they have for the events coming off the detectors… and all the BS they have to deal with working in academia… yeah… I'll take that money, erect a CSP system up in Spain and ask them to come do this with me, just to start off with a fun side project :P