3 ms·

Compute time and storage costs are also bottlenecks, so even though the base calling through VCF is automated large genome centers struggle to keep up with the

• gbveiuwlbiu 11y ago
  When you say compute time, do you have in mind purely quantitative processing/analytics, or streaming and data transfer? Is there any way you could describe a sample "experiment" and the associated storage costs and compute time, just a paper-and-pencil estimation perhaps?
  • zitterbewegung 11y ago
    I went to a python talk where the guy did genomics . I asked him how long his program would take and he said I'll tell you when it's done. This means his analysis programs can take days to weeks and he really didn't know how long the current one can take .
    • akuchling 11y ago
      Possibly C. Titus Brown? Here's his PyCon 2014 talk, "Data intensive biology in the cloud": https://www.youtube.com/watch?v=RCRbfKK57X8 https://www.youtube.com/watch?v=RCRbfKK57X8
• mbreese 11y ago
  Right - but we used to struggle with the output of the last generation of sequencers too. Computing capacity and storage have scaled at about the same rate. Or perhaps it's more accurate to say that the costs have come down, so we can throw more iron at the problem - these are embarrassingly parallel problems... As an example, the last sequencing analysis cluster I setup (~2010) ran with 2x bonded gigabit networks between our compute nodes and storage and had 10TB of network storage. The newest one is running 10GigE, has 200+TB of storage, and is cheaper. And we are a small shop in this field.