7 ms·

The bottleneck in genomics hasn't been cost since about 2012. The chokepoint is analysis. Most biologists don't know how to program, and most programmers don't

by searine 11y ago

The bottleneck in genomics hasn't been cost since about 2012. The chokepoint is analysis. Most biologists don't know how to program, and most programmers don't know biological context, neither know statistics well. I work with so many scientists whose only thought is to sequence first and ask questions later. Usually all the real work ends up falling on the shoulders of one skilled researcher while the rest look on like some unionized road crew. It's only going to get worse, but the good news is, if you are one of the biologists who can program and use statistics then you're in good shape. There is already so much idle data out there already, that you'll never have to spend a dime on sequencing.

adenadel 11y ago
People say this all the time, but with some of the most common applications of high throughput sequencing there are very good canned solutions (using open source software) that you can pay for. DNAnexus, Seven Bridges, and Illumina BaseSpace all provide cloud storage and analysis. Unless you are doing a custom prep for your sequencing one of these probably has an analysis solution for you.
- BKPetkov 11y ago
  Is the time required to upload data to the cloud ever a problem with these solutions? Of course, it depends on what you are trying to do, but suppose you were working with thousands of genomes?
  - adenadel 11y ago
    Sure, all the time. Network and I/O are the biggest blockers for sequence analysis. For any organization that is working with thousands of genomes they probably have their own compute resources. I know of at least one organization who is currently sending thousands of genomes to the cloud for analysis, so it's certainly feasible to some extent.
    - gbveiuwlbiu 11y ago
      Any chance we could follow up on this? I'm conducting research in this space and would love to have a chat regarding barriers to such large-scale analysis. If you could message me at hngenometemp@forward.cat that would be terrific!
  - epistasis 11y ago
    The sequencers can stream to a data analysis center as the data is being generated. It takes a 100mbit stream/$1M of sequencing capital, so network connectivity to transfer to a data center is a tiny tiny cost of the whole ordeal. However, paying for AWS storage is pretty prohibitive, unless you're at a small scale. So big centers will build their own storage facilities. The small data producers like the ones that the thread author talks about can use often use AWS more cost efficiently than building a compute cluster. However, they need to budget for that, which is not always thought of. They may also need to fight their institute's core center so that they can use DNANexus.
    - laurencerowe 11y ago
      S3 storage is pretty cheap, it's the data egress that really costs. For academic centers though there is often an incentive to move things in house due to different treatment for capital expenditures and the opportunity to externalize some of your costs from your grant onto central services.
      - epistasis 11y ago
        Data transfer is less than a single year of Glacier storage, so while it's pricy I wouldn't egress a major portion of the cost. Keeping this data for less than 5-10 years is pretty questionable, since it's so expensive to generate. Eventually it may be cheaper to store the DNA and resequence when if it needs to be looked at again. However, if you're doing petabytes of storage, it's going to me much more economical to have your own storage and compute than to use AWS. Particularly at the rate that academic centers pay for sysadmins.
        laurencerowe 11y ago
        Running a public data portal our egress is higher than our storage costs. (We now proxy downloads through a direct connect to our university network...) Remember to account for future reductions in storage costs. S3 has come down from $0.1500/GB month in 2010 to $0.0300/GB month today. And the recently introduced infrequent access storage tier is under half that again at $0.0125/GB month. It's now significantly cheaper to use S3/Azure/Google than running the storage ourselves.
- jerven 11y ago
  Analysis is custom if you are doing genomics right at this time. Either you are doing something pre-clinical, and you better understand what you are doing i.e. analysis in a black box is not ok. Or you are not dealing with human stuff at all and then there is no analysis in a box on the market. There are tools with GUI's but they don't help that much if you don't understand the underlying systems.
  - adenadel 11y ago
    I don't agree with you. I would argue that Genomics England is doing genomics right with the 100k genomes project and they certainly aren't doing custom analysis. Their pipeline is standard through variant calling and annotation. The WGS pipelines are pretty well established at this time. What you do with your VCFs is what can be customized. There is no reason for a small lab to write their own aligner or variant caller. The most that many do is write wrappers to create "customized pipelines", which is really just reinventing the wheel. EDIT: I want to clarify my mindset. In the environment that I work in there are three stages of analysis: primary, secondary, and tertiary. We mostly ignore the tertiary stage (I'm not actually doing science). Primary analysis is on instrument (base calling), secondary analysis takes you through variant calls and annotation, and tertiary analysis is where the real science happens. When I'm talking about canned analyses I'm talking about what I refer to as secondary analysis. Tertiary analysis is a very hard problem and will often require custom solutions. In my mind I'm separating bioinformatic analysis (secondary analysis) with interpretation (tertiary analysis).
    - jerven 11y ago
      Getting the variants out is where the analysis starts. What does the variation mean, which ones correlate with a detectable phenotype. Which ones are affecting regulatory systems. Genomics England is brilliant because it has good access to excellent NHS data which it will use to do a lot of analysis. Which its not getting off the shelf but a lot of talented clincal+bio+informaticians are working hard on that. Or in other words getting the variants to the human genome reference is just the start of the analysis not the end. i.e. I am more worried about the end of the pipeline than the contents of them. I suspected as much on the 1,2,3 level. But even 2 is not really fixed yet. There are great off the shelf tools but even the concept of variation calling against a reference linear genome is not perfect yet especially with the alt_loci business. Which driving the variation graph work in the GA4GH projects.
- searine 11y ago
  >DNAnexus, Seven Bridges, and Illumina BaseSpace all provide cloud storage and analysis. "Analysis". There is a difference between meaningful analysis which answers long standing questions of biology, and running a program which calls SNPs and spits out genome stats. The hard part is knowing what questions to ask, and how to interpret the answers when you get them. That takes years of training to develop a critical mind.
  - rcthompson 11y ago
    This is absolutely true. I've done differential expression analysis on many RNA-seq and microarray datasets, and despite the fact that the basic pipeline for such analysis is quite straightforward and easily automated, every analysis is asking different biological questions and has different confounding factors that need to be addressed, so the analysis of each dataset ends up being quite different from all the others.
- dikdik 11y ago
  All of these will give you some answers. The reality is there is SO MUCH MORE INFORMATION to be gleaned from our genomes that these solutions do not account for.
arca_vorago 11y ago
Having worked in a genetics lab as a sysadmin, I can agree with this 100%.
mbreese 11y ago
You know... I'm not so sure I completely agree. I mean, yes, the analysis is the bottleneck. Since about 2012 it takes more time to perform the analysis than it does to generate the data. And even though all of the raw data -> BAM -> VCF or counts steps is automated, nothing after that is (or can be, since that's where the science happens). But, the $1k genome really is significant. Costs matter. Researchers will only sequence what they can afford. So, when sequencing a genome was $10k, there weren't too many 30X whole genomes sequenced. Instead we had exome sequencing of individuals or low-depth sequencing of populations. Fast forward a few years, and now the costs have gone down significantly. Because of this, we are now starting to see more whole-genomes. Now, the costs of to sequence a whole-genome at 30X is about the same as targeted exome sequencing (due to the extra capture costs). If it's almost cheaper to just sequence the whole-genome and skip any extra capture probe biases, why not just sequence the whole thing? This is particularly true when looking at complex diseases where the underlying cause(s) may not be in the coding part of a gene. So, in that regard, cost really was a bottleneck. With all of the new whole-genomes available, we can start to ask even tougher questions. And with more data available from more people, we can start to get some real confidence behind our predictions. But, you are right in that it does all lead back to the data-analysis bottleneck. Before we had as more data than we could deal with. Now, we can process data faster, so that's good. But we also have significantly more data to process, so in the end, it's about even. The other issue then becomes data storage. What do you do when you have a few hundred terabytes or a petabyte of sequencing data? S3 can get pretty expensive at that scale. Instead, it's probably just cheaper to store the DNA in a freezer, and if you ever need the data - just resequence it. Edit: reworded a bit
- jghn 11y ago
  Compute time and storage costs are also bottlenecks, so even though the base calling through VCF is automated large genome centers struggle to keep up with the output of the new sequencers
  - gbveiuwlbiu 11y ago
    When you say compute time, do you have in mind purely quantitative processing/analytics, or streaming and data transfer? Is there any way you could describe a sample "experiment" and the associated storage costs and compute time, just a paper-and-pencil estimation perhaps?

by searine 11y ago

adenadel 11y ago

People say this all the time, but with some of the most common applications of high throughput sequencing there are very good canned solutions (using open source software) that you can pay for. DNAnexus, Seven Bridges, and Illumina BaseSpace all provide cloud storage and analysis. Unless you are doing a custom prep for your sequencing one of these probably has an analysis solution for you.

BKPetkov 11y ago
Is the time required to upload data to the cloud ever a problem with these solutions? Of course, it depends on what you are trying to do, but suppose you were working with thousands of genomes?
- adenadel 11y ago
  Sure, all the time. Network and I/O are the biggest blockers for sequence analysis. For any organization that is working with thousands of genomes they probably have their own compute resources. I know of at least one organization who is currently sending thousands of genomes to the cloud for analysis, so it's certainly feasible to some extent.
  - gbveiuwlbiu 11y ago
    Any chance we could follow up on this? I'm conducting research in this space and would love to have a chat regarding barriers to such large-scale analysis. If you could message me at hngenometemp@forward.cat that would be terrific!
- epistasis 11y ago
  The sequencers can stream to a data analysis center as the data is being generated. It takes a 100mbit stream/$1M of sequencing capital, so network connectivity to transfer to a data center is a tiny tiny cost of the whole ordeal. However, paying for AWS storage is pretty prohibitive, unless you're at a small scale. So big centers will build their own storage facilities. The small data producers like the ones that the thread author talks about can use often use AWS more cost efficiently than building a compute cluster. However, they need to budget for that, which is not always thought of. They may also need to fight their institute's core center so that they can use DNANexus.
  - laurencerowe 11y ago
    S3 storage is pretty cheap, it's the data egress that really costs. For academic centers though there is often an incentive to move things in house due to different treatment for capital expenditures and the opportunity to externalize some of your costs from your grant onto central services.
    - epistasis 11y ago
      Data transfer is less than a single year of Glacier storage, so while it's pricy I wouldn't egress a major portion of the cost. Keeping this data for less than 5-10 years is pretty questionable, since it's so expensive to generate. Eventually it may be cheaper to store the DNA and resequence when if it needs to be looked at again. However, if you're doing petabytes of storage, it's going to me much more economical to have your own storage and compute than to use AWS. Particularly at the rate that academic centers pay for sysadmins.
      - laurencerowe 11y ago
        Running a public data portal our egress is higher than our storage costs. (We now proxy downloads through a direct connect to our university network...) Remember to account for future reductions in storage costs. S3 has come down from $0.1500/GB month in 2010 to $0.0300/GB month today. And the recently introduced infrequent access storage tier is under half that again at $0.0125/GB month. It's now significantly cheaper to use S3/Azure/Google than running the storage ourselves.
jerven 11y ago
Analysis is custom if you are doing genomics right at this time. Either you are doing something pre-clinical, and you better understand what you are doing i.e. analysis in a black box is not ok. Or you are not dealing with human stuff at all and then there is no analysis in a box on the market. There are tools with GUI's but they don't help that much if you don't understand the underlying systems.
- adenadel 11y ago
  I don't agree with you. I would argue that Genomics England is doing genomics right with the 100k genomes project and they certainly aren't doing custom analysis. Their pipeline is standard through variant calling and annotation. The WGS pipelines are pretty well established at this time. What you do with your VCFs is what can be customized. There is no reason for a small lab to write their own aligner or variant caller. The most that many do is write wrappers to create "customized pipelines", which is really just reinventing the wheel. EDIT: I want to clarify my mindset. In the environment that I work in there are three stages of analysis: primary, secondary, and tertiary. We mostly ignore the tertiary stage (I'm not actually doing science). Primary analysis is on instrument (base calling), secondary analysis takes you through variant calls and annotation, and tertiary analysis is where the real science happens. When I'm talking about canned analyses I'm talking about what I refer to as secondary analysis. Tertiary analysis is a very hard problem and will often require custom solutions. In my mind I'm separating bioinformatic analysis (secondary analysis) with interpretation (tertiary analysis).
  - jerven 11y ago
    Getting the variants out is where the analysis starts. What does the variation mean, which ones correlate with a detectable phenotype. Which ones are affecting regulatory systems. Genomics England is brilliant because it has good access to excellent NHS data which it will use to do a lot of analysis. Which its not getting off the shelf but a lot of talented clincal+bio+informaticians are working hard on that. Or in other words getting the variants to the human genome reference is just the start of the analysis not the end. i.e. I am more worried about the end of the pipeline than the contents of them. I suspected as much on the 1,2,3 level. But even 2 is not really fixed yet. There are great off the shelf tools but even the concept of variation calling against a reference linear genome is not perfect yet especially with the alt_loci business. Which driving the variation graph work in the GA4GH projects.
searine 11y ago
>DNAnexus, Seven Bridges, and Illumina BaseSpace all provide cloud storage and analysis. "Analysis". There is a difference between meaningful analysis which answers long standing questions of biology, and running a program which calls SNPs and spits out genome stats. The hard part is knowing what questions to ask, and how to interpret the answers when you get them. That takes years of training to develop a critical mind.
- rcthompson 11y ago
  This is absolutely true. I've done differential expression analysis on many RNA-seq and microarray datasets, and despite the fact that the basic pipeline for such analysis is quite straightforward and easily automated, every analysis is asking different biological questions and has different confounding factors that need to be addressed, so the analysis of each dataset ends up being quite different from all the others.
dikdik 11y ago
All of these will give you some answers. The reality is there is SO MUCH MORE INFORMATION to be gleaned from our genomes that these solutions do not account for.

arca_vorago 11y ago

Having worked in a genetics lab as a sysadmin, I can agree with this 100%.

mbreese 11y ago

You know... I'm not so sure I completely agree. I mean, yes, the analysis is the bottleneck. Since about 2012 it takes more time to perform the analysis than it does to generate the data. And even though all of the raw data -> BAM -> VCF or counts steps is automated, nothing after that is (or can be, since that's where the science happens). But, the $1k genome really is significant. Costs matter. Researchers will only sequence what they can afford. So, when sequencing a genome was $10k, there weren't too many 30X whole genomes sequenced. Instead we had exome sequencing of individuals or low-depth sequencing of populations. Fast forward a few years, and now the costs have gone down significantly. Because of this, we are now starting to see more whole-genomes. Now, the costs of to sequence a whole-genome at 30X is about the same as targeted exome sequencing (due to the extra capture costs). If it's almost cheaper to just sequence the whole-genome and skip any extra capture probe biases, why not just sequence the whole thing? This is particularly true when looking at complex diseases where the underlying cause(s) may not be in the coding part of a gene. So, in that regard, cost really was a bottleneck. With all of the new whole-genomes available, we can start to ask even tougher questions. And with more data available from more people, we can start to get some real confidence behind our predictions. But, you are right in that it does all lead back to the data-analysis bottleneck. Before we had as more data than we could deal with. Now, we can process data faster, so that's good. But we also have significantly more data to process, so in the end, it's about even. The other issue then becomes data storage. What do you do when you have a few hundred terabytes or a petabyte of sequencing data? S3 can get pretty expensive at that scale. Instead, it's probably just cheaper to store the DNA in a freezer, and if you ever need the data - just resequence it. Edit: reworded a bit

jghn 11y ago
Compute time and storage costs are also bottlenecks, so even though the base calling through VCF is automated large genome centers struggle to keep up with the output of the new sequencers
- gbveiuwlbiu 11y ago
  When you say compute time, do you have in mind purely quantitative processing/analytics, or streaming and data transfer? Is there any way you could describe a sample "experiment" and the associated storage costs and compute time, just a paper-and-pencil estimation perhaps?