5 ms·

What is the opportunity here -- writing new algorithms, implementing them accurately, optimizing them for special execution architectures, or just building more

What is the opportunity here -- writing new algorithms, implementing them accurately, optimizing them for special execution architectures, or just building more usable tools? I remember Manolis Kellis sprinkled some pretty interesting genomic questions into his Algorithm class's problem sets. There were a number of cool problems about optimally aligning strings, searching within text, etc. This was like 15 years ago and I haven't kept up with the discipline at all. But is there still algorithmic low hanging fruit? I do keep reading about an ongoing series of problems with Microsoft Excel distorting analysis in the scientific literature (https://www.nature.com/articles/d41586-021-02211-4 https://www.nature.com/articles/d41586-021-02211-4) and wondering if the tooling is having trouble..?

boldlybold 4y ago
We're only starting to see the age of genomics accelerated by GPUs. I think it's still early if you have the technical background.
- 331c8c71 4y ago
  Edico developed FPGA-based processing solution for common bioinformatics processing tasks (e.g. dna/rna mapping, variant calling) and the company was bought by Illumina. The product (Dragen) has been around for a few years and now will be integrated in the new generation of sequencers. Extremely impressive technology and a better fit for the niche compared to GPU-based solutions I have seen. More downstream processing and analytics is sometimes closer to traditional ML and naturally there are lots of GPU-based algos.
  - boldlybold 4y ago
    I'm more excited about NVIDIA's acquisition of Parabricks and the version 4.0 of the software that makes it free to use, than I am about DRAGEN. At the very least it's good to have some competition in the space, Illumina's stuff is always SO expensive. We'll have to see what hardware will win in the end.
    - 331c8c71 4y ago
      I have tried both and dragen was more polished and also faster (that depends on the GPU for parabricks of course). Also more features and they keep adding them. Agreed that competition is good to have. There is also Sentieon and similar solutions which run on common hardware but are optimized. Speaking of costs (both upfront and licensing), dragen imo is not expensive relative to sequencing costs (e.g. sequencers and flowcells). Surely it would be expensive to buy for occasional use. By making parabricks free to use Nvidia tries to gain a market share I guess. In professional settings you still end up buying support and likely dedicated hardware (which is comparable in pricing). Good fit for the cloud and research environments that already have access to GPUs and/or are decoupled from actual sequencing.
331c8c71 4y ago
> But is there still algorithmic low hanging fruit? I would say no unless looking at the frontiers of what is done in the wet lab which might require new analytical tools. But this stuff is probably much easier for and much better aligned with someone doing CS in academia. My impression that there is quite some space for ML-based approaches including DL. But even there I would not call it low-hanging.
jltsiren 4y ago
> But is there still algorithmic low hanging fruit? Algorithmic bioinformatics has become a separate research field, because there are so many low-hanging fruit. Biotech companies create new instruments producing new kinds of data, researchers find new uses for the data, and new algorithmic problems emerge all the time. There is also a steady migration of people from theoretical computer science to bioinformatics, because it's often easier to get research funding for something bioinformatics-related than for pure CS.

boldlybold 4y ago

We're only starting to see the age of genomics accelerated by GPUs. I think it's still early if you have the technical background.

331c8c71 4y ago
Edico developed FPGA-based processing solution for common bioinformatics processing tasks (e.g. dna/rna mapping, variant calling) and the company was bought by Illumina. The product (Dragen) has been around for a few years and now will be integrated in the new generation of sequencers. Extremely impressive technology and a better fit for the niche compared to GPU-based solutions I have seen. More downstream processing and analytics is sometimes closer to traditional ML and naturally there are lots of GPU-based algos.
- boldlybold 4y ago
  I'm more excited about NVIDIA's acquisition of Parabricks and the version 4.0 of the software that makes it free to use, than I am about DRAGEN. At the very least it's good to have some competition in the space, Illumina's stuff is always SO expensive. We'll have to see what hardware will win in the end.
  - 331c8c71 4y ago
    I have tried both and dragen was more polished and also faster (that depends on the GPU for parabricks of course). Also more features and they keep adding them. Agreed that competition is good to have. There is also Sentieon and similar solutions which run on common hardware but are optimized. Speaking of costs (both upfront and licensing), dragen imo is not expensive relative to sequencing costs (e.g. sequencers and flowcells). Surely it would be expensive to buy for occasional use. By making parabricks free to use Nvidia tries to gain a market share I guess. In professional settings you still end up buying support and likely dedicated hardware (which is comparable in pricing). Good fit for the cloud and research environments that already have access to GPUs and/or are decoupled from actual sequencing.

331c8c71 4y ago

> But is there still algorithmic low hanging fruit? I would say no unless looking at the frontiers of what is done in the wet lab which might require new analytical tools. But this stuff is probably much easier for and much better aligned with someone doing CS in academia. My impression that there is quite some space for ML-based approaches including DL. But even there I would not call it low-hanging.

jltsiren 4y ago

> But is there still algorithmic low hanging fruit? Algorithmic bioinformatics has become a separate research field, because there are so many low-hanging fruit. Biotech companies create new instruments producing new kinds of data, researchers find new uses for the data, and new algorithmic problems emerge all the time. There is also a steady migration of people from theoretical computer science to bioinformatics, because it's often easier to get research funding for something bioinformatics-related than for pure CS.