4 ms·

> The long read technology comes with a drawback though – high error rates in sequencing of between 15%-40% These error rate estimates are seriously outdated (

by klmr 9y ago

> The long read technology comes with a drawback though – high error rates in sequencing of between 15%-40% These error rate estimates are seriously outdated (the quoted number is from a 2015 paper but it was obsolete pretty much as soon as that paper was published). Long-read technologies are evolving rapidly, so this is important. The state of the art is working with error rates of at most 15% (but usually much less), which, after correction, go down below 8% [1]. This is crucial because it makes the difference between a successful and a failed assembly: 40% error rate essentially means that you need different algorithms to perform sequence assembly (as the article notes, error correction takes up “orders of magnitude” more time than actual assembly). I’d therefore be curious how this Darwin setup performs against conventional, state-of-the-art sequence assembly with state-of-the-art long read sequencing data. [1] https://www.nature.com/articles/nbt.4060 https://www.nature.com/articles/nbt.4060

azeotropic 9y ago
In general, error correction requires alignment, so the correct comparison is error correction using Darwin for alignment vs. error correction using whatever long-read aligner you consider state-of-the-art.
stochastic_monk 9y ago
Read page 8 [206] of this paper to see the numbers they're providing. The PacBio reads are listed at up to 15% error rate. Their experiments were synthetic data from 15-40% error rates. A few comments: 1. This was published in ASPLOS, not a comp bio journal, which should raise some eyebrows. 2. As another commenter said, error correction requires a reference, either by alignment, kmer filtering, or consensus. However, this is usually part of any assembly pipeline. 3. Like you said, they failed to compare to actual state of the art alignment tools. They do cite Canu, but they don't use it as their baseline. The issue here is that the current best long read assemblers use locality-sensitive hashing. The Canu paper even states that miniasm can produce a higher-quality assembly for CHM1 in 1/400 the CPU time (though their discussion of the table selectively ignores this fact). Considering that a combination of algorithm and implementation is able to provide a 400x speedup makes this result seem significantly less impressive, and that a comparison of CPU time is slightly unfair to miniasm considering how perfectly it threadscales, which would make the other tools look even worse by comparison. Additionally, for problems where you really need the improvement in performance, the GPU RAM limitations and communication overhead would likely incur significant penalties to their method. It's not simply a matter of how many reads a second you can process because it's a quadratic overlap problem.
- klmr 9y ago
  Their 40% number comes from the paper I referenced (i.e. that’s the one the Darwin paper references). I completely agree with your point 2. I’m not sure I agree with your first point: I think it’s pretty reasonable for it to have been published in that venue.

by klmr 9y ago

azeotropic 9y ago

In general, error correction requires alignment, so the correct comparison is error correction using Darwin for alignment vs. error correction using whatever long-read aligner you consider state-of-the-art.

stochastic_monk 9y ago

Read page 8 [206] of this paper to see the numbers they're providing. The PacBio reads are listed at up to 15% error rate. Their experiments were synthetic data from 15-40% error rates. A few comments: 1. This was published in ASPLOS, not a comp bio journal, which should raise some eyebrows. 2. As another commenter said, error correction requires a reference, either by alignment, kmer filtering, or consensus. However, this is usually part of any assembly pipeline. 3. Like you said, they failed to compare to actual state of the art alignment tools. They do cite Canu, but they don't use it as their baseline. The issue here is that the current best long read assemblers use locality-sensitive hashing. The Canu paper even states that miniasm can produce a higher-quality assembly for CHM1 in 1/400 the CPU time (though their discussion of the table selectively ignores this fact). Considering that a combination of algorithm and implementation is able to provide a 400x speedup makes this result seem significantly less impressive, and that a comparison of CPU time is slightly unfair to miniasm considering how perfectly it threadscales, which would make the other tools look even worse by comparison. Additionally, for problems where you really need the improvement in performance, the GPU RAM limitations and communication overhead would likely incur significant penalties to their method. It's not simply a matter of how many reads a second you can process because it's a quadratic overlap problem.

klmr 9y ago
Their 40% number comes from the paper I referenced (i.e. that’s the one the Darwin paper references). I completely agree with your point 2. I’m not sure I agree with your first point: I think it’s pretty reasonable for it to have been published in that venue.