4 ms·
Reference compression is interesting... technically, this type of compression should be possible without a reference. E.g. in a worst case scenario, you do a ge
by _yb2s 4y ago
Reference compression is interesting... technically, this type of compression should be possible without a reference. E.g. in a worst case scenario, you do a genome assembly from the reads, and then compress against that reference.
Fascinating to think that in principle, genome assembly and fastq file compression have many parallels, and are essentially the same problem. For example, once you assemble the genome, you can just track the coordinates and 'diff' for each read and the original FASTQ file can be regenerated exactly.
- bnprks 4y agoOne recent method that does this is SPRING: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6662292/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6662292/ Although I'm not familiar with all the details of the method, my understanding is that the reference used by compression can be extremely low quality for biological purposes while still providing good compression -- making it possible to make a quick-and-dirty reference with much less computational time.
- v8xi 4y agoThat would be true for FASTA compression but FASTQ files contain other data. With a MiSeq, for example, 1/3 of the data is the DNA sequences, 1/3 is a quality score (0-37) and 1/3 is a header which includes things like flow cell ID but also a lot of info on the physical registration address of the cluster on the flow cell. In some applications this info is not required, but is from a scientific/data integrity perspective