4 ms·
There seems to be confusion in this thread on what exactly is stored. The human genome is about 3.25 billion bases, or about 6.5 billion bases if you determine
by evandijk70 8y ago
There seems to be confusion in this thread on what exactly is stored.
The human genome is about 3.25 billion bases, or about 6.5 billion bases if you determine both copies of each chromosome. This corresponds to 13 billion bits or about 1.5 Gigabytes, and storing this would be no problem at all, even if there would be a lot of genomes sequenced.
However, the way human genomes are sequenced is different: The genome is fragmented in random pieces of around 500 basepairs long. Illumina sequencing machines typically read 300 basepairs of these fragments with a ~1% error rate.
To be able to determine the entire genome, these reads are mapped to a reference genome, and differences with the reference genome are marked as mutations. However, to distinguish real mutations from random read errors, and to make sure that mutations can be confidently called for regions that where fewer reads mapped a lower coverage through random variation, a ~30x coverage of the genome is needed.
If you want to store the raw sequencing data and not just the called results, a single genome does not require 1.5 GB of storage, but now requires 45 GB storage. It gets even worse when you consider samples from tumours, where biologically relevant mutations are often found in only a subset of cells. To discover these mutations, a coverage of 100x is often recommend, leading to 150 GB of storage for a single genome.
- mmt 8y ago> It gets even worse when you consider samples from tumours, where biologically relevant mutations are often found in only a subset of cells. To discover these mutations, a coverage of 100x is often recommend This point brings up another, implicit, scale multiplier that the average person might assume doesn't exist: that a single individual could need multiple sequences stored. One can't just estimate required total (uncompressed) storage by multiplying population (mod monozygotic multiple births and chimerism) by the size of one instance of raw sequencing data.
- breckuh 8y ago> a single individual could need multiple sequences stored. And multiple sequences could be hundreds, thousands, or more. Example: single-cell RNAseq pipelines that generate sequences for 10k cells at once. Perhaps in the future--as tech improves--the devices themselves might be equipped to run more real-time in memory QC and emit less--but more accurate--data? I believe this is somewhat how CERN does it, where they only record a filtered subset of the data that comes in.
- bane 8y agoYou're also forgetting that most of the popular formats just store the bases as 1-byte per base "GATC"= 4 bytes, when 2 bits per base would be fine if your data only has the 4 unambiguous bases. You could fairly trivially drop the storage of most of these formats by just encoding as 2 bits per base. Another weird thing is that not all possible k-mers appear in DNA and k-mers of most lengths will repeat somewhere at least once. It would probably be best to just pick some useful k-mer length and build a lookup table and just store the byte-encoded indexes that map to the kmers then just gzip that whole mess. You might be able to save more space if it turns out that the indexes turn out to be reusable across genomes and just leave them out of the file. It may also depend on your application, if all you want to know is "does k-mer==GATTACA" exist in my genome, you can just dump all the kmers into some persistent bloom filter and do searches and lookups really fast. Those would be quite small compared.
- ur-whale 8y agoSort of makes you wonder why the word "sequencing" is used to describe the procedure. Sounds more like throwing the vase on the floor and carefully picking up the pieces.
- fapjacks 8y agoThat's quite an apt metaphor. Except you do it thirty times in order to make sure you're putting them back together correctly.