2 ms·
It's necessary to distinguish between the raw output of the sequencing instrument and the actual genomic variables of interest here. Adequate compression of th
by chrchang523 8y ago
It's necessary to distinguish between the raw output of the sequencing instrument and the actual genomic variables of interest here.
Adequate compression of the latter, with appropriate confidence estimates attached, is a solved problem for the most common applications, even when hundreds of thousands of samples are involved: the UK Biobank has released a dataset describing whole-genome variation for almost 500000 British people, and many research groups around the world are successfully analyzing this data today with freely available software running on commodity hardware.
It's the raw measurements that aren't compressing as well as we'd want today. But there are multiple other solutions to this problem besides better compression: we can become good enough at inferring the relevant genomic variables from the measurements that, once we've done so, we can discard the raw data (we don't do this today because the inference process is still very much a work in progress, so the option of running tomorrow's algorithms on today's measurements is too valuable); or measurement becomes cheap enough and biological sample storage stable enough that the measurement process can simply be repeated when necessary; or entirely different measurement process(es) are developed that yield adequate information without simultaneously generating so much nuisance data to preserve.