4 ms·
Who would implement this suggestion? Is it an appeal to folks at large writing tools that interact with genomics data? Due to higher decompression cost, the o
by evrydayhustling 4y ago
Who would implement this suggestion? Is it an appeal to folks at large writing tools that interact with genomics data?
Due to higher decompression cost, the opportunity here seems localized to long-term storage. It feels like it would make more sense as a project (or product!) that implemented an efficient long-term archive (perhaps with a less compressed LRU cache in front).
- michaelbarton 4y agoYes exactly. If the most common tools supported zstd then it would make it easier to store zstd FASTQ. At the moment I believe almost none of them do.
- asdff 4y agoFor those toolmakers they would probably say something like "our tool supports uncompressed data specifically so you can add whatever compression you want in your pipeline for your use cases."
- chrchang523 4y agoIt addresses different bioinformatic use cases, but plink2 is an open-source tool that is widely used and makes extensive use of zstd. In particular, it accepts both gzip- and zstd-compressed text files as input, but output commands that can generate large text files often only have a zstd compression option.