4 ms·
There are. Data types are pretty much standardized on the basic level (eg. fastq, bam, bed, vcf etc.). Else, most is done by the big science consortia like TCGA
by patall 10y ago
There are. Data types are pretty much standardized on the basic level (eg. fastq, bam, bed, vcf etc.). Else, most is done by the big science consortia like TCGA, ICGC & PanCancer. And obviously the big databases from NCBI and EBI-EMBL. Trials are standardized by EMA and FDA.
- dekhn 10y agoFASTQ, BAM, BED, VCF, etc are all standardized in terms of file format, but the semantics of the content are not. Harmonizing data across multiple semantics is still a very challenging problem that few people take seriously.
- patall 10y agoLike the good old "chr1" == "1" problem? Well, yeah that’s true. But still I think we have those consortia in place, why do we need another authority. They just have to set (and somewhat enforce) the rules.
- dekhn 10y agoIt goes far beyond "chr1" = "1". There are many other fields with next-to-impossible to interpret data. The consortia have, at best, advisory roles, they cannot enforce any rules. And given an extensible file format, third parties are going to extend it with poorly defined/standardized data. Then the consumers have to harmonize the data. Data harmonization (or cleaning) it typically one of the bottleneck steps in applying machine learning at scale.