4 ms·

It's great that we have all these research teams tackling the cancer problem from a machine-learning/genomics/personalised-medicine angle... But do they have a

by joeyspn 10y ago

It's great that we have all these research teams tackling the cancer problem from a machine-learning/genomics/personalised-medicine angle... But do they have a common/open standard for technologies (ML, CV, etc), sharing datasets and findings, trials, etc? IMO something like HL7 [0] but for cancer research would be a game-changer. I see a lot of promising approaches but not so good coordination, with everybody working in silos and developing their own framework... or maybe it's what I see on the surface. [0] http://www.hl7.org/ http://www.hl7.org/

patall 10y ago
There are. Data types are pretty much standardized on the basic level (eg. fastq, bam, bed, vcf etc.). Else, most is done by the big science consortia like TCGA, ICGC & PanCancer. And obviously the big databases from NCBI and EBI-EMBL. Trials are standardized by EMA and FDA.
- dekhn 10y ago
  FASTQ, BAM, BED, VCF, etc are all standardized in terms of file format, but the semantics of the content are not. Harmonizing data across multiple semantics is still a very challenging problem that few people take seriously.
  - patall 10y ago
    Like the good old "chr1" == "1" problem? Well, yeah that’s true. But still I think we have those consortia in place, why do we need another authority. They just have to set (and somewhat enforce) the rules.
    - dekhn 10y ago
      It goes far beyond "chr1" = "1". There are many other fields with next-to-impossible to interpret data. The consortia have, at best, advisory roles, they cannot enforce any rules. And given an extensible file format, third parties are going to extend it with poorly defined/standardized data. Then the consumers have to harmonize the data. Data harmonization (or cleaning) it typically one of the bottleneck steps in applying machine learning at scale.

4 ms·

It's great that we have all these research teams tackling the cancer problem from a machine-learning/genomics/personalised-medicine angle... But do they have a

by joeyspn 10y ago

patall 10y ago
There are. Data types are pretty much standardized on the basic level (eg. fastq, bam, bed, vcf etc.). Else, most is done by the big science consortia like TCGA, ICGC & PanCancer. And obviously the big databases from NCBI and EBI-EMBL. Trials are standardized by EMA and FDA.
- dekhn 10y ago
  FASTQ, BAM, BED, VCF, etc are all standardized in terms of file format, but the semantics of the content are not. Harmonizing data across multiple semantics is still a very challenging problem that few people take seriously.
  - patall 10y ago
    Like the good old "chr1" == "1" problem? Well, yeah that’s true. But still I think we have those consortia in place, why do we need another authority. They just have to set (and somewhat enforce) the rules.
    - dekhn 10y ago
      It goes far beyond "chr1" = "1". There are many other fields with next-to-impossible to interpret data. The consortia have, at best, advisory roles, they cannot enforce any rules. And given an extensible file format, third parties are going to extend it with poorly defined/standardized data. Then the consumers have to harmonize the data. Data harmonization (or cleaning) it typically one of the bottleneck steps in applying machine learning at scale.