4 ms·
Has anyone trained a transformer model yet on genotype phenotype mapping? Like feed in loads of human, animal, etc sequences and get it to predict phenotype as
by api 3y ago
Has anyone trained a transformer model yet on genotype phenotype mapping? Like feed in loads of human, animal, etc sequences and get it to predict phenotype as accurately as possible.
This seems like it could be the key to genetic engineering. You could probably build something analogous to a diffusion model for organisms that renders genomes or fine tunes them from phenotypic traits.
- hackpert 3y agoI tried to do this ages ago in 2018 by adapting OpenAI's flow architecture and it sort of seemed to work (was at least promising). With today's models with a significantly more disentangled latent space it should be much easier to do! I saw a transformer trained on the UK Biobank recently, excited for this space!
- pfisherman 3y agoNo. In general there are many orders of magnitude more variations throughout the genome than there are observable phenotypes. We don’t know what genes most cis regulatory elements regulate; and we don’t know what most genes do in which contexts. Hell, we don’t even know how many different types of cells there are in the human body.
- mrtesthah 3y agoIf such a thing were possible, it might involve some of the techniques from this paper: HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution https://arxiv.org/abs/2306.15794 https://arxiv.org/abs/2306.15794
- hackerlight 3y agoThe problem is data scarcity. You're stuck with a few million rows of data, which just isn't enough given how noisy and subtle the genotype->phenotype relationships are. Neural nets work well on language, Chess, Go, etc, because we have gigantic datasets where the subtle patterns can be teased out without overfitting.