5 ms·

Can LLMs read the genome? This one decoded mRNA to make better vaccines

4 points by westurner 3y ago

westurner 3y ago
"A 5′ UTR language model for decoding untranslated regions of mRNA and function predictions" https://www.nature.com/articles/s42256-024-00823-9 https://www.nature.com/articles/s42256-024-00823-9
- westurner 3y ago
  "Language models can explain neurons in language models"; GPT-4 can help explain what GPT-2 neurons do: https://openai.com/research/language-models-can-explain-neurons-in-language-models https://openai.com/research/language-models-can-explain-neur... https://news.ycombinator.com/item?id=35877402 https://news.ycombinator.com/item?id=35877402 IDK if this is a practical limit to such research? From https://news.ycombinator.com/item?id=38753725 https://news.ycombinator.com/item?id=38753725 : >> "Challenges with unsupervised LLM knowledge discovery" (2023) https://arxiv.org/abs/2312.10029 https://arxiv.org/abs/2312.10029 : >> Abstract: We show that existing unsupervised methods on large language model (LLM) activations do not discover knowledge -- instead they seem to discover whatever feature of the activations is most prominent
syndicatedjelly 3y ago
Does it hinder or help our understanding of AI by referring to everything an AI does with human verbs, such as "read"? Animals don't read, yet they have eyes. Why do we say that an AI does?
reify 3y ago
There seems to be two questions here: Can language models read the genome? Quite possibly. This one decoded mRNA to make better vaccines? I doubt it very much. I have recently used Open-whisper and Faster-whisper to translate the French audio of a TV series to English subtitles. I downloaded and tried all available smal and large models on my gaming rig. Whisper is an automatic speech recognition (ASR) system trained on 680,000 hours of multilingual and multitask supervised data collected from the web. You would think this was a simple task for todays super duper AI. Fraidy not!! Accuracy is poor. If AI cannot get simple translation correct what chance do these chancers have. This is also worrying because the W.H.O changed the definition of vaccines. So that mRNA drugs fit into that lovely vaccine window. mRNA drugs are not vaccines. This will not be like the vaccines that we have all used for generations with great success. ‘It’s life Jim, but not as we know it’

4 points by westurner 3y ago

"A 5′ UTR language model for decoding untranslated regions of mRNA and function predictions" https://www.nature.com/articles/s42256-024-00823-9 https://www.nature.com/articles/s42256-024-00823-9

westurner 3y ago
"Language models can explain neurons in language models"; GPT-4 can help explain what GPT-2 neurons do: https://openai.com/research/language-models-can-explain-neurons-in-language-models https://openai.com/research/language-models-can-explain-neur... https://news.ycombinator.com/item?id=35877402 https://news.ycombinator.com/item?id=35877402 IDK if this is a practical limit to such research? From https://news.ycombinator.com/item?id=38753725 https://news.ycombinator.com/item?id=38753725 : >> "Challenges with unsupervised LLM knowledge discovery" (2023) https://arxiv.org/abs/2312.10029 https://arxiv.org/abs/2312.10029 : >> Abstract: We show that existing unsupervised methods on large language model (LLM) activations do not discover knowledge -- instead they seem to discover whatever feature of the activations is most prominent

syndicatedjelly 3y ago

Does it hinder or help our understanding of AI by referring to everything an AI does with human verbs, such as "read"? Animals don't read, yet they have eyes. Why do we say that an AI does?

reify 3y ago

There seems to be two questions here: Can language models read the genome? Quite possibly. This one decoded mRNA to make better vaccines? I doubt it very much. I have recently used Open-whisper and Faster-whisper to translate the French audio of a TV series to English subtitles. I downloaded and tried all available smal and large models on my gaming rig. Whisper is an automatic speech recognition (ASR) system trained on 680,000 hours of multilingual and multitask supervised data collected from the web. You would think this was a simple task for todays super duper AI. Fraidy not!! Accuracy is poor. If AI cannot get simple translation correct what chance do these chancers have. This is also worrying because the W.H.O changed the definition of vaccines. So that mRNA drugs fit into that lovely vaccine window. mRNA drugs are not vaccines. This will not be like the vaccines that we have all used for generations with great success. ‘It’s life Jim, but not as we know it’