10 ms·
Before my comment gets dismissed, I will disclaim I am a professional structural biologist that works in this field every day. These threads are always the sam
by COGlory 4y ago
Before my comment gets dismissed, I will disclaim I am a professional structural biologist that works in this field every day.
These threads are always the same: lots of comments about protein folding, how amazing DeepMind is, how AlphaFold is a success story, how it has flipped an entire field on it's head, etc. The language from Google is so deceptive about what they've actually done, I think it's actually intentionally disingenuous.
At the end of the day, AlphaFold is amazing homology modeling. I love it, I think it's an awesome application of machine learning, and I use it frequently. But it's doing the same thing we've been doing for 2 decades: pattern matching sequences of proteins with unknown structure to sequences of proteins with known structure, and about 2x as well as we used to be able to.
That's extremely useful, but it's not knowledge of protein folding. It can't predict a fold de novo, it can't predict folds that haven't been seen (EDIT: this is maybe not strictly true, depending on how you slice it), it fails in a number of edge cases (remember, in biology, edge cases are everything) and again, I can't stress this enough, we have no new information on how proteins fold. We know all the information (most of at least) for a proteins final fold is in the sequence. But we don't know much about the in-between.
I like AlphaFold, it's convenient and I use it (although for anything serious or anything interacting with anything else, I still need a real structure), but I feel as though it has been intentionally and deceptively oversold. There are 3-4 other deep learning projects I think have had a much greater impact on my field.
EDIT:
See below:
https://news.ycombinator.com/item?id=32265662 https://news.ycombinator.com/item?id=32265662 for information on predicting new folds.
- mupuff1234 4y ago> There are 3-4 other deep learning projects I think have had a much greater impact on my field. Don't leave us hanging... which projects?
- COGlory 4y ago1) Isonet - takes low SNR cryo-electron tomography images (that are extremely dose limited, so just incredibly blurry and frequently useless) and does two things: * Deconvolutes some image aberrations and "de-noises" the images * Compensates for missing wedge artifacts (missing wedge is the fact that the tomography isn't done -90° --> +90°, but usually instead -60° --> +60°, leaving a 30° wedge on the top and bottom of basically no information) which usually are some sort of directionality in image density. So if you have a sphere, the top and bottom will be extremely noisy and stretched up and down (in Z). https://www.biorxiv.org/content/10.1101/2021.07.17.452128v1 https://www.biorxiv.org/content/10.1101/2021.07.17.452128v1 2) Topaz, but topaz really counts as 2 or 3 different algorithms. Topaz has denoising of tomograms and of flat micrographs (i.e. images taken with a microscope, as opposed to 3D tomogram volumes). That denoising is helpful because it increases contrast (which is the fundamental problem in Cryo-EM for looking at biomolecules). Topaz also has a deep learning particle picker which is good at finding views of your protein that are under-represented, or otherwise missing, which again, normally results in artifacts when you build your 3D structure. https://emgweb.nysbc.org/topaz.html https://emgweb.nysbc.org/topaz.html 3) EMAN2 convolutional neural network for tomogram segmentation/Amira CNN for segmentation/flavor of the week CNN for tomogram segmentation. Basically, we can get a 3D volume of a cell or virus or whatever, but then they are noisy. To do anything worthwhile with it, even after denoising, we have to say "this is cell membrane, this is virus, this is nucleic acid" etc. CNNs have proven to be substantially better at doing this (provided you have an adequate "ground truth") than most users. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5623144/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5623144/
- deleted 4y ago[deleted]
- flobosg 4y ago> AlphaFold is amazing homology modeling If it is homology modelling, then how can it work without input template structures?
- COGlory 4y agoIt has template structures. AlphaFold uses the following databases: BFD, MGnify, PDB70, PDB (structures in the mmCIF format), PDB seqres – only for AlphaFold-Multimer, Uniclust30, UniProt – only for AlphaFold-Multimer, UniRef90.
- flobosg 4y agoThose databases are used to derive the evolutionary couplings and distance matrices used by the algorithm. Several of those databases aren’t even structural ones. Furthermore, AlphaFold can function with only a MSA as an input, without retrieving a single PDB coordinate.
- COGlory 4y agoIt's all about boosting signal by finding other proteins that are similar, until you get to the point that you can identify a fold to assign to a region of the protein. That's why some are structural, and some are not. >Furthermore, AlphaFold can function with only a MSA as an input, without retrieving a single PDB coordinate. Yes, it has a very nice model of what sequences should look like in 3D. That model is derived from experimental data. So if I give AlphaFold an MSA of a new, unknown protein fold (substantively away from any known fold), it cannot predict it.
- flobosg 4y ago> Yes, it has a very nice model of what sequences should look like in 3D. A structural model, you would say. > That model is derived from experimental data. That doesn’t make it a template-based model, or a homology one. > if I give AlphaFold an MSA of a new, unknown protein fold (substantively away from any known fold), it cannot predict it That will depend on the number of effective sequences found to derive couplings. Domains with novel folds usually have a low number of remotely homolog sequences and for that reason the method will fail, not just because they are novel.
- Lloydksk 4y agoI mean like whats this about AlphaFold is gone
- adamsmith143 4y ago> it can't predict folds that haven't been seen This seems strange to me. The entire point of these types of models is to predict things on unseen data. Are you saying Deepmind is completely lying about their model? Deepmind solved CASP, isn't the entire point of that competition to predict unseen structures? If AlphaFold doesn't predict anything then what are you using it to do?
- COGlory 4y agoAlphaFold figures out that my input sequence (which has no structural data) is similar to this other protein that has structural data. Or maybe different parts of different proteins. It does this extremely well.
- flobosg 4y agoThis is a gross misrepresentation of the method.
- COGlory 4y agoPerhaps you'd care to explain how? AlphaFold does not work on new folds. It ultimately relies on mapping sequence to structure. It does it better than anyone else, and in ways a human probably couldn't, but if you give it a brand new fold with no relation to other folds, it cannot predict it. I routinely areas of extremely low confidence many of my AlphaFold models. I work in organisms that have virtually 0 sequence identity. This is a problem I deal with every day. I wish AlphaFold worked in the way you are suggesting, but it just flat out does not.
- flobosg 4y ago> It ultimately relies on mapping sequence to structure. So does every structural prediction method. > if you give it a brand new fold with no relation to other folds, it cannot predict it That will depend on the number of effective sequences, not the actual fold. > I work in organisms that have virtually 0 sequence identity. Then the problem is low sequence coverage, not the protein fold. On a side note, there are sensitive homology search protocols that rely very little on actual sequence identity.
- deleted 4y ago[deleted]
- dekhn 4y agoI've directly communicated with the leaders of CASP and at DM that they should stop representing this as a form of protein folding and just call it "crystal/cryoEM structure prediction" (they filter out all the NMR structures from PDB since they aren't good for prediction). They know it's disingenuous and they do it on purpose to give it more impact than it really deserves. I would like to correct somethign here- it does predict structures de novo and predict folds that haven't been seen before. That's because of the design of the NN- it uses sequence information to create structural constraints. If those constraints push the modeller in the direction of a novel fold, it will predict that. To me what's important about this is that it demonstrated the obvious (I predicted this would happen eventually, shortly after losing CASP in 2000).
- COGlory 4y ago>I would like to correct somethign here- it does predict structures de novo and predict folds that haven't been seen before. That's because of the design of the NN- it uses sequence information to create structural constraints. If those constraints push the modeller in the direction of a novel fold, it will predict that. Could you expand on this? Basically it looks at the data, and figures out what's an acceptable position in 3D space for residues to occupy, based on what's known about other structure? I will update my original post to point out I may be not entirely correct there. The distinction I'm trying to make is that there's a difference between looking at pre-existing data and modeling (ultimately homology modeling, but maybe slightly different) and understanding how protein folding works, being able to predict de novo how an amino acid sequence will become a 3D structure. Also thank you for contacting CASP about this.
- dekhn 4y agoFrom what I can tell, the model DM built is mining subtle relationships between aligned columns of multiple sequence alignments and any structural information which is tangibly related to those sequences. Those relationships can be used to infer rough atomic distances ("this atom should be within 3 and 7 angstroms of this other atom"). A large matrix (partially filled out) of distances is output, and those distances are used as constraints in a force field (which also includes lots of prior knowledge about protein structure) and then they run simulations which attempt to minimize both the force field and constraint terms. In principle you don't even need a physical force field- if you have enough distance information between pairs of atoms, you can derive a plausible structure by embedding the distances in R3 (https://en.wikipedia.org/wiki/Distance_geometry https://en.wikipedia.org/wiki/Distance_geometry and https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.215.7674&rep=rep1&type=pdf https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.21... Presumably, the signal they extract includes both rich local interactions (amino acids near in sequence) and distant ones inferred through sequence/structure relationships, and the constraints could in fact push a model towards a novel fold, presumably through some extremely subtle statistical relationships to other evolutionarily related proteins that adopt a different fold.
- ramraj07 4y agoNot sure if you should be reminded of how alpha fold started, it started by winning a competition thought un winnable by academics. Top labs working in protein structure prediction have fundamentally changed direction after alpha fold and are working to do the same even better. This is not the first (or even tenth) time I’m seeing an academic trying to undermine genuine progress almost to the level of gaslighting. Comparing alphafold to conventional homology modeling is disingenuous at its most charitable interpretation. Not sure what else to say. Structural biology has always been the weirdest field I’ve seen, the way students are abused (crystallize and publish in nature or go bust), and how every nature issue will have three structure papers as if that cures cancer every day. I suppose it warps one’s perception of outsiders after being in such a bubble? signed, someone with a PhD in biomedical engineering, did a ton of bio work.
- COGlory 4y ago> Not sure if you should be reminded of how alpha fold started, it started by winning a competition thought un winnable by academics. Top labs working in protein structure prediction have fundamentally changed direction after alpha fold and are working to do the same even better. Not sure what part of "it does homology modeling 2x better" you didn't see in my comment? AlphaFold scored something like 85% in CASP in 2020, in CASP 2016, I-TASSER had I think 42%? So it's ~2x as good as I-TASSER which is exactly what I said in my comment. >This is not the first (or even tenth) time I’m seeing an academic trying to undermine genuine progress almost to the level of gaslighting. Comparing alphafold to conventional homology modeling is disingenuous at its most charitable interpretation. It literally is homology modeling. The deep learning aspect is to boost otherwise unnoticed signal that most homology modeling software couldn't tease out. Also, I don't think I'm gaslighting, but maybe I'm wrong? If anything, I felt gaslit by the language around AlphaFold. >Not sure what else to say. Structural biology has always been the weirdest field I’ve seen, the way students are abused (crystallize and publish in nature or go bust), and how every nature issue will have three structure papers as if that cures cancer every day. I suppose it warps one’s perception of outsiders after being in such a bubble? What on earth are you even talking about? The vast, VAST majority of structures go unpublished ENTIRELY, let alone published in nature. There are almost 200,000 structures on deposit in the PDB.
- deleted 4y ago[deleted]
- zack-m 4y agoYup. It’s great, but there are still many aspects to unpack and work on. Hence why Rosetta is a thing.
- flobosg 4y agoRosetta methods are also moving towards ML. Here’s an article from last week: https://www.science.org/doi/10.1126/science.abn2100 https://www.science.org/doi/10.1126/science.abn2100
- wrycoder 4y ago“Disclaim” stopped me. Disclaim means to deny or renounce.
- Uehreka 4y agoCan we just chill on the whole “using this single word incorrectly breaks your whole argument” thing? A lot of folks on HN end posts about a company with a sentence like “Disclaimer: I used to work for X”. This language (probably taken from contract law or something) is meant an admission of possible bias but in practice is also a signal that this person may know what they’re talking about more-so than the average person. After reading a lot of posts like this, it might feel reasonable for someone to flip the word around say something like “I need to disclaim…” when beginning a post, in order to signal their proximity to a topic or field as well as any sort of insider bias they may possess. So sure, “I need to disclose” would’ve been the better word choice, but we all knew what GP was saying. It seems pedantic to imply otherwise.
- alasdair_ 4y ago>we all knew what GP was saying I was confused initially too.
- wrycoder 4y agoLet me translate. They said, “I will disclaim I am a professional structural biologist that works in this field every day.” That is synonymous with saying, “I will deny I am a professional structural biologist that works in this field every day.” The person posting is actually a structural biologist. What they stated was cognitively dissonant with the intent of their post, and that’s what stopped me. I don’t pay attention to typos or minor usage issues, but in this case, I read two more sentences and said, “What??” EDIT: Two more things. First, I found the post interesting and useful. I didn’t say anything about breaking the argument. Second, “I need to disclose…” is the exact opposite of what they said.
- TotempaaltJ 4y agoLooks like the downvotes say that your interpretation of this language in this context is not the most common interpretation.
- mellosouls 4y agoThese threads are always the same: lots of comments about protein folding, how amazing DeepMind is, how AlphaFold is a success story, how it has flipped an entire field on it's head, etc. I don't think that's necessarily so - there is a lot of justified scepticism about the wilder claims of ML in this forum; it is in fact quite difficult at times to know as an outsider to the field in question how kneejerk it is.
- robbintt 4y agoAdditionally, folding doesn't focus on what matters. Generally you want to understand the active site, you already know the context (globular, membrane, embedded, conjugated) of the protein. It is interesting whether the folding could help identify active sites for further analysis. But -- I don't think alphago is identifying new active sites or improving our understanding of their nuances.
- ehsankia 4y agoRight, but even a speed up / quality increase can flip workflows on their head. Take ray tracing for example, when you speed it up by an order of magnitude, you can suddenly go from taking a break every time you want to render a scene, vs being able to iteratively work on a scene and preview it as you work.
- uoaei 4y agoI got a lot of shit (still do) when the news first broke for pushing back against the notion that AlphaFold "solved" protein folding. People really wanted to attach that word to the achievement. Thank you for providing a nuanced take on exactly why that doesn't make any sense.
- eachro 4y agoI'm curious to read more on the 3-4 other deep learning projects you mentioned that have had a larger impact on your fields. Can you share some links to those works?
- simsla 4y agoI asked a structural biologist friend of mine (world class lab) about the impact of alphafold. They said it's minimal. In most cases, having a "probably" isn't good enough. They use alphafold to get early insights, but then they still use crystallography to confirm the structure. Because at the end of the day, you need to know for sure.
- bawolff 4y agoI'm not a biologist, but that doesn't sound minimal if crystallography is expensive. It sounds like how we model airplanes in computers, but still test the real thing - i wouldn't call the impact of computer modelling on airplane design to be minimal.
- deleted 4y ago[deleted]
- panabee 4y agoas an outsider learning more about protein folding, could you elaborate on the assertion that the sequence is (mostly) all you need (transformer/ML reference intended). doesn't this assume the final fold is static and invariant of environmental and protein interactions? put another way, how do we know that a protein does not fold differently under different environmental conditions or with different molecular interactions? i realize this is a long-held assumption, but after studying scientific research for the past year, i realize many long-held assumptions aren't supported by convincing evidence.
- Protostome 4y agoDisclaimer: I'm a professional (computational) structural biologist. My opinion is slightly different. The problem with the structure prediction problem is not a loss/energy function problem, even if we had an accurate model of all the forces involved we'd still not have an accurate protein structure prediction algorithm. Protein folding is a chaotic process (similar to the 3 body problem). There's an enormous number of interactions involved - between different amino acids, solvent and more. Numerical computation can't solve chaotic systems because floating point numbers have a finite representation, which leads to rounding errors and loss of accuracy. Besides, Short range electro static and van der waals interactions are pretty well understood and before alphafold many algorithms (like Rosetta) were pretty successful in a lot of protein modeling tasks. Therefore, we need a *practical* way to look at protein structure determination that is akin to AlphaFold2.