As someone who works with Transformers and DL in general I decided to chime in: >They can be aware of implicit correlations and constraints that are totally ou

As someone who works with Transformers and DL in general I decided to chime in: >They can be aware of implicit correlations and constraints that are totally outside what may be recognised in the scientific understanding. Hence AlphaFold is quite capable of predicting a structure for which there is no previous direct "analogy". Yes, while an expert of their field, I suspect author doesn't have full understanding of what neural networks or ML is capable of, in this case it's not necessary to have seen fully similar molecules but it is sufficient for AlphaFold to have encountered basic blocks of the vocabulary that are familiar. One can see how this argument would be silly if (hyperbolizing) someone said that just because we are located in a tiny part of observable universe the physics we know to work here wouldn't work in Alpha Centauri. To make this point simpler, this is no different than suggesting that a model will break if it encounters an out-of-vocabulary word, whereas in reality a simple tokenization technique would ensure that in majority of these edge cases, the unknown word would be broken down into subwords, which, in turn, would still be familiar to the model and statistics that model has learned would still give it a reasonable guess as to what the sum of the parts entail. There was a recent work (https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2207.05221, in LLM, not molecular prediction) where models could be shown to "know" (oh G-d how tiresome it is to make these disclaimers, but models are NOT sentient) fairly decently what they don't know. Thus, I wouldn't be surprised if AlphaFold could at least give a confidence score to its prediction of folded protein, helping scientists who use it as a TOOL (which it is, it's not a solution) to exercise caution. >As long as the protein has to follow the laws of physics then AlphaFold as at least a basis to work from in successfully predicting the structure. This is pure speculation. There is no guarantee AT ALL that AlphaFold follows laws of physics other than the way predictions are clipped to be reasonable distances between molecules or other "hacks" authors have added as "inductive" (actually symbolic) biases to the model.