There are two threads here. The first is that it would not be surprising to learn that describing the way that proteins fold is a very hard thing for humans to

There are two threads here. The first is that it would not be surprising to learn that describing the way that proteins fold is a very hard thing for humans to understand. See i.e. 4CT [1] and its computational proofs. The second is that explainability in ML is much more tractable than it was 10 years ago. This is not to say that it's solved, but having solved the predictive problem -- I would expect model simplifications and SME research to proceed more quickly towards understanding the how now. I did some work w/ an Astrophysics postdoc using beta-VAEs [2] to classify astronomical observations, and simplifying models in order to achieve human-explainability proved to not cost as much predictive power as you might expect. It might be that the same holds true here. 1- https://mathworld.wolfram.com/Four-ColorTheorem.html https://mathworld.wolfram.com/Four-ColorTheorem.html 2 - https://paperswithcode.com/method/beta-vae https://paperswithcode.com/method/beta-vae