3 ms·

I have a related question about this. If experimental methods produce results around a score of 90, what is the baseline we are comparing the DeepMind results a

I have a related question about this. If experimental methods produce results around a score of 90, what is the baseline we are comparing the DeepMind results against? If the experimental error is equal to the observed DeepMind error, how can we say which one is actually more erroneous?

0-_-0 6y ago
That's a damn good question, it looks like we don't know how much above 90 AlphaFold is.
- deleted 6y ago
  [deleted]
mrDmrTmrJ 6y ago
Excellent question. At somepoint, I think the only answer is, "have a bunch of different people run a bunch of experiments on the same protein." The threshold for "real" in particle physics is +5 sigma. Which takes a lot of data.
- dnautics 6y ago
  you really can't compare stats like that. Those are independent, uncorrelated measurements. When you take RMSD measurements on a molecule they are not independent (for example, atoms near the core are less likely to be "inaccurate").
cpeterso 6y ago
And is it even meaningful for DeepMind to score better than experimental results? How are DeepMind’s results scored then?
marcosdumay 6y ago
Finding the energy of each configuration should be much easier than finding the lowest-energy configuration. Can that be calculated ab-initio or it is still too expensive?
- crispycrafter2 6y ago
  The problem with ab-initio methods in this context is the sheer number of non-covalent interactions present in these large proteins. A simple protein would require a hybrid quantum mechanic/molecular mechanics simulation to even approximate the vibrational energy required to validate equilibrium. These proteins are so massive that we often use Daltons [1] as an averaged measure of molecular weight. Conceptually one of the most promising applications of quantum computing is theoretical chemistry, and we are only now starting to make progress in this avenue [2]. I anticipate it would require quantum computing to explicitly optimise large folded proteins. 1. https://en.m.wikipedia.org/wiki/Dalton_(unit) https://en.m.wikipedia.org/wiki/Dalton_(unit) 2. https://arxiv.org/abs/2004.04174 https://arxiv.org/abs/2004.04174
radioactivist 6y ago
I think it's that a score of >90 means the result is within the error bars of whatever particular experiment was chosen to be the "reference".
IfOnlyYouKnew 6y ago
The "experiments" here use X-Ray Crystallography. Like most methods of measuring anything, we have a pretty good idea of its accuracy under various conditions. Think of it like satellite imagery of a tree: A score of zero would be a single green-ish pixel, while a score of 100 would show each leaf within the range it naturally moves in due to wind etc. (proteins tend to wiggle quite a bit under natural conditions, as well)

0-_-0 6y ago

That's a damn good question, it looks like we don't know how much above 90 AlphaFold is.

deleted 6y ago
[deleted]

mrDmrTmrJ 6y ago

Excellent question. At somepoint, I think the only answer is, "have a bunch of different people run a bunch of experiments on the same protein." The threshold for "real" in particle physics is +5 sigma. Which takes a lot of data.

dnautics 6y ago
you really can't compare stats like that. Those are independent, uncorrelated measurements. When you take RMSD measurements on a molecule they are not independent (for example, atoms near the core are less likely to be "inaccurate").

cpeterso 6y ago

And is it even meaningful for DeepMind to score better than experimental results? How are DeepMind’s results scored then?

marcosdumay 6y ago

Finding the energy of each configuration should be much easier than finding the lowest-energy configuration. Can that be calculated ab-initio or it is still too expensive?

crispycrafter2 6y ago
The problem with ab-initio methods in this context is the sheer number of non-covalent interactions present in these large proteins. A simple protein would require a hybrid quantum mechanic/molecular mechanics simulation to even approximate the vibrational energy required to validate equilibrium. These proteins are so massive that we often use Daltons [1] as an averaged measure of molecular weight. Conceptually one of the most promising applications of quantum computing is theoretical chemistry, and we are only now starting to make progress in this avenue [2]. I anticipate it would require quantum computing to explicitly optimise large folded proteins. 1. https://en.m.wikipedia.org/wiki/Dalton_(unit) https://en.m.wikipedia.org/wiki/Dalton_(unit) 2. https://arxiv.org/abs/2004.04174 https://arxiv.org/abs/2004.04174

radioactivist 6y ago

I think it's that a score of >90 means the result is within the error bars of whatever particular experiment was chosen to be the "reference".

IfOnlyYouKnew 6y ago

The "experiments" here use X-Ray Crystallography. Like most methods of measuring anything, we have a pretty good idea of its accuracy under various conditions. Think of it like satellite imagery of a tree: A score of zero would be a single green-ish pixel, while a score of 100 would show each leaf within the range it naturally moves in due to wind etc. (proteins tend to wiggle quite a bit under natural conditions, as well)