3 ms·

I know a thing or two about the field. Custom hardware for molecular dynamics allows one to simulate longer timescale biological processes, namely the folding o

by cing 13y ago

I know a thing or two about the field. Custom hardware for molecular dynamics allows one to simulate longer timescale biological processes, namely the folding of larger proteins. More generally, in the study of any size protein, countless molecular dynamics papers are simply plagued with statistical error and cannot be trusted. I think it's unanimous that DE Shaw Research is doing a great service and I'm excited to see the performance of their next supercomputer, Anton2, http://dl.acm.org/citation.cfm?id=2451175 http://dl.acm.org/citation.cfm?id=2451175

varelse 13y ago
But as in all things, Anton makes a controversial set of assumptions and approximations w/r to long-range electrostatics so it's anyone's guess whether their approach is valid or not. All that aside, Anton is a wicked cool piece of hardware able to accelerate their approximation of traditional molecular dynamics by a factor of 50 over GPUs and up to a factor of 500 over CPU clusters, hitting milliseconds of simulated protein time in a single trajectory. And since it's the only hardware capable of simulating a single monotonic trajectory out to that timescale, it's challenging to compare it to other approaches.
- timr 13y ago
  Agreed. DE Shaw's hardware work is very cool, but I'm in the camp that believes that the force fields suck (in general; not just theirs), and bashing through some multi-millisecond trajectories will only highlight their limitations.
  - cing 13y ago
    You should check out Lane et al. 2013 Fig. 1: http://www.sciencedirect.com/science/article/pii/S0959440X12001820 http://www.sciencedirect.com/science/article/pii/S0959440X12... All computational protein folding times match experimental results within 1 order of magnitude. If folding time isn't a good measure of forcefield accuracy then I don't know what is. Sampling error is a worse offender than forcefield accuracy in my opinion.
    - varelse 13y ago
      To be fair, the Pande Lab's data has involved a degree of human intervention during the generation of the multi-state trajectories upon which they're based. And that's a failing that has only recently been addressed. IMO it's indeterminate whether that human intervention could have unwittingly influenced those trajectories. Similarly, a lot of the folded proteins were used to develop the same force fields now used to simulate their folding. I'm not dismissing this data, but I am saying I think the jury is still out on the models until we have a true test set/training set dichotomy, a separation that was an absolute revelation for ab initio structure prediction algorithms in the 1990s. That said, I think we both agree that undersampling is the biggest offender. And it only gets worse with results published for larger systems simulated at the same timescale as much smaller ones.
    - timr 13y ago
      " If folding time isn't a good measure of forcefield accuracy then I don't know what is." Folding kinetics are surprisingly insensitive to detail. For example, it's been understood for a while that even ridiculously simple models can predict transition state energies for simple proteins: http://www.ncbi.nlm.nih.gov/pubmed/10322214?dopt=Abstract http://www.ncbi.nlm.nih.gov/pubmed/10322214?dopt=Abstract http://www.ncbi.nlm.nih.gov/pubmed/10500172?dopt=Abstract http://www.ncbi.nlm.nih.gov/pubmed/10500172?dopt=Abstract But to answer your question, I'd say that "predicting the correct structure of a protein" is the gold-standard benchmark of forcefield accuracy, and MD forcefields are really bad at it. (You could reasonably add other great measures, like: "does the simulation tend to fly apart without hacked-up pseudo-physical constraints?", but that feels like piling on.)
      - cing 13y ago
        I'm biased since I do simulations but you're being too harsh on force fields. All the proteins in the paper I mentioned had their structure correctly predicted starting from an extended state. Correct structure and folding time! That's a serious accomplishment compared to the decade-old papers you're citing.
        13y ago

by cing 13y ago

varelse 13y ago

But as in all things, Anton makes a controversial set of assumptions and approximations w/r to long-range electrostatics so it's anyone's guess whether their approach is valid or not. All that aside, Anton is a wicked cool piece of hardware able to accelerate their approximation of traditional molecular dynamics by a factor of 50 over GPUs and up to a factor of 500 over CPU clusters, hitting milliseconds of simulated protein time in a single trajectory. And since it's the only hardware capable of simulating a single monotonic trajectory out to that timescale, it's challenging to compare it to other approaches.

timr 13y ago
Agreed. DE Shaw's hardware work is very cool, but I'm in the camp that believes that the force fields suck (in general; not just theirs), and bashing through some multi-millisecond trajectories will only highlight their limitations.
- cing 13y ago
  You should check out Lane et al. 2013 Fig. 1: http://www.sciencedirect.com/science/article/pii/S0959440X12001820 http://www.sciencedirect.com/science/article/pii/S0959440X12... All computational protein folding times match experimental results within 1 order of magnitude. If folding time isn't a good measure of forcefield accuracy then I don't know what is. Sampling error is a worse offender than forcefield accuracy in my opinion.
  - varelse 13y ago
    To be fair, the Pande Lab's data has involved a degree of human intervention during the generation of the multi-state trajectories upon which they're based. And that's a failing that has only recently been addressed. IMO it's indeterminate whether that human intervention could have unwittingly influenced those trajectories. Similarly, a lot of the folded proteins were used to develop the same force fields now used to simulate their folding. I'm not dismissing this data, but I am saying I think the jury is still out on the models until we have a true test set/training set dichotomy, a separation that was an absolute revelation for ab initio structure prediction algorithms in the 1990s. That said, I think we both agree that undersampling is the biggest offender. And it only gets worse with results published for larger systems simulated at the same timescale as much smaller ones.
  - timr 13y ago
    " If folding time isn't a good measure of forcefield accuracy then I don't know what is." Folding kinetics are surprisingly insensitive to detail. For example, it's been understood for a while that even ridiculously simple models can predict transition state energies for simple proteins: http://www.ncbi.nlm.nih.gov/pubmed/10322214?dopt=Abstract http://www.ncbi.nlm.nih.gov/pubmed/10322214?dopt=Abstract http://www.ncbi.nlm.nih.gov/pubmed/10500172?dopt=Abstract http://www.ncbi.nlm.nih.gov/pubmed/10500172?dopt=Abstract But to answer your question, I'd say that "predicting the correct structure of a protein" is the gold-standard benchmark of forcefield accuracy, and MD forcefields are really bad at it. (You could reasonably add other great measures, like: "does the simulation tend to fly apart without hacked-up pseudo-physical constraints?", but that feels like piling on.)
    - cing 13y ago
      I'm biased since I do simulations but you're being too harsh on force fields. All the proteins in the paper I mentioned had their structure correctly predicted starting from an extended state. Correct structure and folding time! That's a serious accomplishment compared to the decade-old papers you're citing.
      - 13y ago