10 ms·
"Now we finally get to something interesting: how AF2 actually works! Alas, I will be able to say a lot less than I had hoped for, and here I have to do somethi
by resiros 6y ago
"Now we finally get to something interesting: how AF2 actually works! Alas, I will be able to say a lot less than I had hoped for, and here I have to do something which I very much dislike to do but feel that I must—call out DeepMind for falling short of the standards of academic communication. What was presented on AF2 at CASP14 barely resembled a methods talk. It was exceedingly high-level, heavy on ideas and insinuations but almost entirely devoid of detail"
I agree 100% with this paragraph. I have watched alphafold presentation and was as disappointed as Mohammed. Deepmind kept all the interesting parts of their method secret and Jumper dodged question after the next about the method. I am disappointed because deepmind would not had any chance solving this problem without the CASP community and advances shared over more than 20 years.
- lrossi 6y agoI fully agree with this as well. This is why I think it’s important for research to be carried out by academia: they have no incentive to hide their secret sauce.
- wiz21c 6y agoYep. This is soooo sad. The mighty power of Google allows it to, once again, be the winner takes it all. And unfortunately, get into a position where results could be protected. So Google will probably leverage his commercial advantage for some years before enough details surface so that one can rebuild a solution that can be shared freely. Note this protein folding thing was not solved by other mighty giants such as Johnson&Johnson, Pfizer, etc. So basically, Google has even more power than before. But to be honest, energy production knowledge (nuclear stuff) is not in the hands of governments but in the military-industrial complex... So it's history repeating...
- refulgentis 6y agoThe civilizational struggle lens feels a bit overwrought, OP notes this was on the competition's timeline, not AlphaFold's, and Deep Mind is rushing a paper to follow soon.
- epmaybe 6y agoThere’s plenty of incentive if you are at risk of getting ‘scooped’.. especially if your livelihood depends on a publish or perish model.
- YetAnotherNick 6y agoI don't quite agree with this. I work in this field and it is quite rare to get the code along with paper by people from academia. In fact many a times big companies tries to replicate the the published work and publish the code/model like say TF Model Garden.
- jhbadger 6y agoIt depends on what you mean by "the field". Do you mean machine learning? That might be the case given the corporate dominance of that technique. But in bioinformatics (the domain to which AlphaFold2 is applying ML to) it is generally expected that all data and code be made available for a tool. Otherwise what's the point? If people can't build upon it, science hasn't been done.
- timr 6y agoYou're overstating the situation. The previous top-performing protein folding method is essentially impossible to reproduce from public literature, and available only via licensing agreement with the lab that created it. More generally, while it's far more common to find open-source implementations of lab research than it used to be, it's still quite common for code to be closed, and important methodological details to be secret. Even in the "open" fields of ML, today, I routinely find papers where essential bits of code, parameter sets, etc. are not made available.
- throwaway2245 6y agoIn bioinformatics and academic protein folding in particular, there has been a strong tradition of sharing work and building on others' work. That's arguably why CASP exists, as part of a recognition that the field benefits from collaborative effort to solve the problem - e.g. via public identification of what approaches have worked.
- emteycz 6y agoHow come the next best protein folding software is proprietary, then?
- dnautics 6y agoWhat is "in" and what is "out" of academia. Plenty of important research has been conducted in industry, for centuries, from the invention of student's t test through everything that irving langmuir discovered and nyquist sampling theorem. Some things have been discovered in "basically not academia", too, like the chemiosmotic effect.
- dekhn 6y agoI've seen many academics hide the secret sauce. Getting tenure is competitive and any paper you can publish that gets lots of cites is important. Leaving out tiny details to replicate work is common.
- fh973 6y agoIs it a scientifc result if it isn't reproducible? Would the result have been accepted if it was submitted by an unknown?
- learnstats2 6y agoThe reproducible part is that they can consistently fold proteins - which they can. The fact that they're not telling anybody else how they did it stops it from being public science, but doesn't stop it from being science.
- ISL 6y agoThere are two terms that may be conflated in this particular discussion. Reproducible: Can it be done again? Replicable: Can someone else do it? AF2 appears to be reproducible, but not yet replicable. https://phys.org/news/2019-05-replicability-science.html#:~:text=Reproducibility%20means%20obtaining%20consistent%20computational,has%20obtained%20its%20own%20data https://phys.org/news/2019-05-replicability-science.html#:~:....
- YetAnotherNick 6y ago> Can someone else do it? > but not yet replicable. Why not? I don't see a blocker except for the money to create a team and the computational resources. I think AlphaGo/Zero was harder to replicate than this, still Leela's team managed to get it working.
- NeutronStar 6y agoHow do I replicate the code (procedure) if I can't see the code (procedure) ?
- deleted 6y ago[deleted]
- ignoramous 6y agoDeepMind should re-read their own policies: https://deepmind.com/about/ethics-and-society https://deepmind.com/about/ethics-and-society
- marcinzm 6y agoWhich part specifically? A quick read doesn't indicate anything about being open about the details of their methods. Mostly the policies are about the ethics of how AI is built and used.
- ignoramous 6y ago> We start from the belief that AI should be used for socially beneficial purposes and always remain under meaningful human control. Understanding what this means in practice is essential. > We embrace scientific values like transparency, freedom of thought, and the equality of access...
- emteycz 6y agoThat's wildly different from making everything publicly available.
- ignoramous 6y agoNot if you claim it is essential to "understand what it means in practice"?
- emteycz 6y agoBut it is not. It is essential to do the same thing without spending any effort yourself, it's not essential for understanding the science.
- alphagrep12345 6y agoThey mentioned that they'd publish a paper soon. Why is it a problem, then?