8 ms·

> I don’t think we would do ourselves a service by not recognizing that what just happened presents a serious indictment of academic science. Much like other f

> I don’t think we would do ourselves a service by not recognizing that what just happened presents a serious indictment of academic science. Much like other fields, I do begin to question the academic structure to making advances. It appears something is rotten in the state of academia. Oddly it's academia doing incremental improvements to existing methods but industry making novel leaps and bounds... The other major case in point being NLP

codingslave 6y ago
Academia keeps employing people who have done well in classes and within fine bounds. Its a careerist track. Industry cares about results, its more meritocratic
- rtsil 6y ago
  > Industry cares about results, its more meritocratic Industry cares about positive results. If you're not allowed to fail, you will be afraid to explore. That's what Academia is. Then, the industry reaps the fruit of that exploration, which is as it should be.
  - haihaibye 6y ago
    High impact journals care about positive results and academia very much cares about journals. You need to win grants to survive.
  - njanirudh 6y ago
    If academia actually allowed failure we wouldn't be getting so many tiny incremental growth papers just for the sake of it as in deep learning and machine learning.
  - JPLeRouzic 6y ago
    In my (one point) experience of industry R&D more projects failed than succeeded, but project leaders were not blamed. How could one predict success or failure in risky project proposals? In both cases you learn something.
- Ar-Curunir 6y ago
  I wish people would stop making reductive and false statements. Life isn't this simple.
  - codingslave 6y ago
    This is actually something I feel strongly about. The absent minded creative professor is the one who traditionally has made the breakthroughs. Recent years has instead seen the straight A student with no curiosity making it into programs, when they really have no business doing novel research and are better suited as orderly wage slaves
    - Ar-Curunir 6y ago
      There's a lot of fanciful stuff about "creative" and "absent-minded" people doing the best work, but what actually makes a good researcher is the same as in any other field: (a) curiosity (b) determination, and (c) hard work. PhD programs don't take people who just have a good GPA; you have to have a research record before you're even in the consideration. I've been on an admissions committee, so this is not conjecture.
      - codingslave 6y ago
        Right, but a modern research record is about incremental improvement. The argument being that low hanging fruit is often picked, and so the incremental is natural. My argument is that far too many people are gaming the academic system, using it as a form of status credentialing, which is hurting true academic research.
ac42 6y ago
I think so, too. Linear algebra, control theory and quantum mechanics haven't gotten us anywhere and ivory towers prevail as this machine learning solution to a problem in biological chemistry clearly demonstrates. /s
segfaultbuserr 6y ago
> something is rotten in the state of academia. Oddly it's academia doing incremental improvements to existing methods but industry making novel leaps and bounds... The other major case in point being NLP You have to realize that corporate research labs had a high level of recognition back in the 20th century. Labs like the Bell Labs, the RCA Laboratories, or the IBM Research, privately-funded, had a reputation that met or exceeded the standard of not-for-profit or public-funded academic research institutions. They made some of the most important discoveries in the electronics industry of 20th century, like the point-contact transistor, the MOSFET, VLSI, or the UNIX operating system. They were considered a part of the academia, many scientists were their employees. It's only the 1980s after their death that people had the impression that "important research must come from academia, industry is for incremental changes." So, I'd argue that the division between industry and academia is large, but actually smaller than people's perception. If you consider privately-funded researches by the industry as a part of the academia, the current situation is totally normal, nothing unusual. Interestingly, for those labs to exist, being a monopolistic megacorp is a requirement. It appears to me that today's FAANG monopoly allowed the creation of Google Deepmind and OpenAI, perhaps it's simply a beginning of the repetition of history. The article The death of corporate research labs had an interesting review. I highly recommend to read the article: * The death of corporate research labs > https://blog.dshr.org/2020/05/the-death-of-corporate-research-labs.html https://blog.dshr.org/2020/05/the-death-of-corporate-researc... (HN comment: https://news.ycombinator.com/item?id=232466722 https://news.ycombinator.com/item?id=232466722) To summarize, those great labs existed and made great contributions because of (1) corporate monopoly on the industry, and (2) the pressure from anti-trust laws. First, due to monopoly, the gigantic size allowed the labs to be the center of gravity and to concentrate all talents and projects into a single place, with a huge research budget for basic research. Second, the pressure from anti-trust laws also forced corporations to invent more into basic research to grow the business, because mergers and acquisitions were restricted. In some cases, the pressure from anti-trust laws also made the corporate labs to share their discoveries in a more open manner, examples included advances in semiconductor [1], or the Unix source code. Note: but as HN comments pointed out, somewhat ironically, the success of corporate labs relies on anti-trust pressures, but not the actual monopoly-busting enforcement. The breakup of Bell caused the death of the Bell Labs. Finally their decline, > The more relaxed antitrust environment in the 1980s, however, changed this status quo. Growth through acquisitions became a more viable alternative to internal research, and hence the need to invest in internal research was reduced. And it turns out that managing a corporate research labs without losing money is a tricky problem to solve. If the researches are too goal-oriented, short-termism will dominate, basic research in the labs will be ignored. Thus, basic research in the lab must be independent. However, a lab too isolated from the business can also cause great loss. > Research in corporations is difficult to manage profitably. Research projects have long horizons and few intermediate milestones that are meaningful to non-experts. As a result, research inside companies can only survive if insulated from the short-term performance requirements of business divisions. However, insulating research from business also has perils. [...] Walking this tightrope has been extremely difficult. Greater product market competition, shorter technology life cycles, and more demanding investors have added to this challenge. Companies have increasingly concluded that they can do better by sourcing knowledge from outside, rather than betting on making game-changing discoveries in-house. And the author argued the death of corporate labs decreased productivity. >> An unintended consequence of abandoning anti-trust enforcement was thus a slowing of productivity growth, because the this new division of labor wasn't as effective as the labs: > a new division of innovative labor, with universities focusing on research, large firms focusing on development and commercialization, and spinoffs, startups, and university technology licensing offices responsible for connecting the two. > The translation of scientific knowledge generated in universities to productivity enhancing technical progress has proved to be more difficult to accomplish in practice than expected. Spinoffs, startups, and university licensing offices have not fully filled the gap left by the decline of the corporate lab. Corporate research has a number of characteristics that make it very valuable for science-based innovation and growth. Large corporations have access to significant resources, can more easily integrate multiple knowledge streams, and direct their research toward solving specific practical problems, which makes it more likely for them to produce commercial applications. University research has tended to be curiosity-driven rather than mission-focused. It has favored insight rather than solutions to specific problems, and partly as a consequence, university research has required additional integration and transformation to become economically useful. --- [0] https://www.eetimes.com/podcasts/six-words-that-built-the-ic-industry-unhacking-the-2020-election-epic-leadership-in-an-epidemic/ https://www.eetimes.com/podcasts/six-words-that-built-the-ic... > Honeywell brought a lawsuit against us and said you can’t selectively choose people to divulge your technology to. It’s too important. And if you divulge it to anyone, you’ve got to divulge it to everybody. They filed a lawsuit, and the government came down on their side. And RCA basically had to open up all of its patents to everybody if they opened them up to anybody.
sampo 6y ago
> The other major case in point being NLP Speaking of which, Google Translate was published in 2006, but when did the "learning from data" approach became an accepted idea in machine translation? I think the earlier attempts at machine translation were more about trying to codify grammar rules in software, than doing statistical learning from large text corpuses? I remember in 2002, the approach of leaning protein substructures from data was already the best performing approach in the protein folding problem.
- xbmcuser 6y ago
  I think Netflix model of simplifying then translating should work better with internet forums and blog posts. Looking at how google works I was hoping someone in google wold adopt it and release a competing product against google translate https://arxiv.org/abs/2005.11197 https://arxiv.org/abs/2005.11197
- Polygator 6y ago
  Not really. Using a statistical approach to text modelling, specifically using Markov Chains, was proposed by Shannon in 1948. But yeah, there's a point in the 2000s where generative grammar/ symbolic approaches were pretty much left behind by NN methods. When we discuss Google's input in NLP, the most important contribution is certainly the "Attention is all you Need" paper, which paved the way for BERT and GTP (Alphafold also uses Attention networks, btw)
  - adw 6y ago
    > generative grammar/ symbolic approaches were pretty much left behind by NN methods Which is the same thing as hand-engineered feature stacks being left behind in vision problems, really. The story in every field is more or less "you're not clever enough to engineer good features"; "you might be clever enough to define good symmetries for the feature space in which the features live... maybe" (convolutional neural networks in image problems); "... but maybe not even that" (attention mechanisms).
    - snovv_crash 6y ago
Tomminn 6y ago
Almost every single of the tens of thousand papers by the hundreds of tenured academics in the field of protein folding are made obsolete by 10 google engineers. This is what it's like when someone really moves the needle. And academic science cannot get it's head around it. And yet, none of these scientists will suffer any career consequences. Their irrelevant work will be healthily cited by all the other scientists who are doing and have done irrelevant work. They'll retcon a story in their lit reviews about how their irrelevant work led to this. The career consequences are saved for those who had their eye on the real ball for the last five years, but didn't get there first. For them, the comfortably irrelevant will have the gall to ask in accusatory tones: "What have you been doing these last 5 years?".
- mnky9800n 6y ago
  Did you miss the part where a bunch of the deepmind authors did phds and postdocs in protein folding related fields? They pushed the needle because they understood the field and the new tools. This is how pushing the needle works.
- MMM13SFH 6y ago
  fuck youre a retard, do you think this happened in a vacuum? pathetic wageslave
mensetmanusman 6y ago
Academia is for generating problem solvers. Teams are small and made of people who will be there for around 5 years. A better comparison would be to national labs, but they are tasked with projects that make no sense for industry to tackle. The system is working as intended, all players are needed. The team at Alphafold busted their chops in academia and went on to working on problems they could spend decades on.
- mnky9800n 6y ago
  People seem to forget that you need a system like academia that's allowed to fail. Most companies aren't allowed to fail when they need to have quarterly returns. Of course academia has become more and more competitive. But tbh I think the answer is that the funding hasn't increased equally with the number of quality people who could stay in academia. But who knows.
  - Jach 6y ago
    Stability "like academia" is rich, given all we've heard about "publish or perish". Modern academia is a poor fit for increasingly any case you can think of besides maintaining the status of academia. But sure, there needs to be some stability and ability to "fail"/i.e. produce something worthless. Corporate research departments provide this -- if they didn't, they wouldn't have a research department and indeed many don't, nor do they need to, but this has little to do with quarterly returns. We've also seen a rise of VC-backed research startups (like DeepMind but many others) whose value proposition (to the VC) only makes sense if the goal is to demonstrate a research capacity to get them bought out by a big company, or as a moonshot to out-compete them on an actual product made possible by the research. Investing in these little research startups themselves is also giving companies a way to push research without having to deal with having the researchers as direct employees, and I'm sure makes some of the startup employees feel a bit safer since there's a separation of money and operation influence. One similarity with modern academia is it selects for those who can do good work but who are also good bureaucrats (write grant proposals well, advising politicians, etc), startups have a selection for good work + good at courting VCs. But the startup just needs a few of them, then they can hire people who just want to do good work. Another thing that makes corporate even better is they can occasionally spin off research developments into products, they can have some nice advances that only come when you try to productize, and among other reasons by not having to bother with external publishing (which takes time + fights with lawyers and business people) they can routinely be 10+ years ahead of whatever the state of the art in academia is.
calf 6y ago
Is this a scientific advance or a technological one though? Academia doesn't have the capital like industry or government to implement the latter. In America it's small small groups of young students led by a professor. Not full grown PhDs with Google levels of staff and money.
- adwn 6y ago
  > Is this a scientific advance or a technological one though? Where's the difference? (this is a genuine question, I'm not trying to trick you)
- dwheeler 6y ago
  I would claim that this is a technological advance that is likely to lead to many scientific advances. Perhaps a good analogy are the inventions of the microscope and telescope. They were advances in technology, which then led to advances in science. I don't know if this will have the same effect as the microscope and telescope, but it would be great if it did. It certainly seems extremely promising.
xpe 6y ago
It is not always either/or. It is not always A versus B. These fallacies are too common.
C4stor 6y ago
The deepmind research team is essentially all PhD though, so it seems academia isn't doing such a bad job.
- softwaredoug 6y ago
  Specifically the linked article wonders about the research environment of academia compared to industry. Why teams of hundreds in academia with their own super computing resources couldn’t make the same advances. He posits there’s something not great going on about how academic research environments make advances, the poor incentive structures, the abuse and burnout of PhDs, the lack of open sharing of findings, the obsession with publication quantity over quality... There’s a reason these PhDs at Google mind aren’t in academia after all doing the same work
  - C4stor 6y ago
    Academia is doing a lot of advances every year. The fact it didn't make _this one_ is not really relevant to postulate that academia is inefficient. It happens that the team at deepmind is apparently pretty damn good at deep learning problems, so they're going faster than matching academia labs. It's not to say that academia has none of the problems you mentioned, but it's imo unreasonable to expect that, in a world where both public and private labs exist, only public ones would make advances.
  - krull10 6y ago
    I would say that is most likely due to a massively higher salary and no teaching responsibilities. Academia can’t compete on salary with industry in AI / data science.
lambdamore 6y ago
Deepmind does not foster its future PhD, but, yes, they offer a better rewarding environment for those PhDs to flourish after they get the basic training.
jboggan 6y ago
I used to work in one of the top labs doing protein folding, in fact I recognize some lab mates who survived to have their own labs in this year's CASP top ten ranking. Something is a bit rotten though because I'd guess about half of us burned out of academia entirely, and the state of the art at the time was regrettable. I remember one model doing rather well except the weights of the various physical forces were wrong, so much that some signs were negative and what should have been electrostatic repulsion was actually electrostatic attraction between similar charges. This was 10 years ago of course, but all of the field improvements were relatively incremental and small until AlphaFold came along. Modern academia exists to further modern academia, no more and no less. I became disillusioned of any search for truth and progress during my time there, it was really just about showing up the Baker lab at the next CASP and getting the next round of grants secured.
- softwaredoug 6y ago
  This is exactly my point. There is something rotten in the research labs where society is not getting return on investment for basic research. If the goal is to just produce researchers that can work at corporate research labs, then I feel we could get more bang for the buck. If the goal is to do move research in the public good, something needs to change. Maybe it’s the fact there’s too little money out there, and it causes everyone to chase meager grant money. Or that there’s too much competition between groups. Or a million other reasons. But I’d love to see it fixed and have more faith in public investment in basic science.
  - gwicks56 6y ago
    From my experience working as an engineer in Academia, there is a big problem in anything AI related, in that anyone decent can instantly increase their pay by 200 -1000% just by leaving for one of the big tech companies. Half the phD students don't even bother finishing, before being given an offer they can't refuse. How can academia compete with that, given sky high student debt, an extremely uncertain path to tenure etc etc
  - PaulDavisThe1st 6y ago
    The problem is that people in academia and outside of it were saying the same thing in the 70s, 80s, 90s, 00's ... so if you really want to make the claim that "society is not getting return on investment for basic research", you need to claim with a straight face that this also applies to the last 50 years of academic basic research (at least). Alternatively, explain what has changed and when.
twic 6y ago
The academic groups working on this have a tiny fraction of the resources that Google do.
SiempreViernes 6y ago
The article then goes on to describe a not very general set of circumstances > But in part due to the canonicalization of CASP, protein structure prediction effectively has a two-year clock cycle, where separate research groups guard their discoveries until after CASP results are announced. and further noting > As I discussed earlier, it is clear that between the Xu and Zhang groups enough was known to develop a system that would have perhaps rivaled AlphaFold. Finally, and rather crushingly for your thesis, is the points made about the real industrial groups: > What is worse than academic groups getting scooped by DeepMind? The fact that the collective powers of Novartis, Pfizer, etc, with their hundreds of thousands (~million?) of employees, let an industrial lab that is a complete outsider to the field, with virtually no prior molecular sciences experience, come in and thoroughly beat them on a problem that is, quite frankly, of far greater importance to pharmaceuticals than it is to Alphabet. It is an indictment of the laughable “basic research” groups of these companies, which pay lip service to fundamental science but focus myopically on target-driven research that they managed to so badly embarrass themselves in this episode.
krull10 6y ago
Industry is not going to fund the overwhelming majority of research areas in biology, physics, chemistry, mathematics, etc. Data science and AI are an exception, where people in industry are much better paid, and can get access to much better resources that would be hard to afford in academia... It’s not surprising this type of advance came from an industry funded group. On the other hand, it is academia and its structure that has enabled so many other discoveries, for example, Crispr DNA tech, our understanding of gravitational waves, or the proof of the Poincaré conjecture.

codingslave 6y ago

Academia keeps employing people who have done well in classes and within fine bounds. Its a careerist track. Industry cares about results, its more meritocratic

rtsil 6y ago
> Industry cares about results, its more meritocratic Industry cares about positive results. If you're not allowed to fail, you will be afraid to explore. That's what Academia is. Then, the industry reaps the fruit of that exploration, which is as it should be.
- haihaibye 6y ago
  High impact journals care about positive results and academia very much cares about journals. You need to win grants to survive.
- njanirudh 6y ago
  If academia actually allowed failure we wouldn't be getting so many tiny incremental growth papers just for the sake of it as in deep learning and machine learning.
- JPLeRouzic 6y ago
  In my (one point) experience of industry R&D more projects failed than succeeded, but project leaders were not blamed. How could one predict success or failure in risky project proposals? In both cases you learn something.
Ar-Curunir 6y ago
I wish people would stop making reductive and false statements. Life isn't this simple.
- codingslave 6y ago
  This is actually something I feel strongly about. The absent minded creative professor is the one who traditionally has made the breakthroughs. Recent years has instead seen the straight A student with no curiosity making it into programs, when they really have no business doing novel research and are better suited as orderly wage slaves
  - Ar-Curunir 6y ago
    There's a lot of fanciful stuff about "creative" and "absent-minded" people doing the best work, but what actually makes a good researcher is the same as in any other field: (a) curiosity (b) determination, and (c) hard work. PhD programs don't take people who just have a good GPA; you have to have a research record before you're even in the consideration. I've been on an admissions committee, so this is not conjecture.
    - codingslave 6y ago
      Right, but a modern research record is about incremental improvement. The argument being that low hanging fruit is often picked, and so the incremental is natural. My argument is that far too many people are gaming the academic system, using it as a form of status credentialing, which is hurting true academic research.

ac42 6y ago

I think so, too. Linear algebra, control theory and quantum mechanics haven't gotten us anywhere and ivory towers prevail as this machine learning solution to a problem in biological chemistry clearly demonstrates. /s

segfaultbuserr 6y ago

> something is rotten in the state of academia. Oddly it's academia doing incremental improvements to existing methods but industry making novel leaps and bounds... The other major case in point being NLP You have to realize that corporate research labs had a high level of recognition back in the 20th century. Labs like the Bell Labs, the RCA Laboratories, or the IBM Research, privately-funded, had a reputation that met or exceeded the standard of not-for-profit or public-funded academic research institutions. They made some of the most important discoveries in the electronics industry of 20th century, like the point-contact transistor, the MOSFET, VLSI, or the UNIX operating system. They were considered a part of the academia, many scientists were their employees. It's only the 1980s after their death that people had the impression that "important research must come from academia, industry is for incremental changes." So, I'd argue that the division between industry and academia is large, but actually smaller than people's perception. If you consider privately-funded researches by the industry as a part of the academia, the current situation is totally normal, nothing unusual. Interestingly, for those labs to exist, being a monopolistic megacorp is a requirement. It appears to me that today's FAANG monopoly allowed the creation of Google Deepmind and OpenAI, perhaps it's simply a beginning of the repetition of history. The article The death of corporate research labs had an interesting review. I highly recommend to read the article: * The death of corporate research labs > https://blog.dshr.org/2020/05/the-death-of-corporate-research-labs.html https://blog.dshr.org/2020/05/the-death-of-corporate-researc... (HN comment: https://news.ycombinator.com/item?id=232466722 https://news.ycombinator.com/item?id=232466722) To summarize, those great labs existed and made great contributions because of (1) corporate monopoly on the industry, and (2) the pressure from anti-trust laws. First, due to monopoly, the gigantic size allowed the labs to be the center of gravity and to concentrate all talents and projects into a single place, with a huge research budget for basic research. Second, the pressure from anti-trust laws also forced corporations to invent more into basic research to grow the business, because mergers and acquisitions were restricted. In some cases, the pressure from anti-trust laws also made the corporate labs to share their discoveries in a more open manner, examples included advances in semiconductor [1], or the Unix source code. Note: but as HN comments pointed out, somewhat ironically, the success of corporate labs relies on anti-trust pressures, but not the actual monopoly-busting enforcement. The breakup of Bell caused the death of the Bell Labs. Finally their decline, > The more relaxed antitrust environment in the 1980s, however, changed this status quo. Growth through acquisitions became a more viable alternative to internal research, and hence the need to invest in internal research was reduced. And it turns out that managing a corporate research labs without losing money is a tricky problem to solve. If the researches are too goal-oriented, short-termism will dominate, basic research in the labs will be ignored. Thus, basic research in the lab must be independent. However, a lab too isolated from the business can also cause great loss. > Research in corporations is difficult to manage profitably. Research projects have long horizons and few intermediate milestones that are meaningful to non-experts. As a result, research inside companies can only survive if insulated from the short-term performance requirements of business divisions. However, insulating research from business also has perils. [...] Walking this tightrope has been extremely difficult. Greater product market competition, shorter technology life cycles, and more demanding investors have added to this challenge. Companies have increasingly concluded that they can do better by sourcing knowledge from outside, rather than betting on making game-changing discoveries in-house. And the author argued the death of corporate labs decreased productivity. >> An unintended consequence of abandoning anti-trust enforcement was thus a slowing of productivity growth, because the this new division of labor wasn't as effective as the labs: > a new division of innovative labor, with universities focusing on research, large firms focusing on development and commercialization, and spinoffs, startups, and university technology licensing offices responsible for connecting the two. > The translation of scientific knowledge generated in universities to productivity enhancing technical progress has proved to be more difficult to accomplish in practice than expected. Spinoffs, startups, and university licensing offices have not fully filled the gap left by the decline of the corporate lab. Corporate research has a number of characteristics that make it very valuable for science-based innovation and growth. Large corporations have access to significant resources, can more easily integrate multiple knowledge streams, and direct their research toward solving specific practical problems, which makes it more likely for them to produce commercial applications. University research has tended to be curiosity-driven rather than mission-focused. It has favored insight rather than solutions to specific problems, and partly as a consequence, university research has required additional integration and transformation to become economically useful. --- [0] https://www.eetimes.com/podcasts/six-words-that-built-the-ic-industry-unhacking-the-2020-election-epic-leadership-in-an-epidemic/ https://www.eetimes.com/podcasts/six-words-that-built-the-ic... > Honeywell brought a lawsuit against us and said you can’t selectively choose people to divulge your technology to. It’s too important. And if you divulge it to anyone, you’ve got to divulge it to everybody. They filed a lawsuit, and the government came down on their side. And RCA basically had to open up all of its patents to everybody if they opened them up to anybody.

sampo 6y ago

> The other major case in point being NLP Speaking of which, Google Translate was published in 2006, but when did the "learning from data" approach became an accepted idea in machine translation? I think the earlier attempts at machine translation were more about trying to codify grammar rules in software, than doing statistical learning from large text corpuses? I remember in 2002, the approach of leaning protein substructures from data was already the best performing approach in the protein folding problem.

xbmcuser 6y ago
I think Netflix model of simplifying then translating should work better with internet forums and blog posts. Looking at how google works I was hoping someone in google wold adopt it and release a competing product against google translate https://arxiv.org/abs/2005.11197 https://arxiv.org/abs/2005.11197
Polygator 6y ago
Not really. Using a statistical approach to text modelling, specifically using Markov Chains, was proposed by Shannon in 1948. But yeah, there's a point in the 2000s where generative grammar/ symbolic approaches were pretty much left behind by NN methods. When we discuss Google's input in NLP, the most important contribution is certainly the "Attention is all you Need" paper, which paved the way for BERT and GTP (Alphafold also uses Attention networks, btw)
- adw 6y ago
  > generative grammar/ symbolic approaches were pretty much left behind by NN methods Which is the same thing as hand-engineered feature stacks being left behind in vision problems, really. The story in every field is more or less "you're not clever enough to engineer good features"; "you might be clever enough to define good symmetries for the feature space in which the features live... maybe" (convolutional neural networks in image problems); "... but maybe not even that" (attention mechanisms).
  - snovv_crash 6y ago

Tomminn 6y ago

Almost every single of the tens of thousand papers by the hundreds of tenured academics in the field of protein folding are made obsolete by 10 google engineers. This is what it's like when someone really moves the needle. And academic science cannot get it's head around it. And yet, none of these scientists will suffer any career consequences. Their irrelevant work will be healthily cited by all the other scientists who are doing and have done irrelevant work. They'll retcon a story in their lit reviews about how their irrelevant work led to this. The career consequences are saved for those who had their eye on the real ball for the last five years, but didn't get there first. For them, the comfortably irrelevant will have the gall to ask in accusatory tones: "What have you been doing these last 5 years?".

mnky9800n 6y ago
Did you miss the part where a bunch of the deepmind authors did phds and postdocs in protein folding related fields? They pushed the needle because they understood the field and the new tools. This is how pushing the needle works.
MMM13SFH 6y ago
fuck youre a retard, do you think this happened in a vacuum? pathetic wageslave

mensetmanusman 6y ago

Academia is for generating problem solvers. Teams are small and made of people who will be there for around 5 years. A better comparison would be to national labs, but they are tasked with projects that make no sense for industry to tackle. The system is working as intended, all players are needed. The team at Alphafold busted their chops in academia and went on to working on problems they could spend decades on.

mnky9800n 6y ago
People seem to forget that you need a system like academia that's allowed to fail. Most companies aren't allowed to fail when they need to have quarterly returns. Of course academia has become more and more competitive. But tbh I think the answer is that the funding hasn't increased equally with the number of quality people who could stay in academia. But who knows.
- Jach 6y ago
  Stability "like academia" is rich, given all we've heard about "publish or perish". Modern academia is a poor fit for increasingly any case you can think of besides maintaining the status of academia. But sure, there needs to be some stability and ability to "fail"/i.e. produce something worthless. Corporate research departments provide this -- if they didn't, they wouldn't have a research department and indeed many don't, nor do they need to, but this has little to do with quarterly returns. We've also seen a rise of VC-backed research startups (like DeepMind but many others) whose value proposition (to the VC) only makes sense if the goal is to demonstrate a research capacity to get them bought out by a big company, or as a moonshot to out-compete them on an actual product made possible by the research. Investing in these little research startups themselves is also giving companies a way to push research without having to deal with having the researchers as direct employees, and I'm sure makes some of the startup employees feel a bit safer since there's a separation of money and operation influence. One similarity with modern academia is it selects for those who can do good work but who are also good bureaucrats (write grant proposals well, advising politicians, etc), startups have a selection for good work + good at courting VCs. But the startup just needs a few of them, then they can hire people who just want to do good work. Another thing that makes corporate even better is they can occasionally spin off research developments into products, they can have some nice advances that only come when you try to productize, and among other reasons by not having to bother with external publishing (which takes time + fights with lawyers and business people) they can routinely be 10+ years ahead of whatever the state of the art in academia is.

calf 6y ago

Is this a scientific advance or a technological one though? Academia doesn't have the capital like industry or government to implement the latter. In America it's small small groups of young students led by a professor. Not full grown PhDs with Google levels of staff and money.

adwn 6y ago
> Is this a scientific advance or a technological one though? Where's the difference? (this is a genuine question, I'm not trying to trick you)
dwheeler 6y ago
I would claim that this is a technological advance that is likely to lead to many scientific advances. Perhaps a good analogy are the inventions of the microscope and telescope. They were advances in technology, which then led to advances in science. I don't know if this will have the same effect as the microscope and telescope, but it would be great if it did. It certainly seems extremely promising.

xpe 6y ago

It is not always either/or. It is not always A versus B. These fallacies are too common.

C4stor 6y ago

The deepmind research team is essentially all PhD though, so it seems academia isn't doing such a bad job.

softwaredoug 6y ago
Specifically the linked article wonders about the research environment of academia compared to industry. Why teams of hundreds in academia with their own super computing resources couldn’t make the same advances. He posits there’s something not great going on about how academic research environments make advances, the poor incentive structures, the abuse and burnout of PhDs, the lack of open sharing of findings, the obsession with publication quantity over quality... There’s a reason these PhDs at Google mind aren’t in academia after all doing the same work
- C4stor 6y ago
  Academia is doing a lot of advances every year. The fact it didn't make _this one_ is not really relevant to postulate that academia is inefficient. It happens that the team at deepmind is apparently pretty damn good at deep learning problems, so they're going faster than matching academia labs. It's not to say that academia has none of the problems you mentioned, but it's imo unreasonable to expect that, in a world where both public and private labs exist, only public ones would make advances.
- krull10 6y ago
  I would say that is most likely due to a massively higher salary and no teaching responsibilities. Academia can’t compete on salary with industry in AI / data science.

lambdamore 6y ago

Deepmind does not foster its future PhD, but, yes, they offer a better rewarding environment for those PhDs to flourish after they get the basic training.

jboggan 6y ago

I used to work in one of the top labs doing protein folding, in fact I recognize some lab mates who survived to have their own labs in this year's CASP top ten ranking. Something is a bit rotten though because I'd guess about half of us burned out of academia entirely, and the state of the art at the time was regrettable. I remember one model doing rather well except the weights of the various physical forces were wrong, so much that some signs were negative and what should have been electrostatic repulsion was actually electrostatic attraction between similar charges. This was 10 years ago of course, but all of the field improvements were relatively incremental and small until AlphaFold came along. Modern academia exists to further modern academia, no more and no less. I became disillusioned of any search for truth and progress during my time there, it was really just about showing up the Baker lab at the next CASP and getting the next round of grants secured.

softwaredoug 6y ago
This is exactly my point. There is something rotten in the research labs where society is not getting return on investment for basic research. If the goal is to just produce researchers that can work at corporate research labs, then I feel we could get more bang for the buck. If the goal is to do move research in the public good, something needs to change. Maybe it’s the fact there’s too little money out there, and it causes everyone to chase meager grant money. Or that there’s too much competition between groups. Or a million other reasons. But I’d love to see it fixed and have more faith in public investment in basic science.
- gwicks56 6y ago
  From my experience working as an engineer in Academia, there is a big problem in anything AI related, in that anyone decent can instantly increase their pay by 200 -1000% just by leaving for one of the big tech companies. Half the phD students don't even bother finishing, before being given an offer they can't refuse. How can academia compete with that, given sky high student debt, an extremely uncertain path to tenure etc etc
- PaulDavisThe1st 6y ago
  The problem is that people in academia and outside of it were saying the same thing in the 70s, 80s, 90s, 00's ... so if you really want to make the claim that "society is not getting return on investment for basic research", you need to claim with a straight face that this also applies to the last 50 years of academic basic research (at least). Alternatively, explain what has changed and when.

twic 6y ago

The academic groups working on this have a tiny fraction of the resources that Google do.

SiempreViernes 6y ago

The article then goes on to describe a not very general set of circumstances > But in part due to the canonicalization of CASP, protein structure prediction effectively has a two-year clock cycle, where separate research groups guard their discoveries until after CASP results are announced. and further noting > As I discussed earlier, it is clear that between the Xu and Zhang groups enough was known to develop a system that would have perhaps rivaled AlphaFold. Finally, and rather crushingly for your thesis, is the points made about the real industrial groups: > What is worse than academic groups getting scooped by DeepMind? The fact that the collective powers of Novartis, Pfizer, etc, with their hundreds of thousands (~million?) of employees, let an industrial lab that is a complete outsider to the field, with virtually no prior molecular sciences experience, come in and thoroughly beat them on a problem that is, quite frankly, of far greater importance to pharmaceuticals than it is to Alphabet. It is an indictment of the laughable “basic research” groups of these companies, which pay lip service to fundamental science but focus myopically on target-driven research that they managed to so badly embarrass themselves in this episode.

krull10 6y ago

Industry is not going to fund the overwhelming majority of research areas in biology, physics, chemistry, mathematics, etc. Data science and AI are an exception, where people in industry are much better paid, and can get access to much better resources that would be hard to afford in academia... It’s not surprising this type of advance came from an industry funded group. On the other hand, it is academia and its structure that has enabled so many other discoveries, for example, Crispr DNA tech, our understanding of gravitational waves, or the proof of the Poincaré conjecture.