6 ms·

Mohammed AlQuraishi elaborated on this in his 2018 "What Just Happened?" blog post that was discussed on HN when the news was announced. [1] It's a fairly lon

Mohammed AlQuraishi elaborated on this in his 2018 "What Just Happened?" blog post that was discussed on HN when the news was announced. [1] It's a fairly long read, but he goes into a lot more detail as to why the problem isn't yet solved. Interestingly, he also notes that pharma and academia should feel some embarrassment from DeepMind's achievement: "What is worse than academic groups getting scooped by DeepMind? The fact that the collective powers of Novartis, Pfizer, etc, with their hundreds of thousands (~million?) of employees, let an industrial lab that is a complete outsider to the field, with virtually no prior molecular sciences experience, come in and thoroughly beat them on a problem that is, quite frankly, of far greater importance to pharmaceuticals than it is to Alphabet. It is an indictment of the laughable “basic research” groups of these companies, which pay lip service to fundamental science but focus myopically on target-driven research that they managed to so badly embarrass themselves in this episode." [1] https://moalquraishi.wordpress.com/2018/12/09/alphafold-casp13-what-just-happened/ https://moalquraishi.wordpress.com/2018/12/09/alphafold-casp... (Edited to clarify that the blog post was not recent.)

CydeWeys 6y ago
This blog post is two years old, whereas the linked blog post for this overall discussion is current.
- sp332 6y ago
  DeepMind's first entry into the competition was two years ago.
  - SamBam 6y ago
    Right, so it's not exactly pertinent to whether the most recent work "solved" (for whatever definition of solved) protein folding.
  - hu3 6y ago
    My impression is that DeepMind's performance have improved significantly since then.
  - CydeWeys 6y ago
    And they have improved significantly in the two years since then, so many if not all of these criticisms that apply to their efforts from two years ago may now be out of date.
- kevinskii 6y ago
  You're right, I should have pointed that out. I edited my original post to clarify. Thank you.
mhh__ 6y ago
> The fact that the collective powers of Novartis, Pfizer, etc, with their hundreds of thousands (~million?) of employees Deepmind have at least a few thousand employees, I'm willing to go out on a limb and say that the industry doesn't have "hundreds of thousands" of people working on protein folding alone.
- adtac 6y ago
  well, not everyone at DeepMind is working on protein folding either presumably (disclosure: I work at Google, but I know nothing about DeepMind other than whatever's public). I think the point they're making is that DeepMind has an order of magnitude or two fewer people working on this for a much shorter period of time than pharma companies.
  - est31 6y ago
    In the video on the announcement page, there was a short scene where they showed a Zoom call where the results were announced. I counted 20 people. The paper they published in Nature has 1 authors, so it matches up. Probably this is not the precise team size, but gives you a rough estimate. What's important is I think which people they hired, as well as Deepmind-internal infrastructures. You can just walk over to one of the world experts in deep learning and ask them how to address some issue. In pharma companies they have people who are really smart about drugs and biology, but ML experts would all have to be hired externally, and likely don't represent the top of the field as well as Deepmind does.
- tech-historian 6y ago
  Has Deepmind ever released how many employees it has? Several thousand seems much larger than I would have thought. Please cite your source on that.
  - Permit 6y ago
    Here's a June 2020 article that says they have ~1,000 people: https://www.cnbc.com/2020/06/05/google-deepmind-alphago-buzz-dissipates.html https://www.cnbc.com/2020/06/05/google-deepmind-alphago-buzz...
jeremysalwen 6y ago
This post is from 2018, when AlphaFold won the CASP competition, but nobody was claiming that it "solved" the problem, as you are now hearing in response to AlphaFold's performance at the 2020 competition.
- BHSPitMonkey 6y ago
  The same person who wrote said post in 2018 wrote about the 2020 AlphaFold results: > CASP14 #s just came out and they’re astounding—DeepMind looks to have solved protein structure prediction. Median GDT_TS went from 68.5 (CASP13) to 92.4!!!! Cf. their 2nd best CASP13 struct scored 92.8 (out of 100). Median RMSD is 2.1Å. I think it's over https://twitter.com/MoAlQuraishi/status/1333383634649313280 https://twitter.com/MoAlQuraishi/status/1333383634649313280
CamelCaseName 6y ago
That quote is from 2018. Here is what Mohammed AlQuraishi said in 2020: CASP14 #s just came out and they’re astounding—DeepMind looks to have solved protein structure prediction. Median GDT_TS went from 68.5 (CASP13) to 92.4!!!! Cf. their 2nd best CASP13 struct scored 92.8 (out of 100). Median RMSD is 2.1Å. I think it's over [0] and “I think it’s fair to say this will be very disruptive to the protein-structure-prediction field. I suspect many will leave the field as the core problem has arguably been solved,” he says. “It’s a breakthrough of the first order, certainly one of the most significant scientific results of my lifetime.” [1] [0] https://twitter.com/MoAlQuraishi/status/1333383634649313280 https://twitter.com/MoAlQuraishi/status/1333383634649313280 [1] https://www.nature.com/articles/d41586-020-03348-4 https://www.nature.com/articles/d41586-020-03348-4
- twic 6y ago
  I can't square this with the actual competition results. There are tons of targets where AlphaFold2 didn't score over 70%. It was typically only 10% ahead of the next best program. The metric everyone is looking at, GDT_TS, gives marks for atoms that are within eight angstroms of where they should be - by experimental standards, eight angstroms is a huge miss! There are targets where on some important metrics, like C-alpha RMSD, AlphaFold2 did worse than another program!
  - refulgentis 6y ago
    You're making extremely strong claims, counter to scientists with no incentive to praise Google being quoted with reactions such as expecting a mass exodus from computational biology. I, and I assume we, are open to hearing more but I'm not sure cherry-picking a couple examples is enough to credit your claims contra theirs. For instance, the gentlemen who was presented as a skeptic is instead shown to also say it's solved, and in reply to this you say "everyone" is looking at the "wrong" metric, and it allows errors of 8+ atom widths - he notes the median error is 2.1 angstroms, or 2.1 atom widths.
    - byecomputer 6y ago
      I feel like there's a certain je ne sais quoi to the comments Google employees write where I can just immediately sense them without even needing to check.
zamadatix 6y ago
I think it's arguably the exact opposite. It would have been surprising if medical companies had proven machine learning can model protein folding so much better before the world's leading AI research group could. The demonstration was one of machine learning research being applied to medical problems not medical research being applied to machine learning problems so DeepMind was the one with the head start on expertise.
- tachyonbeam 6y ago
  And let's not forget that deep learning started really exploding and becoming mainstream around 2015-2016. I'm not sure how you could expect someone like Pfizer to hire top experts and build a competitive lab in such a short time, particularly since, Pfizer would want these people to focus exclusively on biotech, which would not be that attractive to most deep learning researchers.
  - nl 6y ago
    OpenAI was founded in Dec 2015. They have a smaller budget than Pfizer and work on less practical problems. It's a lack of vision and leadership from pharma labs.
    - tachyonbeam 6y ago
      > It's a lack of vision and leadership from pharma labs. That's probably true, but I don't think Pfizer could have beat DeepMind, even if they had really tried. DeepMind is in a unique position to recruit lots of young deep learning researchers.
_Wintermute 6y ago
Pharma companies don't feel any embarrassment because it's really not much of a priority for them. Protein structure determination simply isn’t a rate-limiting step in drug discovery in general. [1] [1] https://blogs.sciencemag.org/pipeline/archives/2019/09/25/whats-crucial-and-what-isnt https://blogs.sciencemag.org/pipeline/archives/2019/09/25/wh...
- dekhn 6y ago
  That is true but every pharma has a structural biology department that solves product-related crystal structures with the intent of using that data to speed development. It's an established thing, $100+M/year budget per company. Generally well-respected and considered some of the most important IP the company can generate. They also have theory groups which try to do structure prediction and drug docking but I don't think those groups get any respect inside the companies any more.
banjomet 6y ago
I think it is also very embarrassing to IBM, which has been trying to get into healthcare with Watson.
inopinatus 6y ago
Counterpoint: no-one needs to feel any embarrassment. The advancement of human understanding is not a boxing match. The author acknowledges as much in the preamble, "my mood lifted during the meeting due to the general excitement and quality of discussions, [...] my tribal reflexes gave way to a cooler and more rational assessment".
- stjohnswarts 6y ago
  Tell that to said company's stockholders and get their response :) . In the big picture though is new, correct knowledge is always welcome to the pool.
visarga 6y ago
I bet they are hiring BERT-ologists now.

CydeWeys 6y ago

This blog post is two years old, whereas the linked blog post for this overall discussion is current.

sp332 6y ago
DeepMind's first entry into the competition was two years ago.
- SamBam 6y ago
  Right, so it's not exactly pertinent to whether the most recent work "solved" (for whatever definition of solved) protein folding.
- hu3 6y ago
  My impression is that DeepMind's performance have improved significantly since then.
- CydeWeys 6y ago
  And they have improved significantly in the two years since then, so many if not all of these criticisms that apply to their efforts from two years ago may now be out of date.
kevinskii 6y ago
You're right, I should have pointed that out. I edited my original post to clarify. Thank you.

mhh__ 6y ago

> The fact that the collective powers of Novartis, Pfizer, etc, with their hundreds of thousands (~million?) of employees Deepmind have at least a few thousand employees, I'm willing to go out on a limb and say that the industry doesn't have "hundreds of thousands" of people working on protein folding alone.

adtac 6y ago
well, not everyone at DeepMind is working on protein folding either presumably (disclosure: I work at Google, but I know nothing about DeepMind other than whatever's public). I think the point they're making is that DeepMind has an order of magnitude or two fewer people working on this for a much shorter period of time than pharma companies.
- est31 6y ago
  In the video on the announcement page, there was a short scene where they showed a Zoom call where the results were announced. I counted 20 people. The paper they published in Nature has 1 authors, so it matches up. Probably this is not the precise team size, but gives you a rough estimate. What's important is I think which people they hired, as well as Deepmind-internal infrastructures. You can just walk over to one of the world experts in deep learning and ask them how to address some issue. In pharma companies they have people who are really smart about drugs and biology, but ML experts would all have to be hired externally, and likely don't represent the top of the field as well as Deepmind does.
tech-historian 6y ago
Has Deepmind ever released how many employees it has? Several thousand seems much larger than I would have thought. Please cite your source on that.
- Permit 6y ago
  Here's a June 2020 article that says they have ~1,000 people: https://www.cnbc.com/2020/06/05/google-deepmind-alphago-buzz-dissipates.html https://www.cnbc.com/2020/06/05/google-deepmind-alphago-buzz...

jeremysalwen 6y ago

This post is from 2018, when AlphaFold won the CASP competition, but nobody was claiming that it "solved" the problem, as you are now hearing in response to AlphaFold's performance at the 2020 competition.

BHSPitMonkey 6y ago
The same person who wrote said post in 2018 wrote about the 2020 AlphaFold results: > CASP14 #s just came out and they’re astounding—DeepMind looks to have solved protein structure prediction. Median GDT_TS went from 68.5 (CASP13) to 92.4!!!! Cf. their 2nd best CASP13 struct scored 92.8 (out of 100). Median RMSD is 2.1Å. I think it's over https://twitter.com/MoAlQuraishi/status/1333383634649313280 https://twitter.com/MoAlQuraishi/status/1333383634649313280

CamelCaseName 6y ago

That quote is from 2018. Here is what Mohammed AlQuraishi said in 2020: CASP14 #s just came out and they’re astounding—DeepMind looks to have solved protein structure prediction. Median GDT_TS went from 68.5 (CASP13) to 92.4!!!! Cf. their 2nd best CASP13 struct scored 92.8 (out of 100). Median RMSD is 2.1Å. I think it's over [0] and “I think it’s fair to say this will be very disruptive to the protein-structure-prediction field. I suspect many will leave the field as the core problem has arguably been solved,” he says. “It’s a breakthrough of the first order, certainly one of the most significant scientific results of my lifetime.” [1] [0] https://twitter.com/MoAlQuraishi/status/1333383634649313280 https://twitter.com/MoAlQuraishi/status/1333383634649313280 [1] https://www.nature.com/articles/d41586-020-03348-4 https://www.nature.com/articles/d41586-020-03348-4

twic 6y ago
I can't square this with the actual competition results. There are tons of targets where AlphaFold2 didn't score over 70%. It was typically only 10% ahead of the next best program. The metric everyone is looking at, GDT_TS, gives marks for atoms that are within eight angstroms of where they should be - by experimental standards, eight angstroms is a huge miss! There are targets where on some important metrics, like C-alpha RMSD, AlphaFold2 did worse than another program!
- refulgentis 6y ago
  You're making extremely strong claims, counter to scientists with no incentive to praise Google being quoted with reactions such as expecting a mass exodus from computational biology. I, and I assume we, are open to hearing more but I'm not sure cherry-picking a couple examples is enough to credit your claims contra theirs. For instance, the gentlemen who was presented as a skeptic is instead shown to also say it's solved, and in reply to this you say "everyone" is looking at the "wrong" metric, and it allows errors of 8+ atom widths - he notes the median error is 2.1 angstroms, or 2.1 atom widths.
  - byecomputer 6y ago
    I feel like there's a certain je ne sais quoi to the comments Google employees write where I can just immediately sense them without even needing to check.

zamadatix 6y ago

I think it's arguably the exact opposite. It would have been surprising if medical companies had proven machine learning can model protein folding so much better before the world's leading AI research group could. The demonstration was one of machine learning research being applied to medical problems not medical research being applied to machine learning problems so DeepMind was the one with the head start on expertise.

tachyonbeam 6y ago
And let's not forget that deep learning started really exploding and becoming mainstream around 2015-2016. I'm not sure how you could expect someone like Pfizer to hire top experts and build a competitive lab in such a short time, particularly since, Pfizer would want these people to focus exclusively on biotech, which would not be that attractive to most deep learning researchers.
- nl 6y ago
  OpenAI was founded in Dec 2015. They have a smaller budget than Pfizer and work on less practical problems. It's a lack of vision and leadership from pharma labs.
  - tachyonbeam 6y ago
    > It's a lack of vision and leadership from pharma labs. That's probably true, but I don't think Pfizer could have beat DeepMind, even if they had really tried. DeepMind is in a unique position to recruit lots of young deep learning researchers.

_Wintermute 6y ago

Pharma companies don't feel any embarrassment because it's really not much of a priority for them. Protein structure determination simply isn’t a rate-limiting step in drug discovery in general. [1] [1] https://blogs.sciencemag.org/pipeline/archives/2019/09/25/whats-crucial-and-what-isnt https://blogs.sciencemag.org/pipeline/archives/2019/09/25/wh...

dekhn 6y ago
That is true but every pharma has a structural biology department that solves product-related crystal structures with the intent of using that data to speed development. It's an established thing, $100+M/year budget per company. Generally well-respected and considered some of the most important IP the company can generate. They also have theory groups which try to do structure prediction and drug docking but I don't think those groups get any respect inside the companies any more.

banjomet 6y ago

I think it is also very embarrassing to IBM, which has been trying to get into healthcare with Watson.

inopinatus 6y ago

Counterpoint: no-one needs to feel any embarrassment. The advancement of human understanding is not a boxing match. The author acknowledges as much in the preamble, "my mood lifted during the meeting due to the general excitement and quality of discussions, [...] my tribal reflexes gave way to a cooler and more rational assessment".

stjohnswarts 6y ago
Tell that to said company's stockholders and get their response :) . In the big picture though is new, correct knowledge is always welcome to the pool.

visarga 6y ago

I bet they are hiring BERT-ologists now.