7 ms·

This is DeepMind's modus operandi. Every press release is just utterly hyperbolic nonsense that doesn't withstand the slightest scrutiny. AlphaGo, AlphaFold, Al

This is DeepMind's modus operandi. Every press release is just utterly hyperbolic nonsense that doesn't withstand the slightest scrutiny. AlphaGo, AlphaFold, AlphaDev... they've done literally nothing to improve the human condition, and may have actually made things worse. Go players have DECREASED their Elo after playing AlphaGo (or just quit the game altogether). I would be embarrassed to be associated with DeepMind.

blazespin 3y ago
It's a bit surprising how poorly DeepMind has lived up to their hype. But they're an OK lab, maybe a bit overly vain.
- hugopenmir 3y ago
  I can’t believe this
- TaupeRanger 3y ago
  It's probably very fun to be at DeepMind, I just don't think I'd want to be a part of the cringey hype machine.
  - blazespin 3y ago
    I bet it really sucks, tbh. They did all this over promising and now the only way they can deliver is by grifting like this. That sounds really stressful to me.
    - lurknot 3y ago
      ya bet it sucks making $500-800k/year comp getting access to the best hardware and google datasets
      - blazespin 3y ago
        I've seen people who get promoted above their band, they are not happy campers.
        blitzar 3y ago
        Oh god, promoted as well, I for one am glad they are bearing the burden for me.
        jimsimmons 3y ago
        Why not. It’s not like they have deadlines
        29athrowaway 3y ago
        They still have to publish something in major journals and have presence in major conferences.
        jprd 3y ago
        a.k.a. - they still care enough to have some semblance of shame. There's the rub.
        29athrowaway 3y ago
        At least they've produced tangible value unlike black holes of money like the Human Brain project which has delivered close to nothing in multiple decades despite billions of dollars in investment.
vosper 3y ago
> Go players have DECREASED their ELO after playing AlphaGo (or just quit the game altogether) Can you explain this for someone unfamiliar with the game?
- scotty79 3y ago
  Maybe they just lost so their ELO went down? Or do loses against AI not count?
  - rcxdude 3y ago
    Either way, in ELO it depends on the ELO of your opponent, and so a loss against an AI with an accurate (very high) ELO is not going to lose you much unless you're one of the best players in the world (heck, in chess, Magnus Carlsen's ELO would still go down by literally nothing from losing against stockfish).
- dfan 3y ago
  I am familiar with the game and I cannot explain it. Go is not typically rated with the Elo system, and the quality of top-level human play has increased since 2016.
  - nahstra 3y ago
    I also play some, and yeah that's incorrect. Also, there was recently a bunch of hype about an adversarial strategy that could beat AIs w/o deep readout (i.e. only using the 'policy network' nonlinear ML stuff and not enough actual calculation). Here's a vid of an amateur doing it. https://www.youtube.com/watch?v=H4DvCj4ySKM https://www.youtube.com/watch?v=H4DvCj4ySKM EDIT: Also, if you want to get into it, Micheal Redmond's Go TV on youtube has an amazing beginner playlist, watch some of that then maybe blacktoplay.com and if you likey play :)
  - 29athrowaway 3y ago
    That is correct but in game servers, ranks (e.g.: 1 kyu, 1 dan) are computed via Elo-like means. https://forums.online-go.com/t/how-does-the-rating-system-work/23407/3 https://forums.online-go.com/t/how-does-the-rating-system-wo...
malux85 3y ago
I thought the same thing - it smacks of desperation at the moment, any tiny win is exaggerated . It’s not hard to see why, with the emergence (ha) of OpenAI, Midjourney and all of this generative modelling, what has DeepMind done? I imagine the execs at Google are asking them some very probing questions on their mediocre performance over the last 5 years.
- chaxor 3y ago
  Deepmind has done quite an enormous amount actually, but it's been in academia not in the commercial product sphere. Just because something is not on a little web page available to average Joe's does not mean there isn't value in it. For example, Deepmind's work towards estimating quantum properties of materials via density functional theory may not be the best toy for your grandma to play around with, but it certainly does move academia way further ahead of where it once was.
  - TaupeRanger 3y ago
    Yes, people often mention the number scientific citations that mention AlphaFold as "proof" of its value. Unfortunately, padding researcher CVs is not a net positive for humanity, and so "moving academia further ahead" (by what metric?) is not necessarily a desirable or worthwhile goal if your definition of "ahead" is sufficiently warped as such. Perhaps, one day, the first real human being will be helped by medicine that couldn't have been found/created without AlphaFold. Unless a scientific endeavor is actually useful to humanity, who cares if it "moves ahead"?
  - malux85 3y ago
    I run atomictessellator.com and have been working on many different implementations of density functional theory for the last 10 years, as well as working closely with professors at Stanford university and Oxford university of using advanced, non-static geometry data structure for density functional theory for multiple years, this is a subject I know a LOT about, so I’m glad you brought it up. Deep minds work on Density functional theory was complete rubbish, and everyone in computational chemistry knows it. They simply modelled static geometry and overfit their data, we wanted this methodology to work, computing DFT is expensive, and we did multiple months of rigorous work and the reality of the situation is that a bunch of machine learning engineers with a glancing amount of chemistry knowledge, made approximations that were way too naive, and announced it as a huge success in their typical fashion. What they then count on is people not having enough knowledge of DFT / Quantum property prediction to query their work and make claims like “it certainly does move academia way further ahead” - which is total rubbish. In what way? Why aren’t these models being used in ab initio simulators now? The answer to that is simple: they are not revolutionary, in fact they are not even useful.
rcxdude 3y ago
I think this is a bit far in the other direction. Deepmind's stuff is often deeply impressive, they just have a tendency to exaggerate on top of that.
- jprd 3y ago
  +1 There must be a "demonstrate (1) DeepMind #Win per <interval>" requirement somewhere that gets the once-over from the marketing dept. to meet some MBOs.
programmer_dude 3y ago
BTW Elo is not an abbreviation it is a persons name: https://en.wikipedia.org/wiki/Arpad_Elo https://en.wikipedia.org/wiki/Arpad_Elo
gdy 3y ago
"may have actually made things worse. Go players..." Go players are using AI to get better at Go.
- TaupeRanger 3y ago
  That claim is often made, and never substantiated.
  - nahstra 3y ago
    https://www.amazon.com/How-Play-Way-Explained-illustrative/dp/3940563781/ref=pd_bxgy_vft_none_img_sccl_1/144-5928705-1490518?pd_rd_w=omkkP&content-id=amzn1.sym.26a5c67f-1a30-486b-bb90-b523ad38d5a0&pf_rd_p=26a5c67f-1a30-486b-bb90-b523ad38d5a0&pf_rd_r=800MWH438259C7VFRWCE&pd_rd_wg=amBgs&pd_rd_r=cad7061a-a1f1-4632-becf-94867b2fb38c&pd_rd_i=3940563781&psc=1 https://www.amazon.com/How-Play-Way-Explained-illustrative/d...
  - gdy 3y ago
    Just ask any professional Go player.
    - TaupeRanger 3y ago
      Anecdotes are not data. Look at the game statistics. There is zero evidence that players are playing at a higher level since the inception of AlphaGo.
  - digging 3y ago
    It's made here in response to a claim that Go players are made worse by practicing with AlphaGo, which is also unsubstantiated.
    - TaupeRanger 3y ago
      Wrong. The claim was never made that players got worse, only that their ratings dropped, which is empirically true. After playing AlphaGo, for example, Ke Jie dropped in the rankings and was quickly taken out of 1st place overall. The overall point though, is that AlphaGo produced nothing of value for humans, since there's also no evidence that players have improved since AlphaGo's creation. Factoring in the immense cost and human brainpower wasted on creating a superhuman perfect-information-game-playing program, and it's easily a net negative for humanity.
29athrowaway 3y ago
Around the time of the AlphaGo challenge and afterwards... 1) you could see increased activity in go clubs and online go servers 2) the analysis of the games published by Deepmind has resulted in interesting "discoveries" (or rediscoveries) and changes to what is considered joseki. 3) many people started analyzing their kifus using AI, to find fluctuations in estimated win rate across moves. So I disagree entirely
chaxor 3y ago
This is unbelievably wrong. Deepmind has probably the best academic group in RL. The difference between Deepmind and OpenAI is that Deepmind favors academic endeavours and novelty much more, while completely ignoring any commercialization or products, while OpenAI is the stark opposite in that they almost entirely focus on products first, and typically their academic endeavours are just slight modifications of other works scaled up. Don't get me wrong, Sutskyver (et al) has done incredibly good work previously, but when it comes to the products, they're much more engineering and marketing polish than scientific endeavours. Botvinick's work on metaRL for example is an interesting direction that Deepmind has shown that few other companies that are only interested in engineering would venture towards.
- mtlmtlmtlmtl 3y ago
  That's the thing with Deepmind though. They almost never actually end up advancing things because A) they don't release weights and B) they don't usually develop their ideas into useful tools themselves, forcing others to redo all their work. So yeah, it's essentially a PoC PR stunt factory. Just look at AlphaZero. They make a huge deal about a suspiciously set up match against Stockfish. Supposedly revolutionising computer chess. But the problem is the computer chess community had to redo all of the work, including all the training to build Leela Chess Zero. Due to lack of Google-sized datacentres the training took years to catch up to the weights in AlphaZero. Same thing with AlphaGo, same thing with transformers. Now, in AI, usually getting a proof of concept is the easy part. Developing that into an idea that actually works in real world situations is usually the hardest part. I completely reject your idea that somehow the work by OpenAI is less worthy of recognition. I think that's just nonsense. And surely, Google created Deepmind to actually make them product ideas, not create new competitors, which is what has happened.
  - qumeric 3y ago
    > Now, in AI, usually getting a proof of concept is the easy part. Developing that into an idea that actually works in real world situations is usually the hardest part. I disagree. Of course there is a lot of engineering involved and it's also very important but it's much easier to rebuild things based on published research than develop novel ideas.
chpatrick 3y ago
Massive improvements in protein folding do nothing to improve the human condition? What?
- bamboozled 3y ago
  To be fair I have read that while impressive, still has little practical application?
- TaupeRanger 3y ago
  Name one improvement. Just one thing that has ACTUALLY helped real life humans and been a net positive since AlphaFold's inception 5 years ago.
  - bglazer 3y ago
    We're talking about biomedical science here. Things move slowly because the domain is exceptionally complex and human lives are in the balance. AlphaFold catapulted protein structure prediction forward, and it's hard to overstate how important understanding protein structure is in modern drug development As an example of how this will be used to help actual people, here's a paper that uses AlphaFold to identify the parts of cancer-associated proteins that interact with each other. https://onlinelibrary.wiley.com/doi/full/10.1002/pro.4479 https://onlinelibrary.wiley.com/doi/full/10.1002/pro.4479 The obvious next step is to develop drugs that disrupt these interactions and thereby disrupt cancer. But, it's going to take years, maybe decades before any drug resulting from this research is in actual patients. There are dozens of other papers like this.
    - TaupeRanger 3y ago
      And yet, even after 5 years there's no sign of any real, meaningful drugs, even just Phase I trials. In 5 - 10 years (which will be 10 to 15 years after AlphaFold was released) I am willing to bet real money that there will be zero drugs discovered by AlphaFold that meet the following criteria: 1) The drug couldn't have been discovered without AlphaFold 2) It has been proven to reduce all cause mortality (the thing real patients actually care about) in a randomized controlled clinical trial BETTER than the prior standard of care (or significantly more cheaply, or with significantly reduced side effects)

blazespin 3y ago

It's a bit surprising how poorly DeepMind has lived up to their hype. But they're an OK lab, maybe a bit overly vain.

hugopenmir 3y ago
I can’t believe this
TaupeRanger 3y ago
It's probably very fun to be at DeepMind, I just don't think I'd want to be a part of the cringey hype machine.
- blazespin 3y ago
  I bet it really sucks, tbh. They did all this over promising and now the only way they can deliver is by grifting like this. That sounds really stressful to me.
  - lurknot 3y ago
    ya bet it sucks making $500-800k/year comp getting access to the best hardware and google datasets
    - blazespin 3y ago
      I've seen people who get promoted above their band, they are not happy campers.
      - blitzar 3y ago
        Oh god, promoted as well, I for one am glad they are bearing the burden for me.
      - jimsimmons 3y ago
        Why not. It’s not like they have deadlines
        29athrowaway 3y ago
        They still have to publish something in major journals and have presence in major conferences.
        jprd 3y ago
        a.k.a. - they still care enough to have some semblance of shame. There's the rub.
        29athrowaway 3y ago
        At least they've produced tangible value unlike black holes of money like the Human Brain project which has delivered close to nothing in multiple decades despite billions of dollars in investment.

vosper 3y ago

> Go players have DECREASED their ELO after playing AlphaGo (or just quit the game altogether) Can you explain this for someone unfamiliar with the game?

scotty79 3y ago
Maybe they just lost so their ELO went down? Or do loses against AI not count?
- rcxdude 3y ago
  Either way, in ELO it depends on the ELO of your opponent, and so a loss against an AI with an accurate (very high) ELO is not going to lose you much unless you're one of the best players in the world (heck, in chess, Magnus Carlsen's ELO would still go down by literally nothing from losing against stockfish).
dfan 3y ago
I am familiar with the game and I cannot explain it. Go is not typically rated with the Elo system, and the quality of top-level human play has increased since 2016.
- nahstra 3y ago
  I also play some, and yeah that's incorrect. Also, there was recently a bunch of hype about an adversarial strategy that could beat AIs w/o deep readout (i.e. only using the 'policy network' nonlinear ML stuff and not enough actual calculation). Here's a vid of an amateur doing it. https://www.youtube.com/watch?v=H4DvCj4ySKM https://www.youtube.com/watch?v=H4DvCj4ySKM EDIT: Also, if you want to get into it, Micheal Redmond's Go TV on youtube has an amazing beginner playlist, watch some of that then maybe blacktoplay.com and if you likey play :)
- 29athrowaway 3y ago
  That is correct but in game servers, ranks (e.g.: 1 kyu, 1 dan) are computed via Elo-like means. https://forums.online-go.com/t/how-does-the-rating-system-work/23407/3 https://forums.online-go.com/t/how-does-the-rating-system-wo...

malux85 3y ago

I thought the same thing - it smacks of desperation at the moment, any tiny win is exaggerated . It’s not hard to see why, with the emergence (ha) of OpenAI, Midjourney and all of this generative modelling, what has DeepMind done? I imagine the execs at Google are asking them some very probing questions on their mediocre performance over the last 5 years.

chaxor 3y ago
Deepmind has done quite an enormous amount actually, but it's been in academia not in the commercial product sphere. Just because something is not on a little web page available to average Joe's does not mean there isn't value in it. For example, Deepmind's work towards estimating quantum properties of materials via density functional theory may not be the best toy for your grandma to play around with, but it certainly does move academia way further ahead of where it once was.
- TaupeRanger 3y ago
  Yes, people often mention the number scientific citations that mention AlphaFold as "proof" of its value. Unfortunately, padding researcher CVs is not a net positive for humanity, and so "moving academia further ahead" (by what metric?) is not necessarily a desirable or worthwhile goal if your definition of "ahead" is sufficiently warped as such. Perhaps, one day, the first real human being will be helped by medicine that couldn't have been found/created without AlphaFold. Unless a scientific endeavor is actually useful to humanity, who cares if it "moves ahead"?
- malux85 3y ago
  I run atomictessellator.com and have been working on many different implementations of density functional theory for the last 10 years, as well as working closely with professors at Stanford university and Oxford university of using advanced, non-static geometry data structure for density functional theory for multiple years, this is a subject I know a LOT about, so I’m glad you brought it up. Deep minds work on Density functional theory was complete rubbish, and everyone in computational chemistry knows it. They simply modelled static geometry and overfit their data, we wanted this methodology to work, computing DFT is expensive, and we did multiple months of rigorous work and the reality of the situation is that a bunch of machine learning engineers with a glancing amount of chemistry knowledge, made approximations that were way too naive, and announced it as a huge success in their typical fashion. What they then count on is people not having enough knowledge of DFT / Quantum property prediction to query their work and make claims like “it certainly does move academia way further ahead” - which is total rubbish. In what way? Why aren’t these models being used in ab initio simulators now? The answer to that is simple: they are not revolutionary, in fact they are not even useful.

rcxdude 3y ago

I think this is a bit far in the other direction. Deepmind's stuff is often deeply impressive, they just have a tendency to exaggerate on top of that.

jprd 3y ago
+1 There must be a "demonstrate (1) DeepMind #Win per <interval>" requirement somewhere that gets the once-over from the marketing dept. to meet some MBOs.

programmer_dude 3y ago

BTW Elo is not an abbreviation it is a persons name: https://en.wikipedia.org/wiki/Arpad_Elo https://en.wikipedia.org/wiki/Arpad_Elo

gdy 3y ago

"may have actually made things worse. Go players..." Go players are using AI to get better at Go.

TaupeRanger 3y ago
That claim is often made, and never substantiated.
- nahstra 3y ago
  https://www.amazon.com/How-Play-Way-Explained-illustrative/dp/3940563781/ref=pd_bxgy_vft_none_img_sccl_1/144-5928705-1490518?pd_rd_w=omkkP&content-id=amzn1.sym.26a5c67f-1a30-486b-bb90-b523ad38d5a0&pf_rd_p=26a5c67f-1a30-486b-bb90-b523ad38d5a0&pf_rd_r=800MWH438259C7VFRWCE&pd_rd_wg=amBgs&pd_rd_r=cad7061a-a1f1-4632-becf-94867b2fb38c&pd_rd_i=3940563781&psc=1 https://www.amazon.com/How-Play-Way-Explained-illustrative/d...
- gdy 3y ago
  Just ask any professional Go player.
  - TaupeRanger 3y ago
    Anecdotes are not data. Look at the game statistics. There is zero evidence that players are playing at a higher level since the inception of AlphaGo.
- digging 3y ago
  It's made here in response to a claim that Go players are made worse by practicing with AlphaGo, which is also unsubstantiated.
  - TaupeRanger 3y ago
    Wrong. The claim was never made that players got worse, only that their ratings dropped, which is empirically true. After playing AlphaGo, for example, Ke Jie dropped in the rankings and was quickly taken out of 1st place overall. The overall point though, is that AlphaGo produced nothing of value for humans, since there's also no evidence that players have improved since AlphaGo's creation. Factoring in the immense cost and human brainpower wasted on creating a superhuman perfect-information-game-playing program, and it's easily a net negative for humanity.

29athrowaway 3y ago

Around the time of the AlphaGo challenge and afterwards... 1) you could see increased activity in go clubs and online go servers 2) the analysis of the games published by Deepmind has resulted in interesting "discoveries" (or rediscoveries) and changes to what is considered joseki. 3) many people started analyzing their kifus using AI, to find fluctuations in estimated win rate across moves. So I disagree entirely

chaxor 3y ago

This is unbelievably wrong. Deepmind has probably the best academic group in RL. The difference between Deepmind and OpenAI is that Deepmind favors academic endeavours and novelty much more, while completely ignoring any commercialization or products, while OpenAI is the stark opposite in that they almost entirely focus on products first, and typically their academic endeavours are just slight modifications of other works scaled up. Don't get me wrong, Sutskyver (et al) has done incredibly good work previously, but when it comes to the products, they're much more engineering and marketing polish than scientific endeavours. Botvinick's work on metaRL for example is an interesting direction that Deepmind has shown that few other companies that are only interested in engineering would venture towards.

mtlmtlmtlmtl 3y ago
That's the thing with Deepmind though. They almost never actually end up advancing things because A) they don't release weights and B) they don't usually develop their ideas into useful tools themselves, forcing others to redo all their work. So yeah, it's essentially a PoC PR stunt factory. Just look at AlphaZero. They make a huge deal about a suspiciously set up match against Stockfish. Supposedly revolutionising computer chess. But the problem is the computer chess community had to redo all of the work, including all the training to build Leela Chess Zero. Due to lack of Google-sized datacentres the training took years to catch up to the weights in AlphaZero. Same thing with AlphaGo, same thing with transformers. Now, in AI, usually getting a proof of concept is the easy part. Developing that into an idea that actually works in real world situations is usually the hardest part. I completely reject your idea that somehow the work by OpenAI is less worthy of recognition. I think that's just nonsense. And surely, Google created Deepmind to actually make them product ideas, not create new competitors, which is what has happened.
- qumeric 3y ago
  > Now, in AI, usually getting a proof of concept is the easy part. Developing that into an idea that actually works in real world situations is usually the hardest part. I disagree. Of course there is a lot of engineering involved and it's also very important but it's much easier to rebuild things based on published research than develop novel ideas.

chpatrick 3y ago

Massive improvements in protein folding do nothing to improve the human condition? What?

bamboozled 3y ago
To be fair I have read that while impressive, still has little practical application?
TaupeRanger 3y ago
Name one improvement. Just one thing that has ACTUALLY helped real life humans and been a net positive since AlphaFold's inception 5 years ago.
- bglazer 3y ago
  We're talking about biomedical science here. Things move slowly because the domain is exceptionally complex and human lives are in the balance. AlphaFold catapulted protein structure prediction forward, and it's hard to overstate how important understanding protein structure is in modern drug development As an example of how this will be used to help actual people, here's a paper that uses AlphaFold to identify the parts of cancer-associated proteins that interact with each other. https://onlinelibrary.wiley.com/doi/full/10.1002/pro.4479 https://onlinelibrary.wiley.com/doi/full/10.1002/pro.4479 The obvious next step is to develop drugs that disrupt these interactions and thereby disrupt cancer. But, it's going to take years, maybe decades before any drug resulting from this research is in actual patients. There are dozens of other papers like this.
  - TaupeRanger 3y ago
    And yet, even after 5 years there's no sign of any real, meaningful drugs, even just Phase I trials. In 5 - 10 years (which will be 10 to 15 years after AlphaFold was released) I am willing to bet real money that there will be zero drugs discovered by AlphaFold that meet the following criteria: 1) The drug couldn't have been discovered without AlphaFold 2) It has been proven to reduce all cause mortality (the thing real patients actually care about) in a randomized controlled clinical trial BETTER than the prior standard of care (or significantly more cheaply, or with significantly reduced side effects)