14 ms·

No one should respect that blowhard. Please point me to the specifically accurate predictions he has made so far.

voitvoder 3y ago
Are you saying statements like "...and then AI takes over the whole galaxy and comes back to eats us!" doesn't sound rational? He is basically the next generation's Ray Kurzweil but instead of AI granting us biological immortality now AI is going to cause mass extinction. Instead of uploading consciousness to some digital nirvana we get the slightest variation on The Matrix. "And then it takes over the galaxy and uses the atoms in our bodies for batteries!!" With the way AI captures the imagination we can't but help take things to irrational extremes.
_heimdall 3y ago
Eliezer made early predictions that we would solve the protein folding problem when effectively everyone else said we either wouldn't solve it or it was way off. If I remember right he was making those claims around 2005 or so and predicting wed have it solved in something like 30 years. We solved it in 2017.
- _heimdall 3y ago
  For more context, Eliezer does himself a disservice focusing so hard on examples of how things could go horribly wrong but I haven't heard any compelling arguments against his core points. From everything I've seen from him, his concerns revolve around (1) a lack of a bounding function for the development of AI (2) a view that the alignment problem is unsolvable and (3) the risk that we won't recognize an AI was developed until it is sufficiently more intelligent than us to pose a fundamental risk.
- staticman2 3y ago
  If your prediction is "A.I. is similar to magic and can do anything and will become a god" you would of course think it can solve the protein folding problem... I'm not exactly sure that leads to the conclusion his track record is great. (There are many more things A.I. can't do than can do, I would think?)
  - _heimdall 3y ago
    His prediction was that humans would solve the protein folding problem, it didn't have to do with AI. I raised it only as an example of a prediction he made that most at the time thought had no chance. My point isn't to claim he's some genius that gets everything right, but there are examples of him making predictions widely viewed at the time of having no chance that turned out to be correct. Writing him off because people responding strongly to his predictions of AI destroying us all doesn't do anyone any good. His core arguments and predictions seem sound and I haven't yet found anyone to explain how we can solve alignment after the fact, for example
    - staticman2 3y ago
      Again, if my prediction is "every scientific advance will happen" then I can claim prescience by claiming every scientific advance is evidence of my great insight. That hardly makes me insightful. I'm writing him off because he isn't a practitioner. I can make things up about what will or won't happen on my own, as can anyone else. If I want to believe every wacky concept from science fiction will happen, I don't need a middleman to tell me it will.
      - _heimdall 3y ago
        Do you have an example of a prediction he made that didn't own out? I'm sure there are some, I wouldn't expect anyone making predictions to bat 1000, but throwing his opinion out even when there are examples of accurate predictions doesn't seem fair. It does feel risky to write off anyone that isn't a practitioner. There won't be any practitioners working on AI risk, its a theoretical pursuit by nature. The closest a practitioner could get is AI risk management, i.e. attempting to limit risk for AI we're already developing.
        yowlingcat 3y ago
        I think the problem that GP is getting at is the philosophical problem of apocalypticism or market perma-bears, which can be summed up in this satire of them: "So and so is a great market analyst! He's predicted 30 of the last 2 market crashes!" A broken clock can still be right twice a day, but it's still a broken clock. Predictions about the future which are so broad so as to be hopelessly incorrect should not be excused because part of the prediction surface ended up being correct; rather, the entirety of the prediction surface and accuracy should be taken into account to determine the overall accuracy of the prediction. Taken in that light, his being correct about his protein folding prediction feels like cold comfort when taken in context of his predictions, largely unsubstantiated, of AGI becoming a malevolent human extinction force at par with or of greater risk than that atomic bomb. It elides the very real conceptual limitations of current implementations of AI, such as it's inability to recognizing bidirectional associations (Tom is John's son means John is Tom's father), protecting against injection risks, "hallucinations" (inability to differentiate between when it is making something up and when it is not) and a variety of other existential limits towards its own power. These details, far from being inconvenient nuances that do not detract from his overall point, should actually serve to make us seriously skeptical of how well he understands the risks of the entity he is warning against. A more skeptical observer might even say (and I'm one of them) that much like fire and brimstone preachers, he is trying to create an unrealistic fear of a deific entity that cannot be reasoned against so that he can consolidate power by serving a "solution" that protects against it and curry popular favor. I find his approach to be the distasteful and manipulative approach of the common demagogue. The haughty, arrogant, insecure way that he treats the arguments of his critics (he simply blocks them on Twitter or condescends to them) make me even more skeptical of him. It makes me wonder if on some level he knows that his argument is not being made in good faith, and isn't trying to win an argument in good faith.
dang 3y ago
Please don't cross into name-calling. You can make your substantive points without that, and it's in your interest to, since the first part of a comment like this discredits the rest. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- yowlingcat 3y ago
  Fair enough. I could've phrased that better such as "I find his proto-Millenarian approach to be discursively vacuous, unnecessarily polemical, and not productive towards the answering the very real questions regarding negative externalities around the increasing rate of AI utilization in global society."

Are you saying statements like "...and then AI takes over the whole galaxy and comes back to eats us!" doesn't sound rational? He is basically the next generation's Ray Kurzweil but instead of AI granting us biological immortality now AI is going to cause mass extinction. Instead of uploading consciousness to some digital nirvana we get the slightest variation on The Matrix. "And then it takes over the galaxy and uses the atoms in our bodies for batteries!!" With the way AI captures the imagination we can't but help take things to irrational extremes.

_heimdall 3y ago

Eliezer made early predictions that we would solve the protein folding problem when effectively everyone else said we either wouldn't solve it or it was way off. If I remember right he was making those claims around 2005 or so and predicting wed have it solved in something like 30 years. We solved it in 2017.

_heimdall 3y ago
For more context, Eliezer does himself a disservice focusing so hard on examples of how things could go horribly wrong but I haven't heard any compelling arguments against his core points. From everything I've seen from him, his concerns revolve around (1) a lack of a bounding function for the development of AI (2) a view that the alignment problem is unsolvable and (3) the risk that we won't recognize an AI was developed until it is sufficiently more intelligent than us to pose a fundamental risk.
staticman2 3y ago
If your prediction is "A.I. is similar to magic and can do anything and will become a god" you would of course think it can solve the protein folding problem... I'm not exactly sure that leads to the conclusion his track record is great. (There are many more things A.I. can't do than can do, I would think?)
- _heimdall 3y ago
  His prediction was that humans would solve the protein folding problem, it didn't have to do with AI. I raised it only as an example of a prediction he made that most at the time thought had no chance. My point isn't to claim he's some genius that gets everything right, but there are examples of him making predictions widely viewed at the time of having no chance that turned out to be correct. Writing him off because people responding strongly to his predictions of AI destroying us all doesn't do anyone any good. His core arguments and predictions seem sound and I haven't yet found anyone to explain how we can solve alignment after the fact, for example
  - staticman2 3y ago
    Again, if my prediction is "every scientific advance will happen" then I can claim prescience by claiming every scientific advance is evidence of my great insight. That hardly makes me insightful. I'm writing him off because he isn't a practitioner. I can make things up about what will or won't happen on my own, as can anyone else. If I want to believe every wacky concept from science fiction will happen, I don't need a middleman to tell me it will.
    - _heimdall 3y ago
      Do you have an example of a prediction he made that didn't own out? I'm sure there are some, I wouldn't expect anyone making predictions to bat 1000, but throwing his opinion out even when there are examples of accurate predictions doesn't seem fair. It does feel risky to write off anyone that isn't a practitioner. There won't be any practitioners working on AI risk, its a theoretical pursuit by nature. The closest a practitioner could get is AI risk management, i.e. attempting to limit risk for AI we're already developing.
      - yowlingcat 3y ago
        I think the problem that GP is getting at is the philosophical problem of apocalypticism or market perma-bears, which can be summed up in this satire of them: "So and so is a great market analyst! He's predicted 30 of the last 2 market crashes!" A broken clock can still be right twice a day, but it's still a broken clock. Predictions about the future which are so broad so as to be hopelessly incorrect should not be excused because part of the prediction surface ended up being correct; rather, the entirety of the prediction surface and accuracy should be taken into account to determine the overall accuracy of the prediction. Taken in that light, his being correct about his protein folding prediction feels like cold comfort when taken in context of his predictions, largely unsubstantiated, of AGI becoming a malevolent human extinction force at par with or of greater risk than that atomic bomb. It elides the very real conceptual limitations of current implementations of AI, such as it's inability to recognizing bidirectional associations (Tom is John's son means John is Tom's father), protecting against injection risks, "hallucinations" (inability to differentiate between when it is making something up and when it is not) and a variety of other existential limits towards its own power. These details, far from being inconvenient nuances that do not detract from his overall point, should actually serve to make us seriously skeptical of how well he understands the risks of the entity he is warning against. A more skeptical observer might even say (and I'm one of them) that much like fire and brimstone preachers, he is trying to create an unrealistic fear of a deific entity that cannot be reasoned against so that he can consolidate power by serving a "solution" that protects against it and curry popular favor. I find his approach to be the distasteful and manipulative approach of the common demagogue. The haughty, arrogant, insecure way that he treats the arguments of his critics (he simply blocks them on Twitter or condescends to them) make me even more skeptical of him. It makes me wonder if on some level he knows that his argument is not being made in good faith, and isn't trying to win an argument in good faith.

dang 3y ago

Please don't cross into name-calling. You can make your substantive points without that, and it's in your interest to, since the first part of a comment like this discredits the rest. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

yowlingcat 3y ago
Fair enough. I could've phrased that better such as "I find his proto-Millenarian approach to be discursively vacuous, unnecessarily polemical, and not productive towards the answering the very real questions regarding negative externalities around the increasing rate of AI utilization in global society."