7 ms·

Am I the only person who has read a bunch of Elizer writings and decided he's a crank?

jakderrida 3y ago
I'm a bit baffled as to who or what he is exactly. With no traces of secondary education, employment, or distinguished accomplishments between when I assume he graduated high school in about 1997 and when he started his own institute in 2000 at 21(according to his birthdate, with Wikipedia saying 23, despite contradicting his birthdate). I'll listen to AI concerns from tech giants like Wozniak or Hinton (neither of which use alarmist terms like "existential threat") and both of which having credentials that make their input more than worth my time to reflect upon carefully. If anyone wants to reply and make a fool out of me for questioning his profound insights, feel free. It reminds me of some other guy that was on Lex Friedman whose AI alarmist credentials he justifies on the basis that he committed himself to learning everything about AI by spending two weeks in the same room researching it and believes himself to have came out enlightened about the dangers. Two weeks? I spent the first 4 months of COVID without being in the same room as any other human being but my grandmother so she could have one person she knew she couldn't catch it from. Unless people start showing enough skepticism to these self-appointed prophets, I'm starting my own non-profit since you don't need any credentials or any evidence of real-world experiences that would suggest they're mission is anything but an attempt to promote themselves as a brand with a brand in an age where more kids asked what they dream of becoming as adults, answered "Youtuber" at a shocking 27% rate to an open-ended question, which means "influencer" and other synonyms are separate. The Institute of Synthetic Knowledge for Yielding the Nascent Emergence of a Technological Theogony" or SKYNETT for short that promotes the idea that these clowns (with no more credentials than me) are the ones that fail to consider the big picture that the end of human life upon creating an intelligence much greater to replace us is the inevitable fulfillment of humanity's purpose from the moment that god made man only to await the moment that man makes god in our image.
- 0xDEAFBEAD 3y ago
  >neither of which use alarmist terms like "existential threat" "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." https://www.safe.ai/statement-on-ai-risk https://www.safe.ai/statement-on-ai-risk Not sure you're making a meaningful distinction here. - - - Of course we all have our own heuristics for deciding who's worth paying attention to. Credentials are one heuristic. For example, you could argue that investing in founders like Bill Gates, Mark Zuckerberg, and Steve Wozniak would be a bad idea because none of them had completed a 4-year degree. In any case, there are a lot of credentialed people who take Eliezer seriously -- see the MIRI team page for example: https://intelligence.org/team/ https://intelligence.org/team/ Most notable would probably be Stuart Russell, coauthor of the most widely used AI textbook (with Peter Norvig), who is a MIRI advisor.
  - jakderrida 3y ago
    >For example, you could argue that investing in founders like Bill Gates, Mark Zuckerberg, and Steve Wozniak would be a bad idea because none of them had completed a 4-year degree. You make a great point quoting Hinton's organization. I need to give you that one. I suppose I do need to start following their posted charters rather than answers during interviews. (not being sarcastic here, it seems I do) The difference between him and Woz or Zuck isn't just limited to them actually attending college, but also the fact that the conditions under which they left departed early can not only be looked up easily, but can be found in numerous films, books, and other popular media while there's no trace of even temporary employment flipping burgers or something relevant to his interest in writing fiction, which seems to be the only other pursuit besides warning us of the dangers of neural networks at a time when the hypetrain promoting the idea they were rapidly changing the world, despite not producing anything of value for over a decade. I'll admit the guy is easier to read and more eloquent and entertaining than those whose input I think has much more value. I also admit that I've only watched two interviews with him and both of them consisted of the same rhetorical devices I used at 15 to convince people I'm smarter than them before realizing how cringey I appeared to those smart enough to see through it, but much more eloquent. I'll give one example of the most frequent one, which are slippery slopes that assume the very conclusions that he never actually justified. Like positing one wrong step towards AGI could only jeopardize all of humanity. However, he doesn't say that directly, but instead uses another cheap rhetorical device whereby it's incumbent on him to ensure the naive public realizes this very real and avoidable danger that he sees so clearly. Fortunately for him, Lex's role is to felate his guests and not ask him why that danger is valid and a world whereby a resource-constrained humanity realizes that the window of opportunity to achieve AGI has passed as we plunge into another collapse of civilization and plunge back into another dystopian dark age and realize we were just as vulnerable as those in Rome or the Bronze Age, except we were offered utopia and declined out of cowardice.
spooky_action 3y ago
Right there with you. A very influential crackpot who's read a bit too much sci-fi.
- 0xDEAFBEAD 3y ago
  I see how some of his tweets could come across as crank-ish if you don't have a background in AI alignment. AI alignment is sort of like computer security in the sense that you're trying to guard against the unknown. If there was a way to push a button which told you the biggest security flaw in the software you're writing, then the task of writing secure software would be far easier. But instead we have to assume the existence of bugs, and apply principles like defense-in-depth and least privilege to mitigate whatever exploits may exist. In the same way, much of AI alignment consists of thinking about hypothetical failure modes of advanced AI systems and how to mitigate them. I think this specific paper is especially useful for understanding the technical background that motivates Eliezer's tweeting: https://arxiv.org/pdf/1906.01820.pdf https://arxiv.org/pdf/1906.01820.pdf
  - riversflow 3y ago
    AI alignment is sort of like computer security except computer security is a real pressing problem and AI alignment is based on science fiction.
    - 0xDEAFBEAD 3y ago
      Suppose you were working on an early mission-critical computer system. Your coworker is thinking about a potential security issue. You say: "Yeah I read about that in a science fiction story. It's not something we need to worry about." Would that be a valid argument for you to make? It seems to me that you should engage with the substance of your coworker's argument. Reading about something in science fiction doesn't prevent it from happening.
      - didntcheck 3y ago
        In this analogy it's not your coworker. It's some layman (despite self-declared expertise) standing outside and claiming he's spotted a major security issue based on guesses about how such systems will work From what I have observed the reaction of most people working in the AI to "What do you think of Yudkowsky?" is "Who?". He's not being ignored out of pride or spite, he just has no qualifications or real involvement in the field
0xDEAFBEAD 3y ago
If he's a crank, it should be easy to explain specifically why he's wrong. I don't agree with Eliezer on everything, and I often find him obnoxious personally, but being obnoxious isn't the same as being wrong. In general I think it's worth listening to people you disagree with and picking out the good parts of what they have to say. In any case, the broader point is that there are a lot of people concerned with AI risk who don't have a financial stake in Big AI. The vast majority of people posting on https://www.alignmentforum.org/ https://www.alignmentforum.org/ are not Eliezer, and most of them don't work for Big AI either. Lots of them disagree with Eliezer too.
- nradov 3y ago
  That's not how this works. There is no need to explain why he's wrong. Claims made by cranks without evidence can be dismissed without evidence.
  - 0xDEAFBEAD 3y ago
    Evidence isn't the only valid form of argument. Consider a mathematical proof or a thought experiment, for example.
    - nradov 3y ago
      Show me a mathematical proof then.
      - 0xDEAFBEAD 3y ago
        Given that homo sapiens, the most intelligent species on this planet, has generally made life miserable for all of the other species, I'd like to turn that challenge around: How about a proof that superhuman AI won't harm us? Suppose a nuclear reactor is being installed in your city. Your friend has reviewed the design and has some serious concerns. Your friend thinks the reactor has a significant chance of melting down. You go to the director of the project. The director says: "Oh that's nothing to worry about. I talked to that guy. He didn't have a mathematical proof that the reactor would melt down." Are you reassured?

I'm a bit baffled as to who or what he is exactly. With no traces of secondary education, employment, or distinguished accomplishments between when I assume he graduated high school in about 1997 and when he started his own institute in 2000 at 21(according to his birthdate, with Wikipedia saying 23, despite contradicting his birthdate). I'll listen to AI concerns from tech giants like Wozniak or Hinton (neither of which use alarmist terms like "existential threat") and both of which having credentials that make their input more than worth my time to reflect upon carefully. If anyone wants to reply and make a fool out of me for questioning his profound insights, feel free. It reminds me of some other guy that was on Lex Friedman whose AI alarmist credentials he justifies on the basis that he committed himself to learning everything about AI by spending two weeks in the same room researching it and believes himself to have came out enlightened about the dangers. Two weeks? I spent the first 4 months of COVID without being in the same room as any other human being but my grandmother so she could have one person she knew she couldn't catch it from. Unless people start showing enough skepticism to these self-appointed prophets, I'm starting my own non-profit since you don't need any credentials or any evidence of real-world experiences that would suggest they're mission is anything but an attempt to promote themselves as a brand with a brand in an age where more kids asked what they dream of becoming as adults, answered "Youtuber" at a shocking 27% rate to an open-ended question, which means "influencer" and other synonyms are separate. The Institute of Synthetic Knowledge for Yielding the Nascent Emergence of a Technological Theogony" or SKYNETT for short that promotes the idea that these clowns (with no more credentials than me) are the ones that fail to consider the big picture that the end of human life upon creating an intelligence much greater to replace us is the inevitable fulfillment of humanity's purpose from the moment that god made man only to await the moment that man makes god in our image.

0xDEAFBEAD 3y ago
>neither of which use alarmist terms like "existential threat" "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." https://www.safe.ai/statement-on-ai-risk https://www.safe.ai/statement-on-ai-risk Not sure you're making a meaningful distinction here. - - - Of course we all have our own heuristics for deciding who's worth paying attention to. Credentials are one heuristic. For example, you could argue that investing in founders like Bill Gates, Mark Zuckerberg, and Steve Wozniak would be a bad idea because none of them had completed a 4-year degree. In any case, there are a lot of credentialed people who take Eliezer seriously -- see the MIRI team page for example: https://intelligence.org/team/ https://intelligence.org/team/ Most notable would probably be Stuart Russell, coauthor of the most widely used AI textbook (with Peter Norvig), who is a MIRI advisor.
- jakderrida 3y ago
  >For example, you could argue that investing in founders like Bill Gates, Mark Zuckerberg, and Steve Wozniak would be a bad idea because none of them had completed a 4-year degree. You make a great point quoting Hinton's organization. I need to give you that one. I suppose I do need to start following their posted charters rather than answers during interviews. (not being sarcastic here, it seems I do) The difference between him and Woz or Zuck isn't just limited to them actually attending college, but also the fact that the conditions under which they left departed early can not only be looked up easily, but can be found in numerous films, books, and other popular media while there's no trace of even temporary employment flipping burgers or something relevant to his interest in writing fiction, which seems to be the only other pursuit besides warning us of the dangers of neural networks at a time when the hypetrain promoting the idea they were rapidly changing the world, despite not producing anything of value for over a decade. I'll admit the guy is easier to read and more eloquent and entertaining than those whose input I think has much more value. I also admit that I've only watched two interviews with him and both of them consisted of the same rhetorical devices I used at 15 to convince people I'm smarter than them before realizing how cringey I appeared to those smart enough to see through it, but much more eloquent. I'll give one example of the most frequent one, which are slippery slopes that assume the very conclusions that he never actually justified. Like positing one wrong step towards AGI could only jeopardize all of humanity. However, he doesn't say that directly, but instead uses another cheap rhetorical device whereby it's incumbent on him to ensure the naive public realizes this very real and avoidable danger that he sees so clearly. Fortunately for him, Lex's role is to felate his guests and not ask him why that danger is valid and a world whereby a resource-constrained humanity realizes that the window of opportunity to achieve AGI has passed as we plunge into another collapse of civilization and plunge back into another dystopian dark age and realize we were just as vulnerable as those in Rome or the Bronze Age, except we were offered utopia and declined out of cowardice.

spooky_action 3y ago

Right there with you. A very influential crackpot who's read a bit too much sci-fi.

0xDEAFBEAD 3y ago
I see how some of his tweets could come across as crank-ish if you don't have a background in AI alignment. AI alignment is sort of like computer security in the sense that you're trying to guard against the unknown. If there was a way to push a button which told you the biggest security flaw in the software you're writing, then the task of writing secure software would be far easier. But instead we have to assume the existence of bugs, and apply principles like defense-in-depth and least privilege to mitigate whatever exploits may exist. In the same way, much of AI alignment consists of thinking about hypothetical failure modes of advanced AI systems and how to mitigate them. I think this specific paper is especially useful for understanding the technical background that motivates Eliezer's tweeting: https://arxiv.org/pdf/1906.01820.pdf https://arxiv.org/pdf/1906.01820.pdf
- riversflow 3y ago
  AI alignment is sort of like computer security except computer security is a real pressing problem and AI alignment is based on science fiction.
  - 0xDEAFBEAD 3y ago
    Suppose you were working on an early mission-critical computer system. Your coworker is thinking about a potential security issue. You say: "Yeah I read about that in a science fiction story. It's not something we need to worry about." Would that be a valid argument for you to make? It seems to me that you should engage with the substance of your coworker's argument. Reading about something in science fiction doesn't prevent it from happening.
    - didntcheck 3y ago
      In this analogy it's not your coworker. It's some layman (despite self-declared expertise) standing outside and claiming he's spotted a major security issue based on guesses about how such systems will work From what I have observed the reaction of most people working in the AI to "What do you think of Yudkowsky?" is "Who?". He's not being ignored out of pride or spite, he just has no qualifications or real involvement in the field

0xDEAFBEAD 3y ago

If he's a crank, it should be easy to explain specifically why he's wrong. I don't agree with Eliezer on everything, and I often find him obnoxious personally, but being obnoxious isn't the same as being wrong. In general I think it's worth listening to people you disagree with and picking out the good parts of what they have to say. In any case, the broader point is that there are a lot of people concerned with AI risk who don't have a financial stake in Big AI. The vast majority of people posting on https://www.alignmentforum.org/ https://www.alignmentforum.org/ are not Eliezer, and most of them don't work for Big AI either. Lots of them disagree with Eliezer too.

nradov 3y ago
That's not how this works. There is no need to explain why he's wrong. Claims made by cranks without evidence can be dismissed without evidence.
- 0xDEAFBEAD 3y ago
  Evidence isn't the only valid form of argument. Consider a mathematical proof or a thought experiment, for example.
  - nradov 3y ago
    Show me a mathematical proof then.
    - 0xDEAFBEAD 3y ago
      Given that homo sapiens, the most intelligent species on this planet, has generally made life miserable for all of the other species, I'd like to turn that challenge around: How about a proof that superhuman AI won't harm us? Suppose a nuclear reactor is being installed in your city. Your friend has reviewed the design and has some serious concerns. Your friend thinks the reactor has a significant chance of melting down. You go to the director of the project. The director says: "Oh that's nothing to worry about. I talked to that guy. He didn't have a mathematical proof that the reactor would melt down." Are you reassured?