5 ms·

Yudkowsky doesn't strike me as a crank. Why do you say that?

ForHackernews 5y ago
What has he ever done in AI except talk about it? At best he's an AI promoter, and hype-men are often cranks or scammers (see also: VR, web3, cryptocurrencies, MLM)
- marvin 5y ago
  Yudkowsky is a philosopher, in the sense of someone who thinks a lot about things that haven't been achieved yet. Lots of otherwise smart people (wrongly!) discount the value of philosophy, but it's close by every time there's a paradigm shift in humanity's knowledge. Philosophers can be scientists and vice versa. If anything, I'm surprised that this philosophy isn't mentioned more in a thread where the author gleefully talks about ML being used to create better AI, layer by layer until the thing is even more opaque than what we're currently working with. This is terrifying, as we currently have only very loose ideas about how to reliably ensure that a powerful reinforcement learning system doesn't accidentally optimize for something we don't want. The current paradigm is "turn it off", which works well for now but seems like a fragile shield long-term.
  - the_af 5y ago
    > Yudkowsky is a philosopher At least inasmuch as anyone who thinks about stuff can be considered a philosopher. But he strikes me much more as a self-appointed expert on matters where he shows no achievements. He writes fanfiction about AI rather than actually doing stuff with AI.
    - psyc 5y ago
      Most people who think about stuff don’t leave a mountain of highly organized and entertaining essays for posterity.
      - the_af 5y ago
        That just means he's a prolific writer, which I never questioned. I'm arguing that what he writes is fanfiction (which he takes way too seriously), and that he's not an expert in AI and therefore we shouldn't take his predictions too seriously.
        franknstein 5y ago
        There is long history of science-fiction writers painting correct visions of, at the time seemingly impossible future. Sometimes You don't have to be an expert to notice the trends in certain domains. I'd go even further - it could be easier to notice the big picture without being overly bothered with nitty-gritty technical details - yes, to further the field they are necessary, but to comment the direction the field is heading in and about its implications for society they are not. I don't necessarily agree with the author, I'm just making a general comment.
        the_af 5y ago
        > "There is long history of science-fiction writers painting correct visions of, at the time seemingly impossible future" That's a pretty weak argument. I love scifi, and I love for example Philip Dick's writing, yet I would not consider PKD's opinion on the future of AI/AGI particularly relevant. James Cameron is not an authority on AI either. If people said "Yudkowsky is a nice fanfiction author" it would be one thing. But he considers himself an actual AI researcher, and that's just not right. He is not qualified, and has no accomplishments in the area, other than writing fanfiction about it.
        psyc 5y ago
        You keep hammering on with the same lazy slander. Yudkowsky was well known long before he wrote the popular Harry Potter fanfic, which incidentally is pedagogical / allegorical. Because his main roles are teacher and philosopher, and philosophers do that. Here's a sampling of his non-fiction writing: https://www.lesswrong.com/tag/sequences https://www.lesswrong.com/tag/sequences https://intelligence.org/files/EthicsofAI.pdf https://intelligence.org/files/EthicsofAI.pdf https://arxiv.org/abs/1710.05060 https://arxiv.org/abs/1710.05060 https://intelligence.org/files/AlignmentMachineLearning.pdf https://intelligence.org/files/AlignmentMachineLearning.pdf https://arxiv.org/abs/1401.5577 https://arxiv.org/abs/1401.5577 https://arxiv.org/abs/1401.5577 https://arxiv.org/abs/1401.5577 https://intelligence.org/files/Corrigibility.pdf https://intelligence.org/files/Corrigibility.pdf https://intelligence.org/files/DefinabilityTruthDraft.pdf https://intelligence.org/files/DefinabilityTruthDraft.pdf https://intelligence.org/files/IEM.pdf https://intelligence.org/files/IEM.pdf https://intelligence.org/files/TilingAgentsDraft.pdf https://intelligence.org/files/TilingAgentsDraft.pdf But you'll just say there's nothing of value there, and it's somehow figuratively "fan fiction", because he didn't go to college, and he doesn't work much on ML, which is clearly the end-all of AI.
  - goatlover 5y ago
    Nick Bostrom is a professional philosopher who has also thought a lot about AGI and simulation scenarios. But just because educated, smart people can think a lot about something doesn't it mean it will necessarily happen. I'm sure there's quite a few people who have thought at length about warp drives and wormholes. That doesn't mean we'll ever be able to make use of them.
    - gizajob 5y ago
      Nick Bostrom is also an egotistical grinder, who couldn't programme a microwave.
      - TheRealNGenius 5y ago
        I agree, that guy's also a whole lotta bad takes put all together. Like a slutty potato.
  - ASalazarMX 5y ago
    In the few contact I've had with philosophers, I've got the (maybe wrong and oversimplified, but nonetheless evident) impression that they find philosophy fun and mentally stimulating, and they enjoy academically talking and writing about edge cases no matter their feasibility. I think it's a net good when philosophers mentally explore scenarios most people don't consider, regardless of their plausibility, but they will not be taken more seriously that Peter Watts looking for inspiration for his next scifi book unless they offer some kind of evidence to sustain their conclusions. The handful of philosophers I've known are very smart, interesting people, but except for one who teaches philosophy, they don't live off philosophy. Most have a side job, and one lives with his parents. It's as if philosophy was more of a mentally rewarding hobby than a job.
    - tuatoru 5y ago
      Mentally rewarding hobbies can be fruitful, though. Consider the case of Oliver Heaviside, the person who came up with Maxwell's equations of electromagnetism in the form that we use today, as well as several other useful things. I have no doubt that he was thought a crank by many, nor that many other actual cranks were claiming discoveries based on Maxwell's work. One can only tell the difference in the rear-view mirror. 1. https://en.wikipedia.org/wiki/Oliver_Heaviside https://en.wikipedia.org/wiki/Oliver_Heaviside
      - marvin 5y ago
        Why do people have to use the derogatory word «crank»? It’s perfectly possible to just be smart, sincere and wrong. Talking about paradigm shifts again, not a single one of them matched the consensus at the time. Isaac Newton spent a lot of time thinking about alchemy and religion, for heaven’s sake. During his lifetime, it wasn’t obvious even to the smartest thinkers whether science, the Bible or quasi-religious rituals was the best tool for understanding the world. Conventional thinkers expect that the future will look like what they know, and it leads to frequent, overconfident dismissal of everything that is unconventional. By all means, disagree and explore in your own direction. But don’t go around degrading those you disagree with. It’s just so average.
- psyc 5y ago
  This is very uncharitable. He’s a prolific neopositivist-ish philosopher with a distinct voice. He’s a good decision theorist. He doesn’t publish much himself, but he directly mentors, advises, and collaborates with people who do.
- colinmhayes 5y ago
  He wrote a thesis on decision theory.
  - the_af 5y ago
    He wrote a thesis in the sense of he actually wrote a thesis for a known university, that is referenced and quoted in academia by decision theory scholars, or do you mean he self-published a "thesis" in the sense he wrote something that nobody in the field pays much attention to and that cannot be published in peer reviewed journals? If the latter, anybody can do that.
the_af 5y ago
> Yudkowsky doesn't strike me as a crank. Why do you say that? He is a minor internet celebrity whose only claim to fame is writing fanfiction about AI, the whole "rationality" cult, and is a self-proclaimed expert on matters where he shows no achievements (like AI) while making unsupported doomsday predictions about evil AI, the Singularity, etc. Also, that deal with Roko's Basilisk that he now wishes never occurred (oh, yes, "it was a joke"). Mostly, someone with no studies and no achievements making wild doomsday predictions. Doesn't that strike you as a crank? An analogy would be if I made wild assertions about the physics of the universe without having studied physics, without any lab work, without engaging in discussion with qualified experts, with no peer reviews, and all I presented as evidence of my revolutionary findings about the universe was some fanfiction in my blog. Oh, and I created the Extraordinary Physics Institute.
heavyset_go 5y ago
What are his qualifications?
- fulafel 5y ago
  Isn't this appeal to authority? He's obviously pretty smart and a lot of people take the risk modeling thinking seriously, and his arguments and output deserve to be evaluated on their own merits. (Also on the tech practicioner side there's obviously lots of major figures that don't have formal qualifications)
  - heavyset_go 5y ago
    If we're going to talk about logical fallacies instead of answering the question, your post is an example of the argumentum ad populum fallacy. If someone is going to market themselves as an AI expert, I expect them to have experience and qualifications to back up their opinions.
    - deleted 5y ago
      [deleted]

What has he ever done in AI except talk about it? At best he's an AI promoter, and hype-men are often cranks or scammers (see also: VR, web3, cryptocurrencies, MLM)

marvin 5y ago
Yudkowsky is a philosopher, in the sense of someone who thinks a lot about things that haven't been achieved yet. Lots of otherwise smart people (wrongly!) discount the value of philosophy, but it's close by every time there's a paradigm shift in humanity's knowledge. Philosophers can be scientists and vice versa. If anything, I'm surprised that this philosophy isn't mentioned more in a thread where the author gleefully talks about ML being used to create better AI, layer by layer until the thing is even more opaque than what we're currently working with. This is terrifying, as we currently have only very loose ideas about how to reliably ensure that a powerful reinforcement learning system doesn't accidentally optimize for something we don't want. The current paradigm is "turn it off", which works well for now but seems like a fragile shield long-term.
- the_af 5y ago
  > Yudkowsky is a philosopher At least inasmuch as anyone who thinks about stuff can be considered a philosopher. But he strikes me much more as a self-appointed expert on matters where he shows no achievements. He writes fanfiction about AI rather than actually doing stuff with AI.
  - psyc 5y ago
    Most people who think about stuff don’t leave a mountain of highly organized and entertaining essays for posterity.
    - the_af 5y ago
      That just means he's a prolific writer, which I never questioned. I'm arguing that what he writes is fanfiction (which he takes way too seriously), and that he's not an expert in AI and therefore we shouldn't take his predictions too seriously.
      - franknstein 5y ago
        There is long history of science-fiction writers painting correct visions of, at the time seemingly impossible future. Sometimes You don't have to be an expert to notice the trends in certain domains. I'd go even further - it could be easier to notice the big picture without being overly bothered with nitty-gritty technical details - yes, to further the field they are necessary, but to comment the direction the field is heading in and about its implications for society they are not. I don't necessarily agree with the author, I'm just making a general comment.
        the_af 5y ago
        > "There is long history of science-fiction writers painting correct visions of, at the time seemingly impossible future" That's a pretty weak argument. I love scifi, and I love for example Philip Dick's writing, yet I would not consider PKD's opinion on the future of AI/AGI particularly relevant. James Cameron is not an authority on AI either. If people said "Yudkowsky is a nice fanfiction author" it would be one thing. But he considers himself an actual AI researcher, and that's just not right. He is not qualified, and has no accomplishments in the area, other than writing fanfiction about it.
        psyc 5y ago
        You keep hammering on with the same lazy slander. Yudkowsky was well known long before he wrote the popular Harry Potter fanfic, which incidentally is pedagogical / allegorical. Because his main roles are teacher and philosopher, and philosophers do that. Here's a sampling of his non-fiction writing: https://www.lesswrong.com/tag/sequences https://www.lesswrong.com/tag/sequences https://intelligence.org/files/EthicsofAI.pdf https://intelligence.org/files/EthicsofAI.pdf https://arxiv.org/abs/1710.05060 https://arxiv.org/abs/1710.05060 https://intelligence.org/files/AlignmentMachineLearning.pdf https://intelligence.org/files/AlignmentMachineLearning.pdf https://arxiv.org/abs/1401.5577 https://arxiv.org/abs/1401.5577 https://arxiv.org/abs/1401.5577 https://arxiv.org/abs/1401.5577 https://intelligence.org/files/Corrigibility.pdf https://intelligence.org/files/Corrigibility.pdf https://intelligence.org/files/DefinabilityTruthDraft.pdf https://intelligence.org/files/DefinabilityTruthDraft.pdf https://intelligence.org/files/IEM.pdf https://intelligence.org/files/IEM.pdf https://intelligence.org/files/TilingAgentsDraft.pdf https://intelligence.org/files/TilingAgentsDraft.pdf But you'll just say there's nothing of value there, and it's somehow figuratively "fan fiction", because he didn't go to college, and he doesn't work much on ML, which is clearly the end-all of AI.
- goatlover 5y ago
  Nick Bostrom is a professional philosopher who has also thought a lot about AGI and simulation scenarios. But just because educated, smart people can think a lot about something doesn't it mean it will necessarily happen. I'm sure there's quite a few people who have thought at length about warp drives and wormholes. That doesn't mean we'll ever be able to make use of them.
  - gizajob 5y ago
    Nick Bostrom is also an egotistical grinder, who couldn't programme a microwave.
    - TheRealNGenius 5y ago
      I agree, that guy's also a whole lotta bad takes put all together. Like a slutty potato.
- ASalazarMX 5y ago
  In the few contact I've had with philosophers, I've got the (maybe wrong and oversimplified, but nonetheless evident) impression that they find philosophy fun and mentally stimulating, and they enjoy academically talking and writing about edge cases no matter their feasibility. I think it's a net good when philosophers mentally explore scenarios most people don't consider, regardless of their plausibility, but they will not be taken more seriously that Peter Watts looking for inspiration for his next scifi book unless they offer some kind of evidence to sustain their conclusions. The handful of philosophers I've known are very smart, interesting people, but except for one who teaches philosophy, they don't live off philosophy. Most have a side job, and one lives with his parents. It's as if philosophy was more of a mentally rewarding hobby than a job.
  - tuatoru 5y ago
    Mentally rewarding hobbies can be fruitful, though. Consider the case of Oliver Heaviside, the person who came up with Maxwell's equations of electromagnetism in the form that we use today, as well as several other useful things. I have no doubt that he was thought a crank by many, nor that many other actual cranks were claiming discoveries based on Maxwell's work. One can only tell the difference in the rear-view mirror. 1. https://en.wikipedia.org/wiki/Oliver_Heaviside https://en.wikipedia.org/wiki/Oliver_Heaviside
    - marvin 5y ago
      Why do people have to use the derogatory word «crank»? It’s perfectly possible to just be smart, sincere and wrong. Talking about paradigm shifts again, not a single one of them matched the consensus at the time. Isaac Newton spent a lot of time thinking about alchemy and religion, for heaven’s sake. During his lifetime, it wasn’t obvious even to the smartest thinkers whether science, the Bible or quasi-religious rituals was the best tool for understanding the world. Conventional thinkers expect that the future will look like what they know, and it leads to frequent, overconfident dismissal of everything that is unconventional. By all means, disagree and explore in your own direction. But don’t go around degrading those you disagree with. It’s just so average.
psyc 5y ago
This is very uncharitable. He’s a prolific neopositivist-ish philosopher with a distinct voice. He’s a good decision theorist. He doesn’t publish much himself, but he directly mentors, advises, and collaborates with people who do.
colinmhayes 5y ago
He wrote a thesis on decision theory.
- the_af 5y ago
  He wrote a thesis in the sense of he actually wrote a thesis for a known university, that is referenced and quoted in academia by decision theory scholars, or do you mean he self-published a "thesis" in the sense he wrote something that nobody in the field pays much attention to and that cannot be published in peer reviewed journals? If the latter, anybody can do that.

the_af 5y ago

> Yudkowsky doesn't strike me as a crank. Why do you say that? He is a minor internet celebrity whose only claim to fame is writing fanfiction about AI, the whole "rationality" cult, and is a self-proclaimed expert on matters where he shows no achievements (like AI) while making unsupported doomsday predictions about evil AI, the Singularity, etc. Also, that deal with Roko's Basilisk that he now wishes never occurred (oh, yes, "it was a joke"). Mostly, someone with no studies and no achievements making wild doomsday predictions. Doesn't that strike you as a crank? An analogy would be if I made wild assertions about the physics of the universe without having studied physics, without any lab work, without engaging in discussion with qualified experts, with no peer reviews, and all I presented as evidence of my revolutionary findings about the universe was some fanfiction in my blog. Oh, and I created the Extraordinary Physics Institute.

heavyset_go 5y ago

What are his qualifications?

fulafel 5y ago
Isn't this appeal to authority? He's obviously pretty smart and a lot of people take the risk modeling thinking seriously, and his arguments and output deserve to be evaluated on their own merits. (Also on the tech practicioner side there's obviously lots of major figures that don't have formal qualifications)
- heavyset_go 5y ago
  If we're going to talk about logical fallacies instead of answering the question, your post is an example of the argumentum ad populum fallacy. If someone is going to market themselves as an AI expert, I expect them to have experience and qualifications to back up their opinions.
  - deleted 5y ago
    [deleted]