5 ms·

> Yudkowsky has a very shallow understanding of machine learning/AI, but loves to write in a conspiratorial way with in-group terminology to make people think t

> Yudkowsky has a very shallow understanding of machine learning/AI, but loves to write in a conspiratorial way with in-group terminology to make people think that he's onto some dark secrets. Perhaps I am a little behind the times, but what are reasons/achievements/credentials behind his huge, prophet like status within the cult of rationalism? Beyond writing a popular Harry Potter fanfic, I mean.

entelechy0 4y ago
[dead]
PaulHoule 4y ago
[flagged]
nwah1 4y ago
Charisma, humor, and religious-like ideological framing that can acquire true believers who believe they are saving the world.
agentwiggles 4y ago
I think EY's big contribution to the LessWrong/Rationalist cult/movement/ethos is writing the "Sequences," which purport to be a guide to training yourself to think rationally. (https://www.lesswrong.com/tag/original-sequences https://www.lesswrong.com/tag/original-sequences) The rationalists are an interesting group. I like a lot of the vibe of the whole thing, but there is also some very weird navel gazing that has cropped up. It can be hard to differentiate from the good and the bad. I am overall a little surprised by the vitriol towards EY in this thread though. While I agree that the guy is somewhat shrill and his ideas are pretty out there, he really doesn't seem like a con artist from my standpoint. It seems to me that he is genuinely extremely concerned about AI alignment/risk, and his actions seem to make sense in that framework without needing to ascribe a bunch of weird and sinister motivations to him. Not professing agreement or disagreement with him on any particular thing, just my two cents.
- polygamous_bat 4y ago
  I understand your point, more or less, I just find it funny that people calling themselves "rationalists"/"objectivists"/whatever other funny name just keeps stacking up a bunch of "what if"s and freaking out about it. As somewhat of a cynical person, I also find it funny that > While I agree that the guy is somewhat shrill and his ideas are pretty out there, he really doesn't seem like a con artist from my standpoint. I've heard this exact same sentiment about another popular Effective Altruist in the past year. I will let the reader guess which one.
  - archon1410 4y ago
    >"objectivists"/whatever other funny name just keeps stacking up a bunch of "what if"s and freaking out about it. The Objectivists I know hate hypotheticals and refuge to engage with them with the same dismissive tone used in your comment.
    - dragonwriter 4y ago
      The Objectivists I’ve encountered refuse to engage with lots of things, but still use hypotheticals themselves. They mostly phrase them as generalities with “When…” rather than with “If…”, but they are semantically equivalent.
  - agentwiggles 4y ago
    > I've heard this exact same sentiment about another popular Effective Altruist in the past year. I will let the reader guess which one. I may be missing something, as I don't really follow this "community" besides reading the occasional Slate Star Codex post or whatever. But I don't see much ground for comparison here. SBF (who I assume you're referring to) was in much different position with regards to potential harm as a result of con man antics than EY is. (As far as I can tell). SBF was managing billions of dollars. EY is not. If SBF turns out to be a scammer, real people lose money. If EY is a scammer... then so what? He publishes a bunch of blog posts crying doom about AI that suddenly look less sincere? To what end? I can't see a good reason at this time for assuming his reasons are anything less than what he claims they are.
didericis 4y ago
He’s an excellent writer and has spent a long time trying to come up with high level easily understandable ways to try to measure alignment abstractly through pure logic/outside of any given system. Essentially him and his crowd have been trying to create a rationally grounded purely logical non contradictory moral framework for action from axiomatic first principles (he’s not the first or the last to do this), and they believe such a system is essential if AGI comes to be. I personally think AGI is impossible, his purely logical moral system is impossible, and think what he’s working on is not applicable to current AI (I think he knows this, but his conclusion seems to be that the path to AGI is out of the bag/was made incredibly foolishly and should have been constructed differently with more ability to add guardrails) But I do think the high level abstract way he thinks is fruitful/very valuable, and a good moderating counterbalance to the frenzied “harder better faster stronger” ML stampede that’s not really thinking about the ramifications of what they’re doing with any sort of discipline.
- adastra22 4y ago
  > I personally think AGI is impossible That’s a very weird statement to make. Do you think you have a soul or something? Otherwise how is your own Intelligence something a computer can’t replicate?
  - sebastianz 4y ago
    A computer could indeed replicate our intelligence, but our intelligence might not be sufficient to write the AGI sourcecode :)
  - drdeca 4y ago
    I do think I have a soul, and I don’t think that AGI is impossible. So, I really don’t get why people think AGI is impossible. I mean, I have hope that it won’t ever happen, but not because I suspect it is impossible.
    - didericis 4y ago
      I think it’s impossible because I don’t think you can leapfrog evolution, and there’s almost definitely hidden context essential to our intelligence and perception that you can only get through a very very long history. However I do think it’s quite possible and likely that we’ll create mimics that convince most people they’re intelligent. But I think that’s a weird type of reflection/anything we can make is limited to mirroring our observable past and existing collective thought and can never be truly introspective, because true introspection requires evolutionary context and hidden motivation you can’t transfer.
traverseda 4y ago
>If scientific knowledge were hidden in ancient vaults (rather than hidden in inconvenient pay-for-access journals), at least then people would try to get into the vaults. They'd be desperate to learn science. Especially when they saw the power that Eighth Level Physicists could wield, and were told that they weren't allowed to know the explanation. > [...] >If the Great Secret of Natural Selection, passed down from Darwin Who Is Not Forgotten, was only ever imparted to you after you paid $2000 and went through a ceremony involving torches and robes and masks and sacrificing an ox, then when you were shown the fossils, and shown the optic cable going through the retina under a microscope, and finally told the Truth, you would say "That's the most brilliant thing ever!" and be satisfied. After that, if some other cult tried to tell you it was actually a bearded man in the sky 6000 years ago, you'd laugh like hell. >And you know, it might actually be more fun to do things that way. Especially if the initiation required you to put together some of the evidence for yourself—together, or with classmates—before you could tell your Science Sensei you were ready to advance to the next level. It wouldn't be efficient, sure, but it would be fun. >[...] >And no, I'm not seriously proposing that we try to reverse the last five hundred years of openness and classify all the science secret. ~ Eliezer Yudkowsky, https://www.lesswrong.com/s/6BFkmEgre7uwhDxDR/p/3diLhMELXxM8rFHJj https://www.lesswrong.com/s/6BFkmEgre7uwhDxDR/p/3diLhMELXxM8... That's really all there is to Yudkowsky's persona of occultism at the end of the day. It's an intentional affectation.
uoaei 4y ago
He spends a lot of time writing on LessWrong, noted online agora for the Dunning-Krugers among us.

entelechy0 4y ago

[dead]

PaulHoule 4y ago

[flagged]

nwah1 4y ago

Charisma, humor, and religious-like ideological framing that can acquire true believers who believe they are saving the world.

agentwiggles 4y ago

I think EY's big contribution to the LessWrong/Rationalist cult/movement/ethos is writing the "Sequences," which purport to be a guide to training yourself to think rationally. (https://www.lesswrong.com/tag/original-sequences https://www.lesswrong.com/tag/original-sequences) The rationalists are an interesting group. I like a lot of the vibe of the whole thing, but there is also some very weird navel gazing that has cropped up. It can be hard to differentiate from the good and the bad. I am overall a little surprised by the vitriol towards EY in this thread though. While I agree that the guy is somewhat shrill and his ideas are pretty out there, he really doesn't seem like a con artist from my standpoint. It seems to me that he is genuinely extremely concerned about AI alignment/risk, and his actions seem to make sense in that framework without needing to ascribe a bunch of weird and sinister motivations to him. Not professing agreement or disagreement with him on any particular thing, just my two cents.

polygamous_bat 4y ago
I understand your point, more or less, I just find it funny that people calling themselves "rationalists"/"objectivists"/whatever other funny name just keeps stacking up a bunch of "what if"s and freaking out about it. As somewhat of a cynical person, I also find it funny that > While I agree that the guy is somewhat shrill and his ideas are pretty out there, he really doesn't seem like a con artist from my standpoint. I've heard this exact same sentiment about another popular Effective Altruist in the past year. I will let the reader guess which one.
- archon1410 4y ago
  >"objectivists"/whatever other funny name just keeps stacking up a bunch of "what if"s and freaking out about it. The Objectivists I know hate hypotheticals and refuge to engage with them with the same dismissive tone used in your comment.
  - dragonwriter 4y ago
    The Objectivists I’ve encountered refuse to engage with lots of things, but still use hypotheticals themselves. They mostly phrase them as generalities with “When…” rather than with “If…”, but they are semantically equivalent.
- agentwiggles 4y ago
  > I've heard this exact same sentiment about another popular Effective Altruist in the past year. I will let the reader guess which one. I may be missing something, as I don't really follow this "community" besides reading the occasional Slate Star Codex post or whatever. But I don't see much ground for comparison here. SBF (who I assume you're referring to) was in much different position with regards to potential harm as a result of con man antics than EY is. (As far as I can tell). SBF was managing billions of dollars. EY is not. If SBF turns out to be a scammer, real people lose money. If EY is a scammer... then so what? He publishes a bunch of blog posts crying doom about AI that suddenly look less sincere? To what end? I can't see a good reason at this time for assuming his reasons are anything less than what he claims they are.

didericis 4y ago

He’s an excellent writer and has spent a long time trying to come up with high level easily understandable ways to try to measure alignment abstractly through pure logic/outside of any given system. Essentially him and his crowd have been trying to create a rationally grounded purely logical non contradictory moral framework for action from axiomatic first principles (he’s not the first or the last to do this), and they believe such a system is essential if AGI comes to be. I personally think AGI is impossible, his purely logical moral system is impossible, and think what he’s working on is not applicable to current AI (I think he knows this, but his conclusion seems to be that the path to AGI is out of the bag/was made incredibly foolishly and should have been constructed differently with more ability to add guardrails) But I do think the high level abstract way he thinks is fruitful/very valuable, and a good moderating counterbalance to the frenzied “harder better faster stronger” ML stampede that’s not really thinking about the ramifications of what they’re doing with any sort of discipline.

adastra22 4y ago
> I personally think AGI is impossible That’s a very weird statement to make. Do you think you have a soul or something? Otherwise how is your own Intelligence something a computer can’t replicate?
- sebastianz 4y ago
  A computer could indeed replicate our intelligence, but our intelligence might not be sufficient to write the AGI sourcecode :)
- drdeca 4y ago
  I do think I have a soul, and I don’t think that AGI is impossible. So, I really don’t get why people think AGI is impossible. I mean, I have hope that it won’t ever happen, but not because I suspect it is impossible.
  - didericis 4y ago
    I think it’s impossible because I don’t think you can leapfrog evolution, and there’s almost definitely hidden context essential to our intelligence and perception that you can only get through a very very long history. However I do think it’s quite possible and likely that we’ll create mimics that convince most people they’re intelligent. But I think that’s a weird type of reflection/anything we can make is limited to mirroring our observable past and existing collective thought and can never be truly introspective, because true introspection requires evolutionary context and hidden motivation you can’t transfer.

traverseda 4y ago

>If scientific knowledge were hidden in ancient vaults (rather than hidden in inconvenient pay-for-access journals), at least then people would try to get into the vaults. They'd be desperate to learn science. Especially when they saw the power that Eighth Level Physicists could wield, and were told that they weren't allowed to know the explanation. > [...] >If the Great Secret of Natural Selection, passed down from Darwin Who Is Not Forgotten, was only ever imparted to you after you paid $2000 and went through a ceremony involving torches and robes and masks and sacrificing an ox, then when you were shown the fossils, and shown the optic cable going through the retina under a microscope, and finally told the Truth, you would say "That's the most brilliant thing ever!" and be satisfied. After that, if some other cult tried to tell you it was actually a bearded man in the sky 6000 years ago, you'd laugh like hell. >And you know, it might actually be more fun to do things that way. Especially if the initiation required you to put together some of the evidence for yourself—together, or with classmates—before you could tell your Science Sensei you were ready to advance to the next level. It wouldn't be efficient, sure, but it would be fun. >[...] >And no, I'm not seriously proposing that we try to reverse the last five hundred years of openness and classify all the science secret. ~ Eliezer Yudkowsky, https://www.lesswrong.com/s/6BFkmEgre7uwhDxDR/p/3diLhMELXxM8rFHJj https://www.lesswrong.com/s/6BFkmEgre7uwhDxDR/p/3diLhMELXxM8... That's really all there is to Yudkowsky's persona of occultism at the end of the day. It's an intentional affectation.

uoaei 4y ago

He spends a lot of time writing on LessWrong, noted online agora for the Dunning-Krugers among us.