5 ms·

> "This work is part of the third pillar of our approach to alignment research: we want to automate the alignment research work itself." I feel like this isn't

by ftxbro 3y ago

> "This work is part of the third pillar of our approach to alignment research: we want to automate the alignment research work itself." I feel like this isn't a Yud-approved approach to AI alignment.

shadowgovt 3y ago
"Yud-approved?"
- circuit10 3y ago
  https://en.m.wikipedia.org/wiki/Eliezer_Yudkowsky https://en.m.wikipedia.org/wiki/Eliezer_Yudkowsky
- ftxbro 3y ago
  He's the one in the fedora who is losing patience that otherwise smart sounding people are seriously considering letting AI police itself https://www.youtube.com/watch?v=41SUp-TRVlg https://www.youtube.com/watch?v=41SUp-TRVlg
  - jack_riminton 3y ago
    That's not a fedora, that's his King of the Redditors crown
- aitanabewa 3y ago
  Meaning is approved by Eliezer Yudkowsky. https://en.wikipedia.org/wiki/Eliezer_Yudkowsky https://en.wikipedia.org/wiki/Eliezer_Yudkowsky https://twitter.com/ESYudkowsky https://twitter.com/ESYudkowsky https://www.youtube.com/watch?v=AaTRHFaaPG8 https://www.youtube.com/watch?v=AaTRHFaaPG8 (Lex Fridman Interview)
FeepingCreature 3y ago
Honestly, I think any foundational work on the topic is inherently Yud-favored, compared to the blithe optimism and surface-level analysis at best that is usually applied to the topic. Ie, I think it's not that this shouldn't be done. This should certainly be done. It's just that so many more things than it should be done before we move forward.
Teever 3y ago
Why does this matter?
snapcaster 3y ago
Agreed, Yud does seem to have been right about the course things will take but I'm not confident he actually has any solutions to the problem to offer
- qumpis 3y ago
  Which things has he been right about and when, if you recall?
  - snapcaster 3y ago
    At least to me, this current generation of AI progress is looking a lot more "foomy" than a lot of people besides him predicted
- causalmodels 3y ago
  His solution is a global regulatory regime to ban new large training runs. The tools required to accomplish this are, IMO, out of the question but I will give Yud credit for being honest about them while others who share his viewpoint try to hide the ball.
  - snapcaster 3y ago
    Good point, but I think if he was serious he would be looking to Serena Butler for inspiration not nuclear non-proliferation treaties
bick_nyers 3y ago
These were my thoughts exactly. On one hand, this can enable alignment research to catch up faster. On the other hand, if we are worried about homicidal AI, then putting it in charge of policing itself (and training it to find exploits in a way) is probably not ideal.
ShamelessC 3y ago
You mean Yudkowski? I saw him on Lex Fridman and he was entirely unconvincing. Why is everyone deferring to a bunch of effective altruism advocates when it comes to AI safety?
- ftxbro 3y ago
  > Why is everyone deferring to a bunch of effective altruism advocates when it comes to AI safety? I'm not sure Yudkowski is an EA, but the EAs want him in their polycule.
  - tomjakubowski 3y ago
    He posts on the forum. I'm not sure what more evidence is needed that he's part of it. https://forum.effectivealtruism.org/users/eliezeryudkowsky https://forum.effectivealtruism.org/users/eliezeryudkowsky
    - ftxbro 3y ago
      I guess it's true, not just a rationalist but also effective altruist!
- lubesGordi 3y ago
  I heard him on Lex too, and it seemed to be just a given that AI is going to be deceptive and want to kill us all. I don't think there was a single example of how that could be accomplished given. I'm open to hearing thoughts on this, maybe I'm not creative enough to see the 'obvious' ways this could happen.
  - ethanbond 3y ago
    This is also why I go into chess matches against 1400 elo players. I cannot conceive of the specific ways in which they will beat me (a 600 elo player), so I have good reason to suspect that I can win. I'm willing to bet the future of our species on my consistent victory in these types of matches, in fact.
killthebuddha 3y ago
DISCLAIMER: I think Yudkowsky is a serious thinker and his ideas should be taken seriously, regardless of whether not they are correct. Your comment triggered a random thought: A perfect name for Yudkowsky et al and the AGI doomers is... wait for it... the Yuddites :)
- throwaway2137 3y ago
  Already used on 4chan :)

by ftxbro 3y ago

shadowgovt 3y ago

"Yud-approved?"

circuit10 3y ago
https://en.m.wikipedia.org/wiki/Eliezer_Yudkowsky https://en.m.wikipedia.org/wiki/Eliezer_Yudkowsky
ftxbro 3y ago
He's the one in the fedora who is losing patience that otherwise smart sounding people are seriously considering letting AI police itself https://www.youtube.com/watch?v=41SUp-TRVlg https://www.youtube.com/watch?v=41SUp-TRVlg
- jack_riminton 3y ago
  That's not a fedora, that's his King of the Redditors crown
aitanabewa 3y ago
Meaning is approved by Eliezer Yudkowsky. https://en.wikipedia.org/wiki/Eliezer_Yudkowsky https://en.wikipedia.org/wiki/Eliezer_Yudkowsky https://twitter.com/ESYudkowsky https://twitter.com/ESYudkowsky https://www.youtube.com/watch?v=AaTRHFaaPG8 https://www.youtube.com/watch?v=AaTRHFaaPG8 (Lex Fridman Interview)

FeepingCreature 3y ago

Honestly, I think any foundational work on the topic is inherently Yud-favored, compared to the blithe optimism and surface-level analysis at best that is usually applied to the topic. Ie, I think it's not that this shouldn't be done. This should certainly be done. It's just that so many more things than it should be done before we move forward.

Teever 3y ago

Why does this matter?

snapcaster 3y ago

Agreed, Yud does seem to have been right about the course things will take but I'm not confident he actually has any solutions to the problem to offer

qumpis 3y ago
Which things has he been right about and when, if you recall?
- snapcaster 3y ago
  At least to me, this current generation of AI progress is looking a lot more "foomy" than a lot of people besides him predicted
causalmodels 3y ago
His solution is a global regulatory regime to ban new large training runs. The tools required to accomplish this are, IMO, out of the question but I will give Yud credit for being honest about them while others who share his viewpoint try to hide the ball.
- snapcaster 3y ago
  Good point, but I think if he was serious he would be looking to Serena Butler for inspiration not nuclear non-proliferation treaties

bick_nyers 3y ago

These were my thoughts exactly. On one hand, this can enable alignment research to catch up faster. On the other hand, if we are worried about homicidal AI, then putting it in charge of policing itself (and training it to find exploits in a way) is probably not ideal.

ShamelessC 3y ago

You mean Yudkowski? I saw him on Lex Fridman and he was entirely unconvincing. Why is everyone deferring to a bunch of effective altruism advocates when it comes to AI safety?

ftxbro 3y ago
> Why is everyone deferring to a bunch of effective altruism advocates when it comes to AI safety? I'm not sure Yudkowski is an EA, but the EAs want him in their polycule.
- tomjakubowski 3y ago
  He posts on the forum. I'm not sure what more evidence is needed that he's part of it. https://forum.effectivealtruism.org/users/eliezeryudkowsky https://forum.effectivealtruism.org/users/eliezeryudkowsky
  - ftxbro 3y ago
    I guess it's true, not just a rationalist but also effective altruist!
lubesGordi 3y ago
I heard him on Lex too, and it seemed to be just a given that AI is going to be deceptive and want to kill us all. I don't think there was a single example of how that could be accomplished given. I'm open to hearing thoughts on this, maybe I'm not creative enough to see the 'obvious' ways this could happen.
- ethanbond 3y ago
  This is also why I go into chess matches against 1400 elo players. I cannot conceive of the specific ways in which they will beat me (a 600 elo player), so I have good reason to suspect that I can win. I'm willing to bet the future of our species on my consistent victory in these types of matches, in fact.

killthebuddha 3y ago

DISCLAIMER: I think Yudkowsky is a serious thinker and his ideas should be taken seriously, regardless of whether not they are correct. Your comment triggered a random thought: A perfect name for Yudkowsky et al and the AGI doomers is... wait for it... the Yuddites :)

throwaway2137 3y ago
Already used on 4chan :)