8 ms·

Yudkowsky claimed that the exact basilisk as described by Roko wouldn't work, but he also believes that something like it would.

by Jiro 5y ago

thewakalix 5y ago
Sort of. He believes that if people try to come up with infohazards and then immediately spread them, there’s a nonzero chance that they succeed. It’s unlikely, but still not something to encourage. (This is not the same thing as “being certain that a minor modification of Roko’s basilisk would produce an evil time-travelling AI,” to be clear.)
- user-the-name 5y ago
  This kind of utter gibberish is exactly why people don't take LessWrong seriously.
  - thewakalix 5y ago
    “Ideas can be harmful” is gibberish?
    - Udo 5y ago
      The amount of self-importance on display is what makes it sound like gibberish. A: "I have the most ground-breaking idea ever!" B: "Okay, let's hear it?" A: "No, it's too dangerous, it may well destroy the world, but trust me it's brilliant and completely novel." It's a religion at that point.
      - traverseda 5y ago
        How about bitcoin? It's not so much an idea as a piece of software, but being predicted on wasting energy it's going to do a lot of harm. That's an example of this class of game-theory based ideas. I believe that there are other ideas like that, things that set up spirals that are a lot worse than just "destroy energy to make money". Obviously they're not going to be universally compelling, an idea that's dangerous like that in one context might be perfectly safe in another. It depends on the target audience.
        Udo 5y ago
        > That's an example of this class of game-theory based ideas. That's a good point on its own, but it's not a response. At no time did I or anyone here in this thread assert that "info hazards" do not exist. What is being discussed is whether it's worthwhile to be deferential to a person/community who is making claims of being in possession of an exceptionally catastrophic piece of information that by its nature would preclude public scrutiny on ethical grounds.
        traverseda 5y ago
        I don't think Yudkowsky is making that claim anywhere? >At no time did I or anyone here in this thread assert that "info hazards" do not exist. >What is being discussed is whether it's worthwhile to be deferential to a person who is making claims It's honestly really not that clear what you're trying to discuss, certainly you haven't been making a clear statement like that. I don't think anyone is making claims of being in possession of such an infohazard for one thing, and if there were I don't think anyone would be saying you should be deferential towards someone who claimed to have one, sight unseen. I think you're arguing with a bit of a straw man? Like no one is being all "let's be deferential towards Yudkowsky because he has the idea equivalent of a nuke". He doesn't, to the best of my knowledge, claim to have the idea equivalent of a nuke. I assumed you were claiming that info-hazards don't exist because otherwise what exactly are you claiming? Anyway, if that was your point, granted. Let's not be differential towards people who claim to have info-hazards, unless they can go into a room and make someone laugh themselves to death or something. https://www.youtube.com/watch?v=FBWr1KtnRcI https://www.youtube.com/watch?v=FBWr1KtnRcI
        mhb 5y ago
        He doesn't, to the best of my knowledge, claim to have the idea equivalent of a nuke. Well, he isn't willing to reveal how he convinced his opponents to let the AI out of the box.
        traverseda 5y ago
      - deleted 5y ago
        [deleted]
      - medstrom 5y ago
        Can you link me to when anyone on LessWrong has behaved like your "A"?
    - user-the-name 5y ago
      Now you've just generalised the statement until it becomes a truism. So you are now essentially claiming the statement is garbage because it is devoid of useful meaning.
deleted 5y ago
[deleted]

Yudkowsky claimed that the exact basilisk as described by Roko wouldn't work, but he also believes that something like it would.

by Jiro 5y ago

Yudkowsky claimed that the exact basilisk as described by Roko wouldn't work, but he also believes that something like it would.

thewakalix 5y ago

Sort of. He believes that if people try to come up with infohazards and then immediately spread them, there’s a nonzero chance that they succeed. It’s unlikely, but still not something to encourage. (This is not the same thing as “being certain that a minor modification of Roko’s basilisk would produce an evil time-travelling AI,” to be clear.)

user-the-name 5y ago
This kind of utter gibberish is exactly why people don't take LessWrong seriously.
- thewakalix 5y ago
  “Ideas can be harmful” is gibberish?
  - Udo 5y ago
    The amount of self-importance on display is what makes it sound like gibberish. A: "I have the most ground-breaking idea ever!" B: "Okay, let's hear it?" A: "No, it's too dangerous, it may well destroy the world, but trust me it's brilliant and completely novel." It's a religion at that point.
    - traverseda 5y ago
      How about bitcoin? It's not so much an idea as a piece of software, but being predicted on wasting energy it's going to do a lot of harm. That's an example of this class of game-theory based ideas. I believe that there are other ideas like that, things that set up spirals that are a lot worse than just "destroy energy to make money". Obviously they're not going to be universally compelling, an idea that's dangerous like that in one context might be perfectly safe in another. It depends on the target audience.
      - Udo 5y ago
        > That's an example of this class of game-theory based ideas. That's a good point on its own, but it's not a response. At no time did I or anyone here in this thread assert that "info hazards" do not exist. What is being discussed is whether it's worthwhile to be deferential to a person/community who is making claims of being in possession of an exceptionally catastrophic piece of information that by its nature would preclude public scrutiny on ethical grounds.
        traverseda 5y ago
        I don't think Yudkowsky is making that claim anywhere? >At no time did I or anyone here in this thread assert that "info hazards" do not exist. >What is being discussed is whether it's worthwhile to be deferential to a person who is making claims It's honestly really not that clear what you're trying to discuss, certainly you haven't been making a clear statement like that. I don't think anyone is making claims of being in possession of such an infohazard for one thing, and if there were I don't think anyone would be saying you should be deferential towards someone who claimed to have one, sight unseen. I think you're arguing with a bit of a straw man? Like no one is being all "let's be deferential towards Yudkowsky because he has the idea equivalent of a nuke". He doesn't, to the best of my knowledge, claim to have the idea equivalent of a nuke. I assumed you were claiming that info-hazards don't exist because otherwise what exactly are you claiming? Anyway, if that was your point, granted. Let's not be differential towards people who claim to have info-hazards, unless they can go into a room and make someone laugh themselves to death or something. https://www.youtube.com/watch?v=FBWr1KtnRcI https://www.youtube.com/watch?v=FBWr1KtnRcI
        mhb 5y ago
        He doesn't, to the best of my knowledge, claim to have the idea equivalent of a nuke. Well, he isn't willing to reveal how he convinced his opponents to let the AI out of the box.
        traverseda 5y ago
    - deleted 5y ago
      [deleted]
    - medstrom 5y ago
      Can you link me to when anyone on LessWrong has behaved like your "A"?
  - user-the-name 5y ago
    Now you've just generalised the statement until it becomes a truism. So you are now essentially claiming the statement is garbage because it is devoid of useful meaning.

deleted 5y ago

[deleted]