5 ms·

> What a limited imagination of how an AI would take over. I think you've missed the point of the article. It's not intended to be a picture of how AIs would

> What a limited imagination of how an AI would take over. I think you've missed the point of the article. It's not intended to be a picture of how AIs would actually take over, but rather a lower bound -- a demonstration that AIs could take over even if they are restricted in this way.

kelseyfrog 4y ago
Right, and my point is that this framing is itself dangerously naive.
- Sniffnoy 4y ago
  Again, if you are saying "actually it would be much worse than that", then you aren't disagreeing with the article, you are expanding on it, because the article is deliberately only saying "it would be at least this bad". In no place does it make any claim about an upper bound, so you are not contradicting it. It's confusing to the reader when you frame expanding on someone else's point as if you were instead disagreeing with it. Perhaps you are thinking that a person who doesn't read carefully would get the mistaken impression that the article is claiming "things would be about this bad", instead of the intended "things would be at least this bad (which likely means far worse)"? If so, then you should make that explicit, because then you are once again not contradicting the article, but rather a sloppy thinker's mistaken impression of it. (If I write, for instance, "The dinosaurs lived over a million years ago", the correct response is not, "That is wrong, they lived over 65 million years ago", it is "In fact, more than that is true; they lived over 65 million years ago". The difference is important for not wasting time on imaginary disagreements!) Stating a lower bound does not in any way imply that one believes that lower bound to be anywhere near tight! Especially I don't know why you've inferred that Karnofsky cannot imagine worse scenarios. Given the circles he's in, I would be very surprised if he cannot. He's not including worse scenarios because they're not relevant to the point he's making, which is that this is a lower bound. In fact, we can say more than that -- he's not including worse scenarios because he wants to make the argument as airtight as possible for the skeptical. He wants to use arguments that are as strong as possible to establish a lower bound, not establish a position that is as strong as possible, because that would require using arguments that many would find less acceptable. (This by the way is an illustration by the way of the more general principle of why trying to talk about the author, rather than the article, is a mistake. It's a lot easier to make mistaken inferences about authors! Stick to discussing the actual ideas and you don't have to worry about those mistakes.)
  - deleted 4y ago
    [deleted]
  - kelseyfrog 4y ago
    This sounds very important to you. I give you permission rewrite my comment so that it contains the meaning and wording that is most pleasing to you. I hereby license it under CC0.

kelseyfrog 4y ago

Right, and my point is that this framing is itself dangerously naive.

Sniffnoy 4y ago
Again, if you are saying "actually it would be much worse than that", then you aren't disagreeing with the article, you are expanding on it, because the article is deliberately only saying "it would be at least this bad". In no place does it make any claim about an upper bound, so you are not contradicting it. It's confusing to the reader when you frame expanding on someone else's point as if you were instead disagreeing with it. Perhaps you are thinking that a person who doesn't read carefully would get the mistaken impression that the article is claiming "things would be about this bad", instead of the intended "things would be at least this bad (which likely means far worse)"? If so, then you should make that explicit, because then you are once again not contradicting the article, but rather a sloppy thinker's mistaken impression of it. (If I write, for instance, "The dinosaurs lived over a million years ago", the correct response is not, "That is wrong, they lived over 65 million years ago", it is "In fact, more than that is true; they lived over 65 million years ago". The difference is important for not wasting time on imaginary disagreements!) Stating a lower bound does not in any way imply that one believes that lower bound to be anywhere near tight! Especially I don't know why you've inferred that Karnofsky cannot imagine worse scenarios. Given the circles he's in, I would be very surprised if he cannot. He's not including worse scenarios because they're not relevant to the point he's making, which is that this is a lower bound. In fact, we can say more than that -- he's not including worse scenarios because he wants to make the argument as airtight as possible for the skeptical. He wants to use arguments that are as strong as possible to establish a lower bound, not establish a position that is as strong as possible, because that would require using arguments that many would find less acceptable. (This by the way is an illustration by the way of the more general principle of why trying to talk about the author, rather than the article, is a mistake. It's a lot easier to make mistaken inferences about authors! Stick to discussing the actual ideas and you don't have to worry about those mistakes.)
- deleted 4y ago
  [deleted]
- kelseyfrog 4y ago
  This sounds very important to you. I give you permission rewrite my comment so that it contains the meaning and wording that is most pleasing to you. I hereby license it under CC0.