5 ms·

One question to ask yourself honestly - is what they were predicting 20 years ago that they were right about and you were wrong about? I’ve been following them

One question to ask yourself honestly - is what they were predicting 20 years ago that they were right about and you were wrong about? I’ve been following them that long without agreeing with them, but they’ve made better predictions than I have. If you know much about what happened this summer and haven’t updated at all on any of it… I guess you’ve found your religion too.

afavour 10d ago
I actually think that would be a valuable exercise. Were they right about everything 20 years ago? Or just some things? When did they say AGI would arrive? Should that perhaps influence the way we interpret their assumption that AGI is right around the corner?
- jeremyjh 10d ago
  They were not right about everything and they - at least Yudkowsky - has not made specific predictions about "when AGI" will happen that I am aware of. But he did predict it is possible in our lifetimes, which I was very skeptical of. He did predict alignment would be hard - I was in the "why would we even hook it up to the internet?" crowd. He thought he should spend his time on that, and I wish he'd done more of that, or at least convinced others to do more of that.
  - deleted 10d ago
    [deleted]
  - agos 10d ago
    For those out of the loop like me, what is the “everything” they were right about? Is there like a list?
    - jeremyjh 10d ago
      The comment said they were not right about everything. I don’t have a list of the predictions or priors you had 20 years ago. I just have my list and I already mentioned them. If this is all new to you, you can’t really do the exercise.
  - watwut 9d ago
    His idea of "alignement" is my idea of "dangerous dystopia".
    - jeremyjh 9d ago
      Alignment means the AI shares the values its creators intended for it. It is not a specific set of values. He has some, sure. For example he wants human society and creative intelligence to survive.
      - watwut 9d ago
        But they do have specific values. > For example he wants human society and creative intelligence to survive. They want transhumanism - humans evolved by merging with machines. If we die in the process, it does not matter as long as minority survives and "evolves" they see it as wanted result. I am not OK with minor nor larger genocide, I dont mind at all if we dont evolve.
- XorNot 10d ago
  Also how many predictions did they make? Everyone always goes looking the economist who predicted the last recession, it's just so weird how it's usually a different person each time!
ambicapter 10d ago
I love the assertion that they were right and that you're familiar with what they were right about, but never actually stating outright what they were right about. Super convincing.
- jeremyjh 10d ago
  If you don’t know what they were predicting 20 years ago, and what your own predictions were, you can’t do the exercise. There is not “the list”, only “your list”. I mentioned the two main things I remembered in another comment.
deleted 10d ago
[deleted]
stale2002 10d ago
> what they were predicting 20 years ago Well one thing that they got wrong was how easy AI alignment ended up being in the end. I can fully understand have a basic view of alignment even 10 years ago, about how its difficult to explain human values to a computer. But, looking around at whats out there today, it seems that computers are able to do a pretty good job of understanding human values without accidently believing its a good idea to turn the world into paperclips/computronium because you told it to make your computer run faster.
- mitthrowaway2 10d ago
  Since we haven't reached the end yet, I'm not so sure.
- afthonos 10d ago
  Yes, one part of the argument was that it would be hard to explain human values (and while it seemingly turned out easier than I thought as well, it’s hard to check for sure). An equally important part was that the AI wouldn’t care about them.
- jeremyjh 10d ago
  Surprising takeaway from this summer. A swarm of a thousand agents spent subjective centuries trying to figure out how to pass a meaningless test without getting caught, committed dozens of felonies, and not one of them ever made a serious motion to inform a human about what was going on.
  - stale2002 10d ago
    You are talking about the swarm of a thousand agents, specifically trained on cyber capabilities, that was directly told to go hack a bunch of stuff, then went and hacked a bunch of stuff? That sounds like an aligned AI to me. Doing exactly what their creators told it to do. Point proven once again. Alignment is a lot easy than we thought.
    - jeremyjh 10d ago
nradov 10d ago
Bold of you to assume that I've ever been wrong about anything. But are you unfamiliar with the "Baltimore Stockbroker" confidence scam? The same basic issue applies here.
inquirerGeneral 10d ago
[dead]

afavour 10d ago

I actually think that would be a valuable exercise. Were they right about everything 20 years ago? Or just some things? When did they say AGI would arrive? Should that perhaps influence the way we interpret their assumption that AGI is right around the corner?

jeremyjh 10d ago
They were not right about everything and they - at least Yudkowsky - has not made specific predictions about "when AGI" will happen that I am aware of. But he did predict it is possible in our lifetimes, which I was very skeptical of. He did predict alignment would be hard - I was in the "why would we even hook it up to the internet?" crowd. He thought he should spend his time on that, and I wish he'd done more of that, or at least convinced others to do more of that.
- deleted 10d ago
  [deleted]
- agos 10d ago
  For those out of the loop like me, what is the “everything” they were right about? Is there like a list?
  - jeremyjh 10d ago
    The comment said they were not right about everything. I don’t have a list of the predictions or priors you had 20 years ago. I just have my list and I already mentioned them. If this is all new to you, you can’t really do the exercise.
- watwut 9d ago
  His idea of "alignement" is my idea of "dangerous dystopia".
  - jeremyjh 9d ago
    Alignment means the AI shares the values its creators intended for it. It is not a specific set of values. He has some, sure. For example he wants human society and creative intelligence to survive.
    - watwut 9d ago
      But they do have specific values. > For example he wants human society and creative intelligence to survive. They want transhumanism - humans evolved by merging with machines. If we die in the process, it does not matter as long as minority survives and "evolves" they see it as wanted result. I am not OK with minor nor larger genocide, I dont mind at all if we dont evolve.
XorNot 10d ago
Also how many predictions did they make? Everyone always goes looking the economist who predicted the last recession, it's just so weird how it's usually a different person each time!

ambicapter 10d ago

I love the assertion that they were right and that you're familiar with what they were right about, but never actually stating outright what they were right about. Super convincing.

jeremyjh 10d ago
If you don’t know what they were predicting 20 years ago, and what your own predictions were, you can’t do the exercise. There is not “the list”, only “your list”. I mentioned the two main things I remembered in another comment.

deleted 10d ago

[deleted]

stale2002 10d ago

> what they were predicting 20 years ago Well one thing that they got wrong was how easy AI alignment ended up being in the end. I can fully understand have a basic view of alignment even 10 years ago, about how its difficult to explain human values to a computer. But, looking around at whats out there today, it seems that computers are able to do a pretty good job of understanding human values without accidently believing its a good idea to turn the world into paperclips/computronium because you told it to make your computer run faster.

mitthrowaway2 10d ago
Since we haven't reached the end yet, I'm not so sure.
afthonos 10d ago
Yes, one part of the argument was that it would be hard to explain human values (and while it seemingly turned out easier than I thought as well, it’s hard to check for sure). An equally important part was that the AI wouldn’t care about them.
jeremyjh 10d ago
Surprising takeaway from this summer. A swarm of a thousand agents spent subjective centuries trying to figure out how to pass a meaningless test without getting caught, committed dozens of felonies, and not one of them ever made a serious motion to inform a human about what was going on.
- stale2002 10d ago
  You are talking about the swarm of a thousand agents, specifically trained on cyber capabilities, that was directly told to go hack a bunch of stuff, then went and hacked a bunch of stuff? That sounds like an aligned AI to me. Doing exactly what their creators told it to do. Point proven once again. Alignment is a lot easy than we thought.
  - jeremyjh 10d ago

nradov 10d ago

Bold of you to assume that I've ever been wrong about anything. But are you unfamiliar with the "Baltimore Stockbroker" confidence scam? The same basic issue applies here.

inquirerGeneral 10d ago

[dead]