4 ms·

> The technology they're working on poses the gravest of dangers Extraordinary claims require extraordinary evidence.

mofeien 2y ago
ChatGPT-4 won't kill us, and probably neither will the next generation of models. But if we keep building more and more powerful AI systems, eventually we will reach a point where one will become an existential threat. Present dangers of AI include Fake News, impersonation through deepfakes, biases and discrimination in the models, job loss and economic instability, loss of human purpose through individuals becoming redundant, individuals becoming addicted to the AI systems, use by authoritarian governments, autonomous weapons. Near future dangers include bioweapon engineering capabilities, computer hacks, human disempowerment, existential risk, or a state where the values of the world are locked in by an uncontrolled AI, and possibly filled with suffering if locked in in a bad way, e. g. as we lock in animals. It might take months until AGI, it might take decades, nobody knows for sure. However, we do know that the pace of AI progress is often grossly underestimated. Just three years ago we thought we’d have SAT-passing AI systems in 2055. We got there in April 2023. So it makes sense to act as if we have very little time.
- jeffreygoesto 2y ago
  Reminds me of "now automated driving is only five years away" since 2008...
- bitnasty 2y ago
  You, like many others, miss the number one danger of AI—making people dumber and less capable of reason.
  - ben_w 2y ago
    People have been saying that about everything at least since Socratese was dismissive of writing; probably longer, but we can't tell because writing hadn't been invented yet.
- iudqnolq 2y ago
  If being able to write lots of convincing text could give you immense power why haven't any of the billions of humans that can already do that done so?
  - ben_w 2y ago
    The most competent at that, did so, and are immensely powerful. That's basically all of politics, and not only in a democracy. You do have to be significantly above average to get there. Current generation LLMs are a bit above the human average, enough to be interesting, but probably not enough to be a personal Goebels for every would-be dictator.
    - iudqnolq 2y ago
      Current LLMs are obviously far below the average adult human. Every demo I've ever seen brags about approaching the competency of an ordinary high school intern. Even if they reach above average our whole society is structured around managing billions of above average humans. What's novel?
      - ben_w 2y ago
        > Current LLMs are obviously far below the average adult human. Every demo I've ever seen brags about approaching the competency of an ordinary high school intern. There's a lot of bragging, sure. But what I've seen from the OpenAI models is more like a university level intern or a fresh graduate. Not that it matters, as the point is to aim for where the ball is going rather than where it is now, but I'd say that this makes them above the average human performance (at least, within its domain). > Even if they reach above average our whole society is structured around managing billions of above average humans. That's putting a binary cutoff in an arbitrary place; half the population is always above average on any measure (or indeed topic), what I wrote was "significantly above average". To crudely approximate with IQ, which is a bad measure for humans and much worse measure for AI, there's just over a billion humans of IQ > 115, around 182 million of IQ > 130, and around 8 million of IQ > 145. I'm not sure how much to trust estimates of politician's IQs given the tribalism involved, but surveys of other groups tend to show that around half of the highest performing leaders (CEOs etc) with the most power, are in the top 1% by IQ. (Who knows when, or if, someone will make an AI that's good at the important tasks within of each of those roles). > What's novel? Two things: 1) Breadth: the current vogue is very general models, so they're interns sure (even though I rank them as a higher level of intern than you), but they are interns at everything — there is no human you can hire with a even a mediocre grasp of Mandarin and Arabic and Welsh and python and CSS and calculus and magnetohydrodynamics and psychology and … 2) Price: there is no human who can read a million tokens for a price equal to two days of the UN abject poverty threshold, or write a million tokens for six days of the UN abject poverty threshold. You physically can't work that fast, even if you were typing [a] then [space] ten times a second for 18 hours a day; and you basically can't feed yourself if you earn less than that per day.
- d0gsg0w00f 2y ago
  I'm not that worried because humans are fickle creatures with an insanely sensitive trust/BS detector. One bad interaction out of 200 with a human/model and the human won't trust that human/model anymore. They'll categorize that human/model as a cute toy and look elsewhere for critical decisions or always get a second opinion. My prediction is that digital content will end up just being "cute" and the real interactions/decisions will increasingly be offline and face-to-face. The open digital democracy that everyone hoped for will become tightly closed circles of humans talking face to face.

ChatGPT-4 won't kill us, and probably neither will the next generation of models. But if we keep building more and more powerful AI systems, eventually we will reach a point where one will become an existential threat. Present dangers of AI include Fake News, impersonation through deepfakes, biases and discrimination in the models, job loss and economic instability, loss of human purpose through individuals becoming redundant, individuals becoming addicted to the AI systems, use by authoritarian governments, autonomous weapons. Near future dangers include bioweapon engineering capabilities, computer hacks, human disempowerment, existential risk, or a state where the values of the world are locked in by an uncontrolled AI, and possibly filled with suffering if locked in in a bad way, e. g. as we lock in animals. It might take months until AGI, it might take decades, nobody knows for sure. However, we do know that the pace of AI progress is often grossly underestimated. Just three years ago we thought we’d have SAT-passing AI systems in 2055. We got there in April 2023. So it makes sense to act as if we have very little time.

jeffreygoesto 2y ago
Reminds me of "now automated driving is only five years away" since 2008...
bitnasty 2y ago
You, like many others, miss the number one danger of AI—making people dumber and less capable of reason.
- ben_w 2y ago
  People have been saying that about everything at least since Socratese was dismissive of writing; probably longer, but we can't tell because writing hadn't been invented yet.
iudqnolq 2y ago
If being able to write lots of convincing text could give you immense power why haven't any of the billions of humans that can already do that done so?
- ben_w 2y ago
  The most competent at that, did so, and are immensely powerful. That's basically all of politics, and not only in a democracy. You do have to be significantly above average to get there. Current generation LLMs are a bit above the human average, enough to be interesting, but probably not enough to be a personal Goebels for every would-be dictator.
  - iudqnolq 2y ago
    Current LLMs are obviously far below the average adult human. Every demo I've ever seen brags about approaching the competency of an ordinary high school intern. Even if they reach above average our whole society is structured around managing billions of above average humans. What's novel?
    - ben_w 2y ago
      > Current LLMs are obviously far below the average adult human. Every demo I've ever seen brags about approaching the competency of an ordinary high school intern. There's a lot of bragging, sure. But what I've seen from the OpenAI models is more like a university level intern or a fresh graduate. Not that it matters, as the point is to aim for where the ball is going rather than where it is now, but I'd say that this makes them above the average human performance (at least, within its domain). > Even if they reach above average our whole society is structured around managing billions of above average humans. That's putting a binary cutoff in an arbitrary place; half the population is always above average on any measure (or indeed topic), what I wrote was "significantly above average". To crudely approximate with IQ, which is a bad measure for humans and much worse measure for AI, there's just over a billion humans of IQ > 115, around 182 million of IQ > 130, and around 8 million of IQ > 145. I'm not sure how much to trust estimates of politician's IQs given the tribalism involved, but surveys of other groups tend to show that around half of the highest performing leaders (CEOs etc) with the most power, are in the top 1% by IQ. (Who knows when, or if, someone will make an AI that's good at the important tasks within of each of those roles). > What's novel? Two things: 1) Breadth: the current vogue is very general models, so they're interns sure (even though I rank them as a higher level of intern than you), but they are interns at everything — there is no human you can hire with a even a mediocre grasp of Mandarin and Arabic and Welsh and python and CSS and calculus and magnetohydrodynamics and psychology and … 2) Price: there is no human who can read a million tokens for a price equal to two days of the UN abject poverty threshold, or write a million tokens for six days of the UN abject poverty threshold. You physically can't work that fast, even if you were typing [a] then [space] ten times a second for 18 hours a day; and you basically can't feed yourself if you earn less than that per day.
d0gsg0w00f 2y ago
I'm not that worried because humans are fickle creatures with an insanely sensitive trust/BS detector. One bad interaction out of 200 with a human/model and the human won't trust that human/model anymore. They'll categorize that human/model as a cute toy and look elsewhere for critical decisions or always get a second opinion. My prediction is that digital content will end up just being "cute" and the real interactions/decisions will increasingly be offline and face-to-face. The open digital democracy that everyone hoped for will become tightly closed circles of humans talking face to face.