3 ms·

You hear similar arguments from those who believe climate change is happening but disagree with current efforts to counter-act it. The logic being that right no

by kypro 3y ago

You hear similar arguments from those who believe climate change is happening but disagree with current efforts to counter-act it. The logic being that right now climate change is not causing any major harm and that we can't really predict the future so there's no point in worrying about what might happen in a decade or two. I don't think anyone is arguing that right now climate change or AI is threat to human civilisation. The point is that there are clear trends in place and that those trends are concerning. On AI specifically, it's fairly easy to see how a slightly more advanced LLM could be a destructive force if it was given an unaligned goal by a malicious actor. For example, a slightly more advanced LLM could hack into critical infrastructure killing or injuring many thousands of people. In the near-future AI may help us advance biotech research and it could aid in the creation of bioweapons and other destructive capabilities. Longer-term risks (those maybe a couple of decades out) become much greater and also much harder to predict, but they're worth thinking about and planning for today. For example, what happens when humanity becomes dependant on AI for its labour, or when AI is controlling the majority of our infrastructure? I disagree but can understand the position that AI safety isn't humanities number one risk or priority right now, however I don't understand the dismissive attitude towards what seems like a clear existential risk when you project a decade or two out.

cmilton 3y ago
>that those trends are concerning. Which trends would you be referring to? >it's fairly easy to see how a slightly more advanced LLM could be a destructive force if it was given an unaligned goal by a malicious actor. For example, a slightly more advanced LLM could hack into critical infrastructure killing or injuring many thousands of people. How are you building this progression? Is there any evidence to back up this claim? I am having a hard time discerning this from fear-mongering.
- adamsmith143 3y ago
  If AI improves by 0.0001% per year on your favorite intelligence metric there will eventually be a point where it surpasses human performance and another point where it surpasses all humans combined on that metric. There is danger in that scenario.
  - cmilton 3y ago
    Assuming that timeline, can we agree that we have some time (years?) to hash this out further before succumbing to the ideals of a select few?
    - adamsmith143 3y ago
      The problem is that even with N years until we reach that point it seems likely that it would take 2*N years to build the proper safety mechanisms because at least currently capabilities research is racing far ahead of safety research. Of course we have no way to know how big N really is and recent results like GPT-4, Llama, Gato, etc. have shifted peoples timelines significantly. So even if 5 years ago people like Geoff Hinton though this might be 30-50 years away there are now believable arguments to make that it might be more like 3-10 years.
shrimpx 3y ago
> slightly more advanced LLM I don't think there is a path, that we know if, from GPT4 to a LLM that could take it upon itself to execute complex plans, etc. Current LLM tech 'fizzles out' exponentially in the size of the prompt, and I don't think we have a way out of that. We could speculate though... Basically AI risk proponents make a bunch of assumptions about how powerful next-level AI could be, but in reality we have no clue what this next-level AI is.
- NumberWangMan 3y ago
  When you have no clue, it makes sense to expand your confidence interval in both directions. So it could be a lot longer than we expect, but it could be shorter, too. You shouldn't just say "we have no clue, so it's probably further off and not worth worrying about". Especially since a lot of people made predictions about LLM capabilities and were surprised by how much better they worked than expected.
  - shrimpx 3y ago
    I agree in principle, it's just that the level of expressed worry doesn't seem to match reality. Currently we have no reasonable path to 'scary AGI'. It's some yet newfangled tech we haven't discovered. As an example, consider the invention of motion picture. People were totally bewildered that you can have moving things and people inside a picture. Scaremongers could start claiming "Pretty soon the moving things may come to life and take over the world! Before you know it, they'll run our factories from inside the movies and we'll be their slaves!" That's more or less what this 'scary AGI' hype sounds like to me right now. Btw "That Mitchell and Webb Look" is a great show ;-)

by kypro 3y ago

cmilton 3y ago

>that those trends are concerning. Which trends would you be referring to? >it's fairly easy to see how a slightly more advanced LLM could be a destructive force if it was given an unaligned goal by a malicious actor. For example, a slightly more advanced LLM could hack into critical infrastructure killing or injuring many thousands of people. How are you building this progression? Is there any evidence to back up this claim? I am having a hard time discerning this from fear-mongering.

adamsmith143 3y ago
If AI improves by 0.0001% per year on your favorite intelligence metric there will eventually be a point where it surpasses human performance and another point where it surpasses all humans combined on that metric. There is danger in that scenario.
- cmilton 3y ago
  Assuming that timeline, can we agree that we have some time (years?) to hash this out further before succumbing to the ideals of a select few?
  - adamsmith143 3y ago
    The problem is that even with N years until we reach that point it seems likely that it would take 2*N years to build the proper safety mechanisms because at least currently capabilities research is racing far ahead of safety research. Of course we have no way to know how big N really is and recent results like GPT-4, Llama, Gato, etc. have shifted peoples timelines significantly. So even if 5 years ago people like Geoff Hinton though this might be 30-50 years away there are now believable arguments to make that it might be more like 3-10 years.

shrimpx 3y ago

> slightly more advanced LLM I don't think there is a path, that we know if, from GPT4 to a LLM that could take it upon itself to execute complex plans, etc. Current LLM tech 'fizzles out' exponentially in the size of the prompt, and I don't think we have a way out of that. We could speculate though... Basically AI risk proponents make a bunch of assumptions about how powerful next-level AI could be, but in reality we have no clue what this next-level AI is.

NumberWangMan 3y ago
When you have no clue, it makes sense to expand your confidence interval in both directions. So it could be a lot longer than we expect, but it could be shorter, too. You shouldn't just say "we have no clue, so it's probably further off and not worth worrying about". Especially since a lot of people made predictions about LLM capabilities and were surprised by how much better they worked than expected.
- shrimpx 3y ago
  I agree in principle, it's just that the level of expressed worry doesn't seem to match reality. Currently we have no reasonable path to 'scary AGI'. It's some yet newfangled tech we haven't discovered. As an example, consider the invention of motion picture. People were totally bewildered that you can have moving things and people inside a picture. Scaremongers could start claiming "Pretty soon the moving things may come to life and take over the world! Before you know it, they'll run our factories from inside the movies and we'll be their slaves!" That's more or less what this 'scary AGI' hype sounds like to me right now. Btw "That Mitchell and Webb Look" is a great show ;-)