5 ms·

What is so impressive about ChatGPT that it poses an existential threat? Many experts are highly critical (e.g. LeCun) and/or believe LLMs are nothing more than

What is so impressive about ChatGPT that it poses an existential threat? Many experts are highly critical (e.g. LeCun) and/or believe LLMs are nothing more than stochastic parrots. Anyone who was working with transformers could have seen ChatGPT on the horizon, it wasn’t surprising at all that scaling an autoregressive model can result in something seemingly intelligent. Where is 1 in 10 coming from? Is this a ‘gut feeling’ because one does not understand LLMs or is this factually based? What is my estimate for AI being an existential risk in the next couple of decades? Depends on if we find something that actually resembles AGI which is impossible to predict. Based solely on current technology + scaling I would personally put the chance at essentially 0%.

digging 3y ago
> Where is 1 in 10 coming from? Is this a ‘gut feeling’ because one does not understand LLMs or is this factually based? It's a totally made up example. Its purpose was to engage you in a conversation about what's reasonable if you believe an existential threat is possible in the near term. It doesn't seem you are willing to engage that possibility. > Anyone who was working with transformers could have seen ChatGPT on the horizon So, who did? > [LeCun is] highly critical I did a quick search and found this[1] trash-tier article explaining his views summed as: > He thinks the will to dominate and intelligence are separate things. Orangutan do not have a desire to dominate but are intelligent. They are territorial. I have absolutely no idea where this distinction comes from. Being territorial is a form of domination, it's just not the same as expansionist behavior. If an AI is territorial, and the Earth is its territory, it doesn't need to be expansionist to annihilate threats on Earth. Anyway, LeCun's former colleague Hinton was on NPR yesterday telling the masses that he thinks we could have dangerous AI in less than a decade. It's fair to say there's wide disagreement. I don't think it's fair to say that handwaving means we shouldn't be worried about the problem. Finally, your question: > What is so impressive about ChatGPT that it poses an existential threat? This is a bit of a logical misstep, or I was unclear previously. I don't think anyone sees ChatGPT or anything directly related to it as an existential threat. Rather, the rate of change in capabilities of LLMs is worrying, because if we see a similar rate of change in systems with more agency, it could be a threat. Or, in the words of the article about LeCun's belief: > Lecun says that in a few year LLM (Large Language Models) will go away and replaced with systems that will be guidable to desired goals. So... he is predicting exactly the same situation that everyone is afraid of, and he's asserting that it will work out fine. Somehow. The point is not that any one particular situation will occur. It's that there are an unknown number of factors in what makes an AI "smart enough" and in what makes it "misaligned". There are hypothetical scenarios where a hyperintelligent AI creates goals for itself that necessitate our extinction, and we don't know how those scenarios come into existence or how to stop them. Assuming they won't happen is just burying our heads in the sand. > essentially 0%. I sure hope you're right, but I don't think that's a common position to hold. [1] https://www.nextbigfuture.com/2023/05/metas-yann-lecun-is-confident-that-ai-will-be-safe-and-great.html https://www.nextbigfuture.com/2023/05/metas-yann-lecun-is-co...
- haldujai 3y ago
  Hinton has been exaggerating the capabilities of AI for nearly a decade. As one example, if we listened to him in 2016/17 we would have an even worse radiologist shortage than we do right now. > The point is not that any one particular situation will occur. It's that there are an unknown number of factors in what makes an AI "smart enough" and in what makes it "misaligned". There are hypothetical scenarios where a hyperintelligent AI creates goals for itself that necessitate our extinction, and we don't know how those scenarios come into existence or how to stop them. Assuming they won't happen is just burying our heads in the sand. It’s not assuming it won’t happen or burying our heads in the sands, it’s calling for measured responses. My point is that we can worry and have the discussion without being alarmist and enacting premature regulation. Autonomous AGI, whenever that happens, absolutely poses a threat. But it’s extremely exaggerated to think what we have today, or even what is known on the horizon, represents that. People assume advancements will continue to happen at their current pace. The last 6 years of advancements has been the product of scaling and refining self-attention + transformers from 2017/8. > so who did I’m not sure what you’re getting at. No one had the will/resources to burn billions on a language model that has ??? business plan, doesn’t mean no one saw it coming.
  - digging 3y ago
    > Hinton has been exaggerating the capabilities of AI for nearly a decade. As one example, if we listened to him in 2016/17 we would have an even worse radiologist shortage than we do right now. I can appreciate that. I haven't been listening to him for that long; having no idea what situation you're discussing, I'll take it at face value. > My point is that we can worry and have the discussion without being alarmist and enacting premature regulation. That's pretty reasonable. However, I will reiterate that this: > it’s extremely exaggerated to think what we have today, or even what is known on the horizon, represents that. is not what I'm asserting. It's the unknowns which are dangerous. We're inventing something new in almost every way imaginable. It's not like we're talking about a new theoretical steelmaking process which could upset the economy, we're talking about the potential to build a new life form with godlike abilities. Alarm seems warranted to me. Sure, it might not happen soon... but also, it might happen soon. Shouldn't we be trying to prevent it, or control it, before it happens? > People assume advancements will continue to happen at their current pace. I think people assume advancements will accelerate, actually.
a257 3y ago
When it comes to the question of when strong AI will come, I think our society's reaction to ChatGPT is more worrying than the technology itself. Namely, because of ChatGPT, we see major companies, governments, etc clamoring over AI today like never before. The magnitude of this hype and its resulting funding rush is unprecedented. Nobody could have predicted that this would have happened so quickly.

digging 3y ago

> Where is 1 in 10 coming from? Is this a ‘gut feeling’ because one does not understand LLMs or is this factually based? It's a totally made up example. Its purpose was to engage you in a conversation about what's reasonable if you believe an existential threat is possible in the near term. It doesn't seem you are willing to engage that possibility. > Anyone who was working with transformers could have seen ChatGPT on the horizon So, who did? > [LeCun is] highly critical I did a quick search and found this[1] trash-tier article explaining his views summed as: > He thinks the will to dominate and intelligence are separate things. Orangutan do not have a desire to dominate but are intelligent. They are territorial. I have absolutely no idea where this distinction comes from. Being territorial is a form of domination, it's just not the same as expansionist behavior. If an AI is territorial, and the Earth is its territory, it doesn't need to be expansionist to annihilate threats on Earth. Anyway, LeCun's former colleague Hinton was on NPR yesterday telling the masses that he thinks we could have dangerous AI in less than a decade. It's fair to say there's wide disagreement. I don't think it's fair to say that handwaving means we shouldn't be worried about the problem. Finally, your question: > What is so impressive about ChatGPT that it poses an existential threat? This is a bit of a logical misstep, or I was unclear previously. I don't think anyone sees ChatGPT or anything directly related to it as an existential threat. Rather, the rate of change in capabilities of LLMs is worrying, because if we see a similar rate of change in systems with more agency, it could be a threat. Or, in the words of the article about LeCun's belief: > Lecun says that in a few year LLM (Large Language Models) will go away and replaced with systems that will be guidable to desired goals. So... he is predicting exactly the same situation that everyone is afraid of, and he's asserting that it will work out fine. Somehow. The point is not that any one particular situation will occur. It's that there are an unknown number of factors in what makes an AI "smart enough" and in what makes it "misaligned". There are hypothetical scenarios where a hyperintelligent AI creates goals for itself that necessitate our extinction, and we don't know how those scenarios come into existence or how to stop them. Assuming they won't happen is just burying our heads in the sand. > essentially 0%. I sure hope you're right, but I don't think that's a common position to hold. [1] https://www.nextbigfuture.com/2023/05/metas-yann-lecun-is-confident-that-ai-will-be-safe-and-great.html https://www.nextbigfuture.com/2023/05/metas-yann-lecun-is-co...

haldujai 3y ago
Hinton has been exaggerating the capabilities of AI for nearly a decade. As one example, if we listened to him in 2016/17 we would have an even worse radiologist shortage than we do right now. > The point is not that any one particular situation will occur. It's that there are an unknown number of factors in what makes an AI "smart enough" and in what makes it "misaligned". There are hypothetical scenarios where a hyperintelligent AI creates goals for itself that necessitate our extinction, and we don't know how those scenarios come into existence or how to stop them. Assuming they won't happen is just burying our heads in the sand. It’s not assuming it won’t happen or burying our heads in the sands, it’s calling for measured responses. My point is that we can worry and have the discussion without being alarmist and enacting premature regulation. Autonomous AGI, whenever that happens, absolutely poses a threat. But it’s extremely exaggerated to think what we have today, or even what is known on the horizon, represents that. People assume advancements will continue to happen at their current pace. The last 6 years of advancements has been the product of scaling and refining self-attention + transformers from 2017/8. > so who did I’m not sure what you’re getting at. No one had the will/resources to burn billions on a language model that has ??? business plan, doesn’t mean no one saw it coming.
- digging 3y ago
  > Hinton has been exaggerating the capabilities of AI for nearly a decade. As one example, if we listened to him in 2016/17 we would have an even worse radiologist shortage than we do right now. I can appreciate that. I haven't been listening to him for that long; having no idea what situation you're discussing, I'll take it at face value. > My point is that we can worry and have the discussion without being alarmist and enacting premature regulation. That's pretty reasonable. However, I will reiterate that this: > it’s extremely exaggerated to think what we have today, or even what is known on the horizon, represents that. is not what I'm asserting. It's the unknowns which are dangerous. We're inventing something new in almost every way imaginable. It's not like we're talking about a new theoretical steelmaking process which could upset the economy, we're talking about the potential to build a new life form with godlike abilities. Alarm seems warranted to me. Sure, it might not happen soon... but also, it might happen soon. Shouldn't we be trying to prevent it, or control it, before it happens? > People assume advancements will continue to happen at their current pace. I think people assume advancements will accelerate, actually.

a257 3y ago

When it comes to the question of when strong AI will come, I think our society's reaction to ChatGPT is more worrying than the technology itself. Namely, because of ChatGPT, we see major companies, governments, etc clamoring over AI today like never before. The magnitude of this hype and its resulting funding rush is unprecedented. Nobody could have predicted that this would have happened so quickly.