5 ms·

Why do we give this people credit by giving for granted that AGI is somehow magically around the corner? It is not, and everyone’s delusional.

m3kw9 2y ago
AI alignment is a job so there is negative incentive to make it more than it is. Right now it’s painfully clear it is what’s happening at OpenAI
nodesocket 2y ago
I mean you have to think the people who are building it know a lot more than you do. If they are concerned so much so that they quitting and forfeiting millions of dollars there should be reason to be concerned.
- etrautmann 2y ago
  Not necessarily. There are plenty of reasonable perspectives on the current attitudes towards AI safety. Assuming that current employees have a secret they’re not sharing about what’s around the corner is just one of many reasonable possibilities.
  - whimsicalism 2y ago
    If there are plenty of reasonable possibilities, then it seems like certainty is misplaced
  - m3kw9 2y ago
    Most% of the imagination of killer AI comes from movies, bad incentives and wanting to get attention.
- m3kw9 2y ago
  So you agree with Sam’s stance since he likely knows way more than anyone?
whimsicalism 2y ago
You definitely don't have a basis to make such a claim. It very well might not be - but it seems silly to be certain that it is not given the increase in capabilities. People who bet on scaling in 2018 (which is really when these bets were starting to be made) were the minority then and have consistently proven the naysayers incorrect for 6 years now. Only now are we even close to reaching opinion parity on this.
MVissers 2y ago
If you look at compute scaling and model improvement on that compute, we’re going to get there pretty fast. Both compute, architecture and cost matter, and those have been improving like crazy. I don’t know about you, but I didn’t expect GPT-4o to come out at half the price one year later, with real-time voice, image and text. There is zero sign of slowing, Nvidia keeps building beefier GPUs specialized in LLMs, the world keeps adding more data centers and humanity keeps creating exponentially more data world wide. You can plot this exponential growth out over time and calculate when these models will have the complexity of the brain. Then you can assume some penalty for shitty architecture (that gets better over time), and you’ll have a ballpark estimate. Somewhere 2027-2030 we’ll have models that most of us consider AGI today. Now, societal impact is hard to predict since that’ll depend on regulators as well.
- Aloisius 2y ago
  LLMs don't reason. Without reasoning, you don't have AGI. I can't wait until we're past the peak on the hype cycle because people's understanding of what these models can and can't do or how close we are to AGI isn't grounded in reality.
  - drdeca 2y ago
    It seems unclear to me what it is that LLMs can’t/don’t do. I agree that there are substantial limitations of current ones, but I have little idea of which of these limitations are fundamental to the architecture (as well as which are a result of limits of the scale, and which are a result of limits to the training data (using the current training strategies), and which are a result of the overall training methods). Do you have some insights you could share on this question? I mean, “they can’t reason” might be true, but I’m not exactly sure what it is that it is saying that they can’t do.
    - elicksaur 2y ago
      Think about the difference between what an LLM is doing and a human is doing in response to being told they are wrong. If you tell a human, “You are wrong on this, and here is why.” They may reject it, but their mental model is updating with the new information that you, the speaker, think X for YZQ reasons. Their response is based on a judgment of how trustworthy you are and the credibility of the evidence. For an LLM, the response is not based on these logical connections, but simply the additional prompt context of YZQ tokens being close to each other. This is not “logic” in any traditional sense or in the sense of how a human incorporates and responds to this new information. The LLM’s method of responding is also inherent to the architecture of the model. It’s predicting tokens based on input. It’s not reasoning. Critically, this flaw is inherent in all LLM output. Giving an LLM’s output the power to affect real world activities means trusting that the decision can be made by sophisticated word association rather than more complex reasoning. There may be lots of decisions where word association is all you need, but I doubt that is the case for all decisions humans make.
bambax 2y ago
Yes! It's debatable current models demonstrate any kind of "intelligence", but AGI is so far off it's ridiculous to even mention it. But not everyone's delusional. Here's what Yann Le Cun is currently saying: > It seems to me that before "urgently figuring out how to control AI systems much smarter than us" we need to have the beginning of a hint of a design for a system smarter than a house cat. and > don't confuse the superhuman knowledge accumulation and retrieval abilities of current LLMs with actual intelligence https://x.com/ylecun/status/1791890883425570823 https://x.com/ylecun/status/1791890883425570823
- llamaimperative 2y ago
  Okay, so describe to me when and how we’ll detect and mitigate problems as we approach AGI or superintelligence. The promise of upside grows exactly in lockstep with the danger of downside. It is absolutely rational to say: “if our AI safety research lab is being hijacked by commercial interests before it’s even commercially viable as a business, then how will we possibly pull the plug on the final descent?” P.S. neither Yann nor me nor you nor Sam nor Greg know what “real intelligence” is, and we certainly don’t know what superhuman intelligence is just the same way an ant couldn’t possibly know what human intelligence is. It is inconceivable to us — that is the point of intelligence.
  - nathanasmith 2y ago
    We're about as close to AGI as we are to developing superluminal space flight and working time machines.
    - llamaimperative 2y ago
      Umm, no. We can point to concrete mathematical and empirical evidence of those two things being impossible. Point to the physical laws that indicate AGI or superintelligence is impossible or even maybe-impossible. No such laws exist.
      - nathanasmith 2y ago
deleted 2y ago
[deleted]
stefan_ 2y ago
This discussion is tremendously annoying because like the whole "room temperature superconductor" thing, half of it is people who have read far too many science fiction books
- 0xDEAFBEAD 2y ago
  A lot of serious scholars are concerned, including Hinton and Bengio https://www.safe.ai/work/statement-on-ai-risk#open-letter https://www.safe.ai/work/statement-on-ai-risk#open-letter
ksynwa 2y ago
Baffles me that this company has (or had) the smartest people in the industry who understand the workings of their product better than anyone and then you go read their charter where they sagely talk about ethically ushering in AGI like it's nothing. The whole drama with this infighting about safety feels like a terrible good cop bad cop routine.
tgv 2y ago
AGI isn't close, but as long as people keep throwing billions at it, the research will produce something. We now have publicly accessible models to generate revenge porn. I guess that wasn't the intention, but here we are.
elktown 2y ago
The only reason "AGI" is around the corner is because people with vested interests - emotional, profit etc - are going to redefine it to suit their purposes. We've already seen this with AI which is now just any bog standard algorithm/app with an attached marketing sticker.

m3kw9 2y ago

AI alignment is a job so there is negative incentive to make it more than it is. Right now it’s painfully clear it is what’s happening at OpenAI

nodesocket 2y ago

I mean you have to think the people who are building it know a lot more than you do. If they are concerned so much so that they quitting and forfeiting millions of dollars there should be reason to be concerned.

etrautmann 2y ago
Not necessarily. There are plenty of reasonable perspectives on the current attitudes towards AI safety. Assuming that current employees have a secret they’re not sharing about what’s around the corner is just one of many reasonable possibilities.
- whimsicalism 2y ago
  If there are plenty of reasonable possibilities, then it seems like certainty is misplaced
- m3kw9 2y ago
  Most% of the imagination of killer AI comes from movies, bad incentives and wanting to get attention.
m3kw9 2y ago
So you agree with Sam’s stance since he likely knows way more than anyone?

whimsicalism 2y ago

You definitely don't have a basis to make such a claim. It very well might not be - but it seems silly to be certain that it is not given the increase in capabilities. People who bet on scaling in 2018 (which is really when these bets were starting to be made) were the minority then and have consistently proven the naysayers incorrect for 6 years now. Only now are we even close to reaching opinion parity on this.

MVissers 2y ago

If you look at compute scaling and model improvement on that compute, we’re going to get there pretty fast. Both compute, architecture and cost matter, and those have been improving like crazy. I don’t know about you, but I didn’t expect GPT-4o to come out at half the price one year later, with real-time voice, image and text. There is zero sign of slowing, Nvidia keeps building beefier GPUs specialized in LLMs, the world keeps adding more data centers and humanity keeps creating exponentially more data world wide. You can plot this exponential growth out over time and calculate when these models will have the complexity of the brain. Then you can assume some penalty for shitty architecture (that gets better over time), and you’ll have a ballpark estimate. Somewhere 2027-2030 we’ll have models that most of us consider AGI today. Now, societal impact is hard to predict since that’ll depend on regulators as well.

Aloisius 2y ago
LLMs don't reason. Without reasoning, you don't have AGI. I can't wait until we're past the peak on the hype cycle because people's understanding of what these models can and can't do or how close we are to AGI isn't grounded in reality.
- drdeca 2y ago
  It seems unclear to me what it is that LLMs can’t/don’t do. I agree that there are substantial limitations of current ones, but I have little idea of which of these limitations are fundamental to the architecture (as well as which are a result of limits of the scale, and which are a result of limits to the training data (using the current training strategies), and which are a result of the overall training methods). Do you have some insights you could share on this question? I mean, “they can’t reason” might be true, but I’m not exactly sure what it is that it is saying that they can’t do.
  - elicksaur 2y ago
    Think about the difference between what an LLM is doing and a human is doing in response to being told they are wrong. If you tell a human, “You are wrong on this, and here is why.” They may reject it, but their mental model is updating with the new information that you, the speaker, think X for YZQ reasons. Their response is based on a judgment of how trustworthy you are and the credibility of the evidence. For an LLM, the response is not based on these logical connections, but simply the additional prompt context of YZQ tokens being close to each other. This is not “logic” in any traditional sense or in the sense of how a human incorporates and responds to this new information. The LLM’s method of responding is also inherent to the architecture of the model. It’s predicting tokens based on input. It’s not reasoning. Critically, this flaw is inherent in all LLM output. Giving an LLM’s output the power to affect real world activities means trusting that the decision can be made by sophisticated word association rather than more complex reasoning. There may be lots of decisions where word association is all you need, but I doubt that is the case for all decisions humans make.

bambax 2y ago

Yes! It's debatable current models demonstrate any kind of "intelligence", but AGI is so far off it's ridiculous to even mention it. But not everyone's delusional. Here's what Yann Le Cun is currently saying: > It seems to me that before "urgently figuring out how to control AI systems much smarter than us" we need to have the beginning of a hint of a design for a system smarter than a house cat. and > don't confuse the superhuman knowledge accumulation and retrieval abilities of current LLMs with actual intelligence https://x.com/ylecun/status/1791890883425570823 https://x.com/ylecun/status/1791890883425570823

llamaimperative 2y ago
Okay, so describe to me when and how we’ll detect and mitigate problems as we approach AGI or superintelligence. The promise of upside grows exactly in lockstep with the danger of downside. It is absolutely rational to say: “if our AI safety research lab is being hijacked by commercial interests before it’s even commercially viable as a business, then how will we possibly pull the plug on the final descent?” P.S. neither Yann nor me nor you nor Sam nor Greg know what “real intelligence” is, and we certainly don’t know what superhuman intelligence is just the same way an ant couldn’t possibly know what human intelligence is. It is inconceivable to us — that is the point of intelligence.
- nathanasmith 2y ago
  We're about as close to AGI as we are to developing superluminal space flight and working time machines.
  - llamaimperative 2y ago
    Umm, no. We can point to concrete mathematical and empirical evidence of those two things being impossible. Point to the physical laws that indicate AGI or superintelligence is impossible or even maybe-impossible. No such laws exist.
    - nathanasmith 2y ago

deleted 2y ago

[deleted]

stefan_ 2y ago

This discussion is tremendously annoying because like the whole "room temperature superconductor" thing, half of it is people who have read far too many science fiction books

0xDEAFBEAD 2y ago
A lot of serious scholars are concerned, including Hinton and Bengio https://www.safe.ai/work/statement-on-ai-risk#open-letter https://www.safe.ai/work/statement-on-ai-risk#open-letter

ksynwa 2y ago

Baffles me that this company has (or had) the smartest people in the industry who understand the workings of their product better than anyone and then you go read their charter where they sagely talk about ethically ushering in AGI like it's nothing. The whole drama with this infighting about safety feels like a terrible good cop bad cop routine.

tgv 2y ago

AGI isn't close, but as long as people keep throwing billions at it, the research will produce something. We now have publicly accessible models to generate revenge porn. I guess that wasn't the intention, but here we are.

elktown 2y ago

The only reason "AGI" is around the corner is because people with vested interests - emotional, profit etc - are going to redefine it to suit their purposes. We've already seen this with AI which is now just any bog standard algorithm/app with an attached marketing sticker.