11 ms·

Sam and Ilya have recently made public statements about AGI that appear to highlight a fundamental disagreement between them. Sam claims LLMs aren't sufficient

Sam and Ilya have recently made public statements about AGI that appear to highlight a fundamental disagreement between them. Sam claims LLMs aren't sufficient for AGI (rightfully so). Ilya claims the transformer architecture, with some modification for efficiency, is actually sufficient for AGI. Obviously transformers are the core component of LLMs today, and the devil is in the details (a future model may resemble the transformers of today, while also being dynamic in terms of training data/experience), but the jury is still out. In either case, publicly disagreeing on the future direction of OpenAI may be indicative of deeper problems internally.

anvuong 3y ago
That is just disagreement in technological aspects, which any well-functioning company should always have a healthy amount of within its leadership. The press release literally said he was fired because he was lying. I haven't seen anything like that from a board of a big company for a very long time.
- kolja005 3y ago
  Additionally, OpenAI can just put resources towards both approaches in order to settle this dispute. The whole point of research is that you don't know the conclusions ahead of time.
  - capableweb 3y ago
    Seemingly, OpenAIs priorities shifted after the public ChatGPT release, and seems to be more and more geared towards selling to consumers, rather than a research lab that it seemed they initially were aiming for. I'm sure this was part of the disagreement as Sam is "capitalism incarnated" while Ilya gives of much different feelings.
    - thelittleone 3y ago
      Maybe some promise was made by Sam to MS for the funding that the board didn't approve. He may have expected the board to accept the terms he agreed to but they fired him instead.
  - ilaksh 3y ago
    That might be part of it. They announced that they were dedicating compute to researching superintelligence alignment. When they launched the new stuff on Dev Day, there was not enough compute and the service was disrupted. It may have also interfered with Ilya's team's allocation and stopped their research. If that happened (speculation) then those resources weren't really dedicated to the research team.
jprete 3y ago
The question has enormous implications for OpenAI because of the specifics of their nonprofit charter. If Altman left out facts to keep the board from deciding they were at the AGI phase of OpenAI, or even to prevent them from doing a fair evaluation, then he absolutely materially misled them and prevented them from doing their jobs.
- DebtDeflation 3y ago
  If it turns out that the ouster was over a difference of opinion re: focusing on open research vs commercial success, then I don't think their current Rube Goldberg corporate structure of a non profit with a for profit subsidiary will survive. They will split up into two separate companies. Once that happens, Microsoft will find someone to sell them a 1.1% ownership stake and then immediately commence a hostile takeover.
  - acmiyaguchi 3y ago
    Interesting, is this how they're currently structured? It sounds a lot like Mozilla with the Mozilla Foundation and Corporation.
abra0 3y ago
>rightfully so How the hell can people be so confident about this? You describe two smart people reasonably disagreeing about a complicated topic
- jumploops 3y ago
  The LLMs of today are just multidimensional mirrors that contain humanity's knowledge. They don't advance that knowledge, they just regurgitate it, remix it, and expose patterns. We train them. They are very convincing, and show that the Turing test may be flawed. Given that AGI means reaching "any intellectual task that human beings can perform", we need a system that can go beyond lexical reasoning and actually contribute (on it's own) to advance our total knowledge. Anything less isn't AGI. Ilya may be right that a super-scaled transformer model (with additional mechanics beyond today's LLMs) will achieve AGI, or he may be wrong. Therefore something more than an LLM is needed to reach AGI, what that is, we don't yet know!
  - dboreham 3y ago
    Prediction: there isn't a difference. The apparent difference is a manifestation of human brain delusion about how human brains work. The Turing test is a beautiful proof of this phenomenon: so and so thing is impossibility hard only achievable via magic capabilities of human brains...oops no actually it's easily achievable now so we better re-define our test. This cycle Will continue until the singularly. Disclosure: I've been long term skeptical about AI but that writing is up on the wall now.
    - mlyle 3y ago
      Clearly there's a difference, because the architectures we have don't know how to persist information or further train. Without persistence outside of the context window, they can't even maintain a dynamic, stable higher level goal. Whether you can bolt something small to these architectures for persistence and do some small things and get AGI is an open question, but what we have is clearly insufficient by design. I expect it's something in-between: our current approaches are a fertile ground for improving towards AGI, but it's also not a trivial further step to get there.
cedws 3y ago
>Ilya claims the transformer architecture, with some modification for efficiency, is actually sufficient for AGI. I thought this guy was supposed to know what he's talking about? There was a paper that shows LLMs cannot generalise[0]. Anybody who's used ChatGPT can see there's imperfections. [0] https://arxiv.org/abs/2309.12288 https://arxiv.org/abs/2309.12288
- lijok 3y ago
  They should fire Ilya and get you in there
- razodactyl 3y ago
  LLMs can't generalise no, but the meta-architecture around them can 100%. Think about the RLHF component that trains LLMs. It's the training itself that generalises - not the final model that becomes a static component.
- justrealist 3y ago
  [flagged]
  - cedws 3y ago
    There is lack of humility in making such an assertion about a path to AGI.
  - cscurmudgeon 3y ago
    It is funny believing we have AGI and yet resorting ad-hominem to prove/defend that! At some point in time Ilya was a nobody going against the gods of AI/ML. Just slightly over a decade ago neural networks were a joke in AI.
- tinco 3y ago
  Humans don't work this way either. You don't need the LLM to do the logic, you just need the LLM to prepare the information so it can be fed into a logic engine. Just like humans do when they shut down their system 1 brain and go into system 2 slow mode. I'm in the definitely ready for AGI camp. But it's not going to be a single model that's going to do the AGI magic trick, it's going to be an engineered system consisting of multiple communicating models hooked up using traditional engineering techniques.
MVissers 3y ago
Rightfully so? No-one knows. But I sure would trust the scientist leading the endeavor more than a business person that has interest in saying the opposite to avoid immediate regulations.
calf 3y ago
Did Ilya give a reason why transformers are theoretically sufficient? I've watched him talk in a CS seminar and he's certainly interesting to listen to.
- kolja005 3y ago
  From the interviews with him that I have seen, Sutskever thinks that language model is a sufficient pretraining task because there is a great deal of reasoning involved in next token prediction. The example he used was that suppose you fed a murder mystery novel to a language model and then prompted it with the phrase "The person who committed the model was: ". The model would unquestionably need to reason in order to come to the right conclusion, but at the same time it is just predicting the next token.

anvuong 3y ago

That is just disagreement in technological aspects, which any well-functioning company should always have a healthy amount of within its leadership. The press release literally said he was fired because he was lying. I haven't seen anything like that from a board of a big company for a very long time.

kolja005 3y ago
Additionally, OpenAI can just put resources towards both approaches in order to settle this dispute. The whole point of research is that you don't know the conclusions ahead of time.
- capableweb 3y ago
  Seemingly, OpenAIs priorities shifted after the public ChatGPT release, and seems to be more and more geared towards selling to consumers, rather than a research lab that it seemed they initially were aiming for. I'm sure this was part of the disagreement as Sam is "capitalism incarnated" while Ilya gives of much different feelings.
  - thelittleone 3y ago
    Maybe some promise was made by Sam to MS for the funding that the board didn't approve. He may have expected the board to accept the terms he agreed to but they fired him instead.
- ilaksh 3y ago
  That might be part of it. They announced that they were dedicating compute to researching superintelligence alignment. When they launched the new stuff on Dev Day, there was not enough compute and the service was disrupted. It may have also interfered with Ilya's team's allocation and stopped their research. If that happened (speculation) then those resources weren't really dedicated to the research team.

jprete 3y ago

The question has enormous implications for OpenAI because of the specifics of their nonprofit charter. If Altman left out facts to keep the board from deciding they were at the AGI phase of OpenAI, or even to prevent them from doing a fair evaluation, then he absolutely materially misled them and prevented them from doing their jobs.

DebtDeflation 3y ago
If it turns out that the ouster was over a difference of opinion re: focusing on open research vs commercial success, then I don't think their current Rube Goldberg corporate structure of a non profit with a for profit subsidiary will survive. They will split up into two separate companies. Once that happens, Microsoft will find someone to sell them a 1.1% ownership stake and then immediately commence a hostile takeover.
- acmiyaguchi 3y ago
  Interesting, is this how they're currently structured? It sounds a lot like Mozilla with the Mozilla Foundation and Corporation.

abra0 3y ago

>rightfully so How the hell can people be so confident about this? You describe two smart people reasonably disagreeing about a complicated topic

jumploops 3y ago
The LLMs of today are just multidimensional mirrors that contain humanity's knowledge. They don't advance that knowledge, they just regurgitate it, remix it, and expose patterns. We train them. They are very convincing, and show that the Turing test may be flawed. Given that AGI means reaching "any intellectual task that human beings can perform", we need a system that can go beyond lexical reasoning and actually contribute (on it's own) to advance our total knowledge. Anything less isn't AGI. Ilya may be right that a super-scaled transformer model (with additional mechanics beyond today's LLMs) will achieve AGI, or he may be wrong. Therefore something more than an LLM is needed to reach AGI, what that is, we don't yet know!
- dboreham 3y ago
  Prediction: there isn't a difference. The apparent difference is a manifestation of human brain delusion about how human brains work. The Turing test is a beautiful proof of this phenomenon: so and so thing is impossibility hard only achievable via magic capabilities of human brains...oops no actually it's easily achievable now so we better re-define our test. This cycle Will continue until the singularly. Disclosure: I've been long term skeptical about AI but that writing is up on the wall now.
  - mlyle 3y ago
    Clearly there's a difference, because the architectures we have don't know how to persist information or further train. Without persistence outside of the context window, they can't even maintain a dynamic, stable higher level goal. Whether you can bolt something small to these architectures for persistence and do some small things and get AGI is an open question, but what we have is clearly insufficient by design. I expect it's something in-between: our current approaches are a fertile ground for improving towards AGI, but it's also not a trivial further step to get there.

cedws 3y ago

>Ilya claims the transformer architecture, with some modification for efficiency, is actually sufficient for AGI. I thought this guy was supposed to know what he's talking about? There was a paper that shows LLMs cannot generalise[0]. Anybody who's used ChatGPT can see there's imperfections. [0] https://arxiv.org/abs/2309.12288 https://arxiv.org/abs/2309.12288

lijok 3y ago
They should fire Ilya and get you in there
razodactyl 3y ago
LLMs can't generalise no, but the meta-architecture around them can 100%. Think about the RLHF component that trains LLMs. It's the training itself that generalises - not the final model that becomes a static component.
justrealist 3y ago
[flagged]
- cedws 3y ago
  There is lack of humility in making such an assertion about a path to AGI.
- cscurmudgeon 3y ago
  It is funny believing we have AGI and yet resorting ad-hominem to prove/defend that! At some point in time Ilya was a nobody going against the gods of AI/ML. Just slightly over a decade ago neural networks were a joke in AI.
tinco 3y ago
Humans don't work this way either. You don't need the LLM to do the logic, you just need the LLM to prepare the information so it can be fed into a logic engine. Just like humans do when they shut down their system 1 brain and go into system 2 slow mode. I'm in the definitely ready for AGI camp. But it's not going to be a single model that's going to do the AGI magic trick, it's going to be an engineered system consisting of multiple communicating models hooked up using traditional engineering techniques.

MVissers 3y ago

Rightfully so? No-one knows. But I sure would trust the scientist leading the endeavor more than a business person that has interest in saying the opposite to avoid immediate regulations.

calf 3y ago

Did Ilya give a reason why transformers are theoretically sufficient? I've watched him talk in a CS seminar and he's certainly interesting to listen to.

kolja005 3y ago
From the interviews with him that I have seen, Sutskever thinks that language model is a sufficient pretraining task because there is a great deal of reasoning involved in next token prediction. The example he used was that suppose you fed a murder mystery novel to a language model and then prompted it with the phrase "The person who committed the model was: ". The model would unquestionably need to reason in order to come to the right conclusion, but at the same time it is just predicting the next token.