2 ms·

Well, duh, this is basically the goal of Artificial Intelligence. With no timeframe, well, sure, that seems plausible. But we're not there yet, and we're not

Well, duh, this is basically the goal of Artificial Intelligence. With no timeframe, well, sure, that seems plausible. But we're not there yet, and we're not close either. I do think that recent experimentations and products can be useful and maybe also serve as a brick for something more powerful, but this is not simply a scaling problem at this stage. I think that many of us noticed how the hype is currently cooling down, a new AI winter is also plausible. Managing expectations should be the norm in this field, given its history...

lIl-IIIl 2y ago
There's a timeframe. It's the first 5 words of the tweet.
IAmGraydon 2y ago
I think we're far closer to AGI than most might realize. It possible that these systems may self-improve, and companies like OpenAI may already be doing this very thing. If that's the case, the time to AGI will decrease at an exponential, compounding rate.
- baryphonic 2y ago
  EDIT: cite source I have yet to see any evidence of this. To the contrary, we have seen for years that advances in LLMs require orders of magnitude more parameters/power for each generation. Neural architecture search has been underwhelming. RLHF seems to have regressed models like GPT3.5+, rather than improving them. And recently, researchers concluded that multi-modal models require exponentially more data for each extra "mode" being added to the model.[0] Even in my own experimentation, I've tried to get some AGI-like behavior, and it just isn't there. I have convinced GPT4, for instance, to generate XML source for an SVG, but it looks nothing like what I describe. I'd argue that these models don't generalize well at all, and I'd bet that, like with Moore's Law, advances in AI will require continual discovery of incremental new innovations and occasionally new architectures. [0]https://arxiv.org/abs/2404.04125 https://arxiv.org/abs/2404.04125

lIl-IIIl 2y ago

There's a timeframe. It's the first 5 words of the tweet.

IAmGraydon 2y ago

I think we're far closer to AGI than most might realize. It possible that these systems may self-improve, and companies like OpenAI may already be doing this very thing. If that's the case, the time to AGI will decrease at an exponential, compounding rate.

baryphonic 2y ago
EDIT: cite source I have yet to see any evidence of this. To the contrary, we have seen for years that advances in LLMs require orders of magnitude more parameters/power for each generation. Neural architecture search has been underwhelming. RLHF seems to have regressed models like GPT3.5+, rather than improving them. And recently, researchers concluded that multi-modal models require exponentially more data for each extra "mode" being added to the model.[0] Even in my own experimentation, I've tried to get some AGI-like behavior, and it just isn't there. I have convinced GPT4, for instance, to generate XML source for an SVG, but it looks nothing like what I describe. I'd argue that these models don't generalize well at all, and I'd bet that, like with Moore's Law, advances in AI will require continual discovery of incremental new innovations and occasionally new architectures. [0]https://arxiv.org/abs/2404.04125 https://arxiv.org/abs/2404.04125