3 ms·

I think we're far closer to AGI than most might realize. It possible that these systems may self-improve, and companies like OpenAI may already be doing this ve

• baryphonic 2y ago
  EDIT: cite source I have yet to see any evidence of this. To the contrary, we have seen for years that advances in LLMs require orders of magnitude more parameters/power for each generation. Neural architecture search has been underwhelming. RLHF seems to have regressed models like GPT3.5+, rather than improving them. And recently, researchers concluded that multi-modal models require exponentially more data for each extra "mode" being added to the model.[0] Even in my own experimentation, I've tried to get some AGI-like behavior, and it just isn't there. I have convinced GPT4, for instance, to generate XML source for an SVG, but it looks nothing like what I describe. I'd argue that these models don't generalize well at all, and I'd bet that, like with Moore's Law, advances in AI will require continual discovery of incremental new innovations and occasionally new architectures. [0]https://arxiv.org/abs/2404.04125 https://arxiv.org/abs/2404.04125