5 ms·

Well, LLMs are not AGI. They have serious limitations [1] and honestly my fear is that it’s too soon. I agree Google is ruled by managers (that’s why I hated it

Well, LLMs are not AGI. They have serious limitations [1] and honestly my fear is that it’s too soon. I agree Google is ruled by managers (that’s why I hated it), and my fear is that the managers have FOMO and want to push to productize asap even tho the tech isn’t ready yet. Look what happened when Google tried to throw an LLM in to search. Absolute shitshow. That’s not ready to become any kind of product! If they kill R&D now to focus on productizing something that is half baked, they will fail to develop those new inventions which might get us to AGI. When I worked at Google X Robotics I was hired on to the remnants of the last research team, which was dissolved six months after I started (I was moved to hardware test engineer). Our subteam really wanted to research multi-finger grippers but we got overruled, so the robot had to do everything with a two finger pinch gripper. Which is fine for research but absolutely unsuitable for real world tasks. It couldn’t even operate a spray bottle without special attachments and they thought it was going to clean people’s homes! [1] I am sharing this one a lot lately but I’m very moved by Yann LeCun’s arguments about the limits of autoregressive approaches here. As a robotics engineer I have been dismayed at all the attention LLMs are getting despite serious limitations that make them generally unsuitable to solve some of the most important problems in robotics. https://youtu.be/1lHFUR-yD6I https://youtu.be/1lHFUR-yD6I

ignoramous 2y ago
btw, Sustkever is on record that Transformers are enough to achieve AGI: https://www.youtube.com/watch?v=kW0SLbtGMcg&t=33s https://www.youtube.com/watch?v=kW0SLbtGMcg&t=33s
- TaylorAlexander 2y ago
  Okay. Well he’s not omniscient. He gives a pretty weak argument by analogy, while LeCun makes multiple detailed arguments which I find compelling.
  - ignoramous 2y ago
    > weak analogy tbf, Ilya was on a VC podcast not at a tech conf. > Well he’s not omniscient. Neither is Yann (who has since proposed a different architecture / vision which is yet to take off), but my comment was meant to highlight a recent claim from another accomplished researcher in the field.
    - TaylorAlexander 2y ago
      Right but I’m not talking about claims, I’m talking about arguments. As a robotics engineer running a farming robot project, I am all too aware of the serious differences in computational challenges and dataset availability between text and image data on the web and the kinds of problems that once faces in robotics. LeCun doesn’t just claim LLMs aren’t up to the task, he provides a detailed list of provable shortcomings which I feel overall make for a compelling argument. I’m hopeful that his JEPA may shed some light on possible solutions, but it’s also a fact that one can find issues with proposed engineering solutions even if they don’t have their own better solution. It you say you have a faster than light space engine designed, one wouldn’t need to have their own functional design to show why yours didn’t work (although such expertise is always helpful). And, well, he did invent convolutional neural networks, tho as I say such expertise is not required to raise valid arguments against some proposed solution.
      - ignoramous 2y ago
        fwiw, I agree with Yann & you (and many others!) on the shortcomings of LLMs (not from a position of authority, but as a matter of opinion). I meant to counter-balance OP's point in that there are other equally accomplished individuals who aren't swayed by Yann's (and others accelerationists like Andrew Ng) arguments or claims.
- dr_dshiv 2y ago
  And Peter Norvig argues that AGI has already been achieved by transformers https://www.noemamag.com/artificial-general-intelligence-is-already-here https://www.noemamag.com/artificial-general-intelligence-is-...
robertlagrant 2y ago
> my fear is that the managers have FOMO and want to push to productize asap even tho the tech isn’t ready yet This is exactly it. With the limitations ChatGPT is encountering around safety and hallucination, Google probably should've just said "we're working on something awesome - hold on" and kept plugging away before releasing, instead of ex-Product CEO making them release something now, even if half of the demo video is fake.
- KoolKat23 2y ago
  But stonks don't go up then. These days stock price bumps are more lucrative to shareholders than actual company returns.
  - robertlagrant 2y ago
    But have stonks gone up?
    - KoolKat23 2y ago
      I'm sure it's bursting right now but: "Companies that mentioned AI in earnings saw their stocks rise 4.6% on average, a study from Wall Street Zen found." https://markets.businessinsider.com/news/stocks/ai-stock-market-bubble-chart-earnings-investors-hype-apple-nvidia-2023-9 https://markets.businessinsider.com/news/stocks/ai-stock-mar...
golergka 2y ago
I'm using chatGPT to Google something for around a year (or whenever bing browsing became available), and I'm yet to set a single hallucination based on web search results. May be Google is just not very good at this.
- freilanzer 2y ago
  > I'm using chatGPT to Google something for around a year Maybe I'm too tired, but I don't understand this sentence.
  - jazzyjackson 2y ago
    he's been bingin' since last summer, daddy-o
  - xdavidliu 2y ago
    They have been using ChatGPT to perform all searches for a year, where in the past they would have used Google.

ignoramous 2y ago

btw, Sustkever is on record that Transformers are enough to achieve AGI: https://www.youtube.com/watch?v=kW0SLbtGMcg&t=33s https://www.youtube.com/watch?v=kW0SLbtGMcg&t=33s

TaylorAlexander 2y ago
Okay. Well he’s not omniscient. He gives a pretty weak argument by analogy, while LeCun makes multiple detailed arguments which I find compelling.
- ignoramous 2y ago
  > weak analogy tbf, Ilya was on a VC podcast not at a tech conf. > Well he’s not omniscient. Neither is Yann (who has since proposed a different architecture / vision which is yet to take off), but my comment was meant to highlight a recent claim from another accomplished researcher in the field.
  - TaylorAlexander 2y ago
    Right but I’m not talking about claims, I’m talking about arguments. As a robotics engineer running a farming robot project, I am all too aware of the serious differences in computational challenges and dataset availability between text and image data on the web and the kinds of problems that once faces in robotics. LeCun doesn’t just claim LLMs aren’t up to the task, he provides a detailed list of provable shortcomings which I feel overall make for a compelling argument. I’m hopeful that his JEPA may shed some light on possible solutions, but it’s also a fact that one can find issues with proposed engineering solutions even if they don’t have their own better solution. It you say you have a faster than light space engine designed, one wouldn’t need to have their own functional design to show why yours didn’t work (although such expertise is always helpful). And, well, he did invent convolutional neural networks, tho as I say such expertise is not required to raise valid arguments against some proposed solution.
    - ignoramous 2y ago
      fwiw, I agree with Yann & you (and many others!) on the shortcomings of LLMs (not from a position of authority, but as a matter of opinion). I meant to counter-balance OP's point in that there are other equally accomplished individuals who aren't swayed by Yann's (and others accelerationists like Andrew Ng) arguments or claims.
dr_dshiv 2y ago
And Peter Norvig argues that AGI has already been achieved by transformers https://www.noemamag.com/artificial-general-intelligence-is-already-here https://www.noemamag.com/artificial-general-intelligence-is-...

robertlagrant 2y ago

> my fear is that the managers have FOMO and want to push to productize asap even tho the tech isn’t ready yet This is exactly it. With the limitations ChatGPT is encountering around safety and hallucination, Google probably should've just said "we're working on something awesome - hold on" and kept plugging away before releasing, instead of ex-Product CEO making them release something now, even if half of the demo video is fake.

KoolKat23 2y ago
But stonks don't go up then. These days stock price bumps are more lucrative to shareholders than actual company returns.
- robertlagrant 2y ago
  But have stonks gone up?
  - KoolKat23 2y ago
    I'm sure it's bursting right now but: "Companies that mentioned AI in earnings saw their stocks rise 4.6% on average, a study from Wall Street Zen found." https://markets.businessinsider.com/news/stocks/ai-stock-market-bubble-chart-earnings-investors-hype-apple-nvidia-2023-9 https://markets.businessinsider.com/news/stocks/ai-stock-mar...

golergka 2y ago

I'm using chatGPT to Google something for around a year (or whenever bing browsing became available), and I'm yet to set a single hallucination based on web search results. May be Google is just not very good at this.

freilanzer 2y ago
> I'm using chatGPT to Google something for around a year Maybe I'm too tired, but I don't understand this sentence.
- jazzyjackson 2y ago
  he's been bingin' since last summer, daddy-o
- xdavidliu 2y ago
  They have been using ChatGPT to perform all searches for a year, where in the past they would have used Google.