20 ms·

People have been saying this at least for the last 10 years. I think if advancements stop being made at a rapid clip, the bubble might implode. But every year i

People have been saying this at least for the last 10 years. I think if advancements stop being made at a rapid clip, the bubble might implode. But every year it seems like its a whole new ballgame. When LLMs start playing the stock market autonomously, then we should start worrying.

marcosdumay 1y ago
Oh, man, there have been 10 years since ChatGPT made the news in 2022? How time flies!
- seanmcdirmid 1y ago
  People have been predicting an AI winter ever since DNNs hit it big, and that was 2012/2013 or so. And then every year since 2015 they've been saying "this is the year the AI bubble will finally burst!" And then repeat that the next year. ChatGPT was just another big leap in a bunch of big leaps that occurred in the last 15 years. You are being disingenuous in pretending that people didn't see AI as being hyped before ChatGPT came out.
  - marcosdumay 1y ago
    The finance bubble is pretty much a post-ChatGTP phenomenon. There was some hype about AI before it, for completely unrelated products, with varying levels of real value behind them. What bubble there was about those has already popped long ago, missing the headlines because of the crypto hype.
    - seanmcdirmid 1y ago
      If you've been with HN for awhile, or you search the archives, similar topics to "Is the AI booming turning into an AI bubble" kept popping up long before CHatGPT came out. And this was DNNs, and it was same people (e.g. Hinton and his students) who were at the forefront of that also. And no, their previous projects didn't go bust.
      - marcosdumay 1y ago
        Well, ok, good point. There has been people claiming that "their work will stop scaling any time now" for some 12 years. (To be fair, it can't scale forever and seems to have stopped scaling by now.) That is a hype bubble. I maintain that it's not the bubble the article and everybody is talking about. People are talking about the billions (but the plural doesn't do it justice) running around looking for a return. A lot of their earlier projects weren't even supposed to make a profit.
        seanmcdirmid 1y ago
        > There has been people claiming that "their work will stop scaling any time now" for some 12 years. (To be fair, it can't scale forever and seems to have stopped scaling by now.) Their work kept scaling, they just branched out into generative AI to keep going. Now, eventually they won't be able to scale up or branch out anymore, and I guess your bet is that this will happen before AGI is reached?
  - jraph 1y ago
    AI was over-hyped. I remember how "AI" felt like marketing bullshit at the lab when I was there between 2015 and 2020. Generative AI took it at the next level though. There is no comparison. The meaning of AI has brutally changed. It used to refer to ML, deep learning and all these things. Now, it is a synonym of generative AI.
    - seanmcdirmid 1y ago
      The original paper where they took wikipedia as a corpus and generated a character by character generator (the precursor to the token-based transformer stuff) came out in 2012 (I think?). It isn't unrelated to the AI that was hyped up then, it is and was a direct evolution of the same ML we were talking about today (Hinton was an author on that paper).
      - jraph 1y ago
        Yeah yeah, there was also SCIgen in 2005 (when 3 MIT students submitted a fake paper and got it accepted). I believe the difference is scale. It's not just Wikipedia, it's the whole web. Billions and billions are burned (and it seems highly unsustainable, where previously, stuff was made to work in public labs - possibly with things like Grid5000, but that's nothing compared to what's training and running LLMs today). One fifth to one third of HN's front page is AI. The thing is mainstream and omnipresent (before 2022, that stuff was mostly people working in research and tech companies, the informed general public might have heard of ML). Around 2015, we were there: https://xkcd.com/1425/ https://xkcd.com/1425/ (explain xkcd says September 24, 2014)

marcosdumay 1y ago

Oh, man, there have been 10 years since ChatGPT made the news in 2022? How time flies!

seanmcdirmid 1y ago
People have been predicting an AI winter ever since DNNs hit it big, and that was 2012/2013 or so. And then every year since 2015 they've been saying "this is the year the AI bubble will finally burst!" And then repeat that the next year. ChatGPT was just another big leap in a bunch of big leaps that occurred in the last 15 years. You are being disingenuous in pretending that people didn't see AI as being hyped before ChatGPT came out.
- marcosdumay 1y ago
  The finance bubble is pretty much a post-ChatGTP phenomenon. There was some hype about AI before it, for completely unrelated products, with varying levels of real value behind them. What bubble there was about those has already popped long ago, missing the headlines because of the crypto hype.
  - seanmcdirmid 1y ago
    If you've been with HN for awhile, or you search the archives, similar topics to "Is the AI booming turning into an AI bubble" kept popping up long before CHatGPT came out. And this was DNNs, and it was same people (e.g. Hinton and his students) who were at the forefront of that also. And no, their previous projects didn't go bust.
    - marcosdumay 1y ago
      Well, ok, good point. There has been people claiming that "their work will stop scaling any time now" for some 12 years. (To be fair, it can't scale forever and seems to have stopped scaling by now.) That is a hype bubble. I maintain that it's not the bubble the article and everybody is talking about. People are talking about the billions (but the plural doesn't do it justice) running around looking for a return. A lot of their earlier projects weren't even supposed to make a profit.
      - seanmcdirmid 1y ago
        > There has been people claiming that "their work will stop scaling any time now" for some 12 years. (To be fair, it can't scale forever and seems to have stopped scaling by now.) Their work kept scaling, they just branched out into generative AI to keep going. Now, eventually they won't be able to scale up or branch out anymore, and I guess your bet is that this will happen before AGI is reached?
- jraph 1y ago
  AI was over-hyped. I remember how "AI" felt like marketing bullshit at the lab when I was there between 2015 and 2020. Generative AI took it at the next level though. There is no comparison. The meaning of AI has brutally changed. It used to refer to ML, deep learning and all these things. Now, it is a synonym of generative AI.
  - seanmcdirmid 1y ago
    The original paper where they took wikipedia as a corpus and generated a character by character generator (the precursor to the token-based transformer stuff) came out in 2012 (I think?). It isn't unrelated to the AI that was hyped up then, it is and was a direct evolution of the same ML we were talking about today (Hinton was an author on that paper).
    - jraph 1y ago
      Yeah yeah, there was also SCIgen in 2005 (when 3 MIT students submitted a fake paper and got it accepted). I believe the difference is scale. It's not just Wikipedia, it's the whole web. Billions and billions are burned (and it seems highly unsustainable, where previously, stuff was made to work in public labs - possibly with things like Grid5000, but that's nothing compared to what's training and running LLMs today). One fifth to one third of HN's front page is AI. The thing is mainstream and omnipresent (before 2022, that stuff was mostly people working in research and tech companies, the informed general public might have heard of ML). Around 2015, we were there: https://xkcd.com/1425/ https://xkcd.com/1425/ (explain xkcd says September 24, 2014)