6 ms·

It's doubtful if there even is a race anymore. The last significant AI advancement in the consumer LLM space was fluent human language synthesis around 2020, wi

by caseyy 1y ago

It's doubtful if there even is a race anymore. The last significant AI advancement in the consumer LLM space was fluent human language synthesis around 2020, with its following assistant/chat interface. Since then, everything has been incremental — larger models, new ways to prompt them, cheaper ways to run them, more human feedback, and gaming evaluations. The wisest move in the chatbot business might be to wait and see if anyone discovers anything profitable before spending more effort and wasting more money on chat R&D, which includes most agentic stuff. Reliable assistants or something along those lines might be the next big breakthrough (if you ask certain futurologists), but the technology we have seems unsuitable for any provable reliability. ML can be applied in a thousand ways other than LLMs, and many will positively impact our lives and create their own markets. But OpenAI is not in that business. I think the writing is on the wall, and Sama's vocal fry, "AGI is close," and humanity verification crypto coins are smoke and mirrors.

roflmaostc 1y ago
Just to get things right. The big AI LLM hype started end of 2022 with the launch of ChatGPT, DALL-E 2, .... Most people in society connect AI directly to ChatGPT and hence OpenAI. And there has been a lot of progress in image generation, video generation, ... So I think your timeline and views are slightly off.
- caseyy 1y ago
  > Just to get things right. The big AI LLM hype started end of 2022 with the launch of ChatGPT, DALL-E 2, .... GPT-2 was released in 2019, GPT-3 in 2020. I'd say 2020 is significant because that's when people seriously considered the Turing test passed reliably for the first time. But for the sake of this argument, it hardly matters what date years back we choose. There's been enough time since then to see the plateau. > Most people in society connect AI directly to ChatGPT and hence OpenAI. I'd double-check that assumption. Many people I've spoken to take a moment to remember that "AI" stands for artificial intelligence. Outside of tongue-in-cheek jokes, OpenAI has about 50% market share in LLMs, but you can't forget that Samsung makes AI washing machines, let alone all the purely fraudulent uses of the "AI" label. > And there has been a lot of progress in image generation, video generation, ... These are entirely different architectures from LLM/chat though. But you're right that OpenAI does that, too. When I said that they don't stray much from chat, I was thinking more about AlexNet and the broad applications of ML in general. But you're right, OpenAI also did/does diffusion, GANs, transformer vision. This doesn't change my views much on chat being "not seeing the forest for the trees" though. In the big picture, I think there aren't many hockey sticks/exponentials left in LLMs to discover. That is not true about other AI/ML.
  - tomnipotent 1y ago
    ChatGPT was not released to the general public until November 2022, and the mobile apps were not released until May 2023. For most of the world LLM's did not exist before those dates.
    - asadotzler 1y ago
      LLM AI hype started well before ChatGPT. This site and many others were littered with OpenAI stories calling it the next Bell Labs or Xerox PARC and other such nonsense going back to 2016. And GPT stories kicked into high gear all over the web and TV in 2019 in the lead-up to GPT-2 when OpenAI was telling the world it was too dangerous to release. Certainly by 2021 and early 2022, LLM AI was being reported on all over the place. >For most of the world LLM's did not exist before those dates. Just because people don't use something doesn't mean they don't know about it. Plenty of people were hearing about the existential threat of (LLM) AI long before ChatGPT. Fox News and CNN had stories on GPT-2 years before ChatGPT was even a thing. Exposure doesn't get much more mainstream than that.
      - tomnipotent 1y ago
        > LLM AI was being reported on all over the place. No, it wasn't. As a proxy, here's HN results prior to November, 2022 - 13 results. https://hn.algolia.com/?dateEnd=1667260800&dateRange=custom&dateStart=1577836800&page=0&prefix=false&query=LLM&sort=byPopularity&type=story https://hn.algolia.com/?dateEnd=1667260800&dateRange=custom&... Here's Google Trends, showing a clear uptick May 2023, and basically no search volume before (the small increase Feb. 2023 probably Meta's Llama). https://trends.google.com/trends/explore?date=today%205-y&geo=US&q=llm&hl=en-US https://trends.google.com/trends/explore?date=today%205-y&ge... https://trends.google.com/trends/explore?date=today%205-y&geo=US&q=gpt&hl=en-US https://trends.google.com/trends/explore?date=today%205-y&ge... As another proxy, compare Nvidia revenues - $26.91bln in 2022, $26.97bln in 2023, $60bln 2024, $130bln 2025. I think it's clear the hype didn't start until 2023. You're welcome to point out articles and stores before this time period "hyping" LLM's, but what I remember is that before ChatGPT there was very little conversation around LLM's.
paulddraper 1y ago
You saying —- with a straight face —- that post 2020 LLM AIs have made only incremental progress?
- caseyy 1y ago
  Yep, compared to beating the Turing test, the progress has been linear with exponentially growing investment. That's diminishing marginal returns.
  - deleted 1y ago
    [deleted]
- ReptileMan 1y ago
  Yes. But they have also improved a lot. Incremental just means that the function is going up without breaking points. We haven't seen anything revolutionary, just evolutionary in the last 3 years. But the models do provide 2 or 3 times more value. So their pace of advancement is not slow.
  - bonoboTP 1y ago
    The better you know a field the more it looks incremental. In other words, incrementalness is more a function of how much attention you pay or how deep you research it. Relativity and quantum mechanics were also incremental. Copernicus and Kepler were incremental. Deep learning itself was incremental. Based on almost identical networks from the 90s (CNN), which were using methods from the 80s (backprop) on architectures from the 70s (neocognitron) using activation functions from the 60s and the basic neuron model from the 40s (McCullough and Pitts), which was just a mathematization of observations in biology via microscopy integration with mathematical logic and electrical logic gates developed around the same time (Shannon), so it's just logic as formalized by Gödel and others and it goes back to Hilbert's program, which can be extrapolated from Leibniz etc. etc. It's not hard to say that "it's really just previous thing X plus previous thing Y, nothing new under the sun" to literally anything. "It just suddenly appeared out of nowhere" is just a perception based on missing info. Many average people think ChatGPT was a sudden innovation specifically by OpenAI seemingly out of nowhere. Because they didn't follow it.
orionsbelt 1y ago
Saying LLMs have only incrementally improved is like saying my 13 year old has only incrementally approved over the last 5 years. Sure, it's been a set of continuous improvements, but that has taken it from a toy to genuinely insanely useful. Personally, deep research and o3 have been transformative, taking LLMs from something I have never used to something that I am using daily. Even if the progress ends up plateauing (which I do not believe will happen in the near term), behaviors are changing; OpenAI is capturing users, and taking them from companies like Google. Google may be able to fight back and win - Gemini 2.5 Pro is great - but any company sitting this out risks being unable to capture users back from Open AI at a later date.
- bigstrat2003 1y ago
  No, it's still just a toy. Until they can make the models actually consistently good at things, they aren't going to be useful. Right now they still BS you far too much to trust them, and because you have to double check their work every time they are worse than no tool at all.
- csours 1y ago
  To extend your illustration, 5 years ago no one could train an LLM with the capabilities of a 13 year old human; now many companies can both train LLMs and integrate them into products. > taken it from a toy to genuinely insanely useful. Really?
- devjab 1y ago
  > any company sitting this out risks being unable to capture users back from Open AI at a later date. Why? I paid for Claude for a while, but with Deepseek, Gemini and the free hits on Mistral, ChatGPT, Claude and Perplexity I'm not sure why I would now. This is anecdotal of course, but I'm very rarely unique in my behaviour. I think the best the subscription companies can hope for is that their subscribers don't realize that Deepseek and Gemini can basically do all you need for free.
  - poormathskills 1y ago
    >I'm very rarely unique in my behaviour I cannot stress this enough: if you know what Deepseek, Claude, Mistral, and Perplexity are, you are not a typical consumer. Arguably, if you have used a single one of those brands you are not a typical consumer. The vast majority of people have used ChatGPT and nothing else, except maybe clicking on Gemini or Meta AI by accident.
fastball 1y ago
Seems like an arbitrary distinction. I'd say Chain-of-Thought has massively improved LLM output. Is that "incremental"? Why is that more incremental than the move from GPT-2 to GPT-3? Sure you can say that this is when LLMs first passed some sort of Turing test, but fundamentally there was no technological difference from GPT-3 to GPT-4. In fact I would say the quality of GPT-4 unlocked thousands (millions?) more use-cases that were not very viable with the quality delivered by GPT-3. I don't see any reason for more use-cases to keep being unlocked by LLM improvements.

by caseyy 1y ago

roflmaostc 1y ago

Just to get things right. The big AI LLM hype started end of 2022 with the launch of ChatGPT, DALL-E 2, .... Most people in society connect AI directly to ChatGPT and hence OpenAI. And there has been a lot of progress in image generation, video generation, ... So I think your timeline and views are slightly off.

caseyy 1y ago
> Just to get things right. The big AI LLM hype started end of 2022 with the launch of ChatGPT, DALL-E 2, .... GPT-2 was released in 2019, GPT-3 in 2020. I'd say 2020 is significant because that's when people seriously considered the Turing test passed reliably for the first time. But for the sake of this argument, it hardly matters what date years back we choose. There's been enough time since then to see the plateau. > Most people in society connect AI directly to ChatGPT and hence OpenAI. I'd double-check that assumption. Many people I've spoken to take a moment to remember that "AI" stands for artificial intelligence. Outside of tongue-in-cheek jokes, OpenAI has about 50% market share in LLMs, but you can't forget that Samsung makes AI washing machines, let alone all the purely fraudulent uses of the "AI" label. > And there has been a lot of progress in image generation, video generation, ... These are entirely different architectures from LLM/chat though. But you're right that OpenAI does that, too. When I said that they don't stray much from chat, I was thinking more about AlexNet and the broad applications of ML in general. But you're right, OpenAI also did/does diffusion, GANs, transformer vision. This doesn't change my views much on chat being "not seeing the forest for the trees" though. In the big picture, I think there aren't many hockey sticks/exponentials left in LLMs to discover. That is not true about other AI/ML.
- tomnipotent 1y ago
  ChatGPT was not released to the general public until November 2022, and the mobile apps were not released until May 2023. For most of the world LLM's did not exist before those dates.
  - asadotzler 1y ago
    LLM AI hype started well before ChatGPT. This site and many others were littered with OpenAI stories calling it the next Bell Labs or Xerox PARC and other such nonsense going back to 2016. And GPT stories kicked into high gear all over the web and TV in 2019 in the lead-up to GPT-2 when OpenAI was telling the world it was too dangerous to release. Certainly by 2021 and early 2022, LLM AI was being reported on all over the place. >For most of the world LLM's did not exist before those dates. Just because people don't use something doesn't mean they don't know about it. Plenty of people were hearing about the existential threat of (LLM) AI long before ChatGPT. Fox News and CNN had stories on GPT-2 years before ChatGPT was even a thing. Exposure doesn't get much more mainstream than that.
    - tomnipotent 1y ago
      > LLM AI was being reported on all over the place. No, it wasn't. As a proxy, here's HN results prior to November, 2022 - 13 results. https://hn.algolia.com/?dateEnd=1667260800&dateRange=custom&dateStart=1577836800&page=0&prefix=false&query=LLM&sort=byPopularity&type=story https://hn.algolia.com/?dateEnd=1667260800&dateRange=custom&... Here's Google Trends, showing a clear uptick May 2023, and basically no search volume before (the small increase Feb. 2023 probably Meta's Llama). https://trends.google.com/trends/explore?date=today%205-y&geo=US&q=llm&hl=en-US https://trends.google.com/trends/explore?date=today%205-y&ge... https://trends.google.com/trends/explore?date=today%205-y&geo=US&q=gpt&hl=en-US https://trends.google.com/trends/explore?date=today%205-y&ge... As another proxy, compare Nvidia revenues - $26.91bln in 2022, $26.97bln in 2023, $60bln 2024, $130bln 2025. I think it's clear the hype didn't start until 2023. You're welcome to point out articles and stores before this time period "hyping" LLM's, but what I remember is that before ChatGPT there was very little conversation around LLM's.

paulddraper 1y ago

You saying —- with a straight face —- that post 2020 LLM AIs have made only incremental progress?

caseyy 1y ago
Yep, compared to beating the Turing test, the progress has been linear with exponentially growing investment. That's diminishing marginal returns.
- deleted 1y ago
  [deleted]
ReptileMan 1y ago
Yes. But they have also improved a lot. Incremental just means that the function is going up without breaking points. We haven't seen anything revolutionary, just evolutionary in the last 3 years. But the models do provide 2 or 3 times more value. So their pace of advancement is not slow.
- bonoboTP 1y ago
  The better you know a field the more it looks incremental. In other words, incrementalness is more a function of how much attention you pay or how deep you research it. Relativity and quantum mechanics were also incremental. Copernicus and Kepler were incremental. Deep learning itself was incremental. Based on almost identical networks from the 90s (CNN), which were using methods from the 80s (backprop) on architectures from the 70s (neocognitron) using activation functions from the 60s and the basic neuron model from the 40s (McCullough and Pitts), which was just a mathematization of observations in biology via microscopy integration with mathematical logic and electrical logic gates developed around the same time (Shannon), so it's just logic as formalized by Gödel and others and it goes back to Hilbert's program, which can be extrapolated from Leibniz etc. etc. It's not hard to say that "it's really just previous thing X plus previous thing Y, nothing new under the sun" to literally anything. "It just suddenly appeared out of nowhere" is just a perception based on missing info. Many average people think ChatGPT was a sudden innovation specifically by OpenAI seemingly out of nowhere. Because they didn't follow it.

orionsbelt 1y ago

Saying LLMs have only incrementally improved is like saying my 13 year old has only incrementally approved over the last 5 years. Sure, it's been a set of continuous improvements, but that has taken it from a toy to genuinely insanely useful. Personally, deep research and o3 have been transformative, taking LLMs from something I have never used to something that I am using daily. Even if the progress ends up plateauing (which I do not believe will happen in the near term), behaviors are changing; OpenAI is capturing users, and taking them from companies like Google. Google may be able to fight back and win - Gemini 2.5 Pro is great - but any company sitting this out risks being unable to capture users back from Open AI at a later date.

bigstrat2003 1y ago
No, it's still just a toy. Until they can make the models actually consistently good at things, they aren't going to be useful. Right now they still BS you far too much to trust them, and because you have to double check their work every time they are worse than no tool at all.
csours 1y ago
To extend your illustration, 5 years ago no one could train an LLM with the capabilities of a 13 year old human; now many companies can both train LLMs and integrate them into products. > taken it from a toy to genuinely insanely useful. Really?
devjab 1y ago
> any company sitting this out risks being unable to capture users back from Open AI at a later date. Why? I paid for Claude for a while, but with Deepseek, Gemini and the free hits on Mistral, ChatGPT, Claude and Perplexity I'm not sure why I would now. This is anecdotal of course, but I'm very rarely unique in my behaviour. I think the best the subscription companies can hope for is that their subscribers don't realize that Deepseek and Gemini can basically do all you need for free.
- poormathskills 1y ago
  >I'm very rarely unique in my behaviour I cannot stress this enough: if you know what Deepseek, Claude, Mistral, and Perplexity are, you are not a typical consumer. Arguably, if you have used a single one of those brands you are not a typical consumer. The vast majority of people have used ChatGPT and nothing else, except maybe clicking on Gemini or Meta AI by accident.

fastball 1y ago

Seems like an arbitrary distinction. I'd say Chain-of-Thought has massively improved LLM output. Is that "incremental"? Why is that more incremental than the move from GPT-2 to GPT-3? Sure you can say that this is when LLMs first passed some sort of Turing test, but fundamentally there was no technological difference from GPT-3 to GPT-4. In fact I would say the quality of GPT-4 unlocked thousands (millions?) more use-cases that were not very viable with the quality delivered by GPT-3. I don't see any reason for more use-cases to keep being unlocked by LLM improvements.