3 ms·

Saying LLMs have only incrementally improved is like saying my 13 year old has only incrementally approved over the last 5 years. Sure, it's been a set of conti

Saying LLMs have only incrementally improved is like saying my 13 year old has only incrementally approved over the last 5 years. Sure, it's been a set of continuous improvements, but that has taken it from a toy to genuinely insanely useful. Personally, deep research and o3 have been transformative, taking LLMs from something I have never used to something that I am using daily. Even if the progress ends up plateauing (which I do not believe will happen in the near term), behaviors are changing; OpenAI is capturing users, and taking them from companies like Google. Google may be able to fight back and win - Gemini 2.5 Pro is great - but any company sitting this out risks being unable to capture users back from Open AI at a later date.

bigstrat2003 1y ago
No, it's still just a toy. Until they can make the models actually consistently good at things, they aren't going to be useful. Right now they still BS you far too much to trust them, and because you have to double check their work every time they are worse than no tool at all.
csours 1y ago
To extend your illustration, 5 years ago no one could train an LLM with the capabilities of a 13 year old human; now many companies can both train LLMs and integrate them into products. > taken it from a toy to genuinely insanely useful. Really?
devjab 1y ago
> any company sitting this out risks being unable to capture users back from Open AI at a later date. Why? I paid for Claude for a while, but with Deepseek, Gemini and the free hits on Mistral, ChatGPT, Claude and Perplexity I'm not sure why I would now. This is anecdotal of course, but I'm very rarely unique in my behaviour. I think the best the subscription companies can hope for is that their subscribers don't realize that Deepseek and Gemini can basically do all you need for free.
- poormathskills 1y ago
  >I'm very rarely unique in my behaviour I cannot stress this enough: if you know what Deepseek, Claude, Mistral, and Perplexity are, you are not a typical consumer. Arguably, if you have used a single one of those brands you are not a typical consumer. The vast majority of people have used ChatGPT and nothing else, except maybe clicking on Gemini or Meta AI by accident.
  - qcic 1y ago
    I doubt it. Google is shoving Gemini on everyone’s face through search, and Meta AI is embedded in every Meta product. Heck, instagram created a bot marketplace. They might not “know” the brand as well as ChatGPT, but the average consumer has definitely been exposed to those at the very least. DeepSeek also made a lot of noise, to the point that, anecdotally, I’ve seen a lot of people outside of tech using it.
mplewis 1y ago
It's been five years. There is no AI killer app. Agentic coding is still hot garbage. Normal people don't want to use AI tools despite them being shoved into every SaaS under the sun. LLMs are most famous among non-tech users for telling you to put glue into pizza. No one has been able to scale their chatbots into something profitable, and no one can put a date on when they'll be profitable. Why are you still pretending anything is going to come out of this?
apsurd 1y ago
I can't square how OpenAi can capture users and presumably retain them when the incumbents have been capturing users for multiple decades and why can they not retain them? If every major player had an AI option, i'm just not understanding how because OpenAi moved first or got big first, the hugely massively successful companies that did the same thing for multiple decades don't have the same advantage?
- orionsbelt 1y ago
  Who knows how this will play out, but user behavior is always somewhat sticky and OpenAI now has 400M+ weekly active users. Currently, I'm not sure there is much of a moat, as many would jump if, say, Google released a model that is 10x better. However, there are myriad ways that OpenAI could slowly try to make their userbase even stickier: 1. OpenAI is apparently in the process of building a social network. 2. OpenAI is apparently working with Jonny Ive on some sort of hardware. 3. OpenAI is increasingly working on "memory" as a LLM feature. Users may be less likely to switch as an LLM increasingly feels like a person that knows you, understands you, has a history with you, etc. 4. Google and MSFT are leveraging their existing strengths. Perhaps you will stick with Gemini given deep integration with Android, Google Drive, Sheets, Docs, etc. 5. LLMs, as depressing as this sounds, will increasingly be used for romantic/friend purposes. These users may not want to switch, as it would be like breaking up and finding a new partner. 6. Your chat history, if it can't be easily exported/imported, may be a sticky feature, especially if it can be improved (e.g. easily search, cross-reference, chats, like a supercharged interconnecting note app with brains). I could list 100 more of these. Perhaps none of the above will happen, but again, they have 400M weekly users and they will find ways to keep them. It's a lot easier to keep users that have a habit of showing up, then getting them in the first place. There's a reason that Google is treating this like an emergency; they are at serious risk of having their search cash cow permanently disrupted if they don't act fast to win.
  - apsurd 1y ago
    Very thought provoking reply. #3 sounds the most sticky to me, in the product sense that you'd build "your own LLM/agent" and plug it other services. I heard this on a product podcast [1], think of it like Okta SSO integration: access controls for your personal/sensitive LLM stuff vs all other services trying to get you to use their LLM. #5 stands out as well as a substantial barrier. The rest to me our sticky, but no more uniquely sticky than any other service that retains data. Like the switching cost of email or a browser. It does stick but not insurmountable and once the switch is made, it's like why did I wait so long? (I'm a Safari user!) Anyway, thanks for the thoughtful reply. [1] https://www.reforge.com/podcast/unsolicited-feedback/the-great-ai-corporate-divide https://www.reforge.com/podcast/unsolicited-feedback/the-gre...

bigstrat2003 1y ago

No, it's still just a toy. Until they can make the models actually consistently good at things, they aren't going to be useful. Right now they still BS you far too much to trust them, and because you have to double check their work every time they are worse than no tool at all.

csours 1y ago

To extend your illustration, 5 years ago no one could train an LLM with the capabilities of a 13 year old human; now many companies can both train LLMs and integrate them into products. > taken it from a toy to genuinely insanely useful. Really?

devjab 1y ago

> any company sitting this out risks being unable to capture users back from Open AI at a later date. Why? I paid for Claude for a while, but with Deepseek, Gemini and the free hits on Mistral, ChatGPT, Claude and Perplexity I'm not sure why I would now. This is anecdotal of course, but I'm very rarely unique in my behaviour. I think the best the subscription companies can hope for is that their subscribers don't realize that Deepseek and Gemini can basically do all you need for free.

poormathskills 1y ago
>I'm very rarely unique in my behaviour I cannot stress this enough: if you know what Deepseek, Claude, Mistral, and Perplexity are, you are not a typical consumer. Arguably, if you have used a single one of those brands you are not a typical consumer. The vast majority of people have used ChatGPT and nothing else, except maybe clicking on Gemini or Meta AI by accident.
- qcic 1y ago
  I doubt it. Google is shoving Gemini on everyone’s face through search, and Meta AI is embedded in every Meta product. Heck, instagram created a bot marketplace. They might not “know” the brand as well as ChatGPT, but the average consumer has definitely been exposed to those at the very least. DeepSeek also made a lot of noise, to the point that, anecdotally, I’ve seen a lot of people outside of tech using it.

mplewis 1y ago

It's been five years. There is no AI killer app. Agentic coding is still hot garbage. Normal people don't want to use AI tools despite them being shoved into every SaaS under the sun. LLMs are most famous among non-tech users for telling you to put glue into pizza. No one has been able to scale their chatbots into something profitable, and no one can put a date on when they'll be profitable. Why are you still pretending anything is going to come out of this?

apsurd 1y ago

I can't square how OpenAi can capture users and presumably retain them when the incumbents have been capturing users for multiple decades and why can they not retain them? If every major player had an AI option, i'm just not understanding how because OpenAi moved first or got big first, the hugely massively successful companies that did the same thing for multiple decades don't have the same advantage?

orionsbelt 1y ago
Who knows how this will play out, but user behavior is always somewhat sticky and OpenAI now has 400M+ weekly active users. Currently, I'm not sure there is much of a moat, as many would jump if, say, Google released a model that is 10x better. However, there are myriad ways that OpenAI could slowly try to make their userbase even stickier: 1. OpenAI is apparently in the process of building a social network. 2. OpenAI is apparently working with Jonny Ive on some sort of hardware. 3. OpenAI is increasingly working on "memory" as a LLM feature. Users may be less likely to switch as an LLM increasingly feels like a person that knows you, understands you, has a history with you, etc. 4. Google and MSFT are leveraging their existing strengths. Perhaps you will stick with Gemini given deep integration with Android, Google Drive, Sheets, Docs, etc. 5. LLMs, as depressing as this sounds, will increasingly be used for romantic/friend purposes. These users may not want to switch, as it would be like breaking up and finding a new partner. 6. Your chat history, if it can't be easily exported/imported, may be a sticky feature, especially if it can be improved (e.g. easily search, cross-reference, chats, like a supercharged interconnecting note app with brains). I could list 100 more of these. Perhaps none of the above will happen, but again, they have 400M weekly users and they will find ways to keep them. It's a lot easier to keep users that have a habit of showing up, then getting them in the first place. There's a reason that Google is treating this like an emergency; they are at serious risk of having their search cash cow permanently disrupted if they don't act fast to win.
- apsurd 1y ago
  Very thought provoking reply. #3 sounds the most sticky to me, in the product sense that you'd build "your own LLM/agent" and plug it other services. I heard this on a product podcast [1], think of it like Okta SSO integration: access controls for your personal/sensitive LLM stuff vs all other services trying to get you to use their LLM. #5 stands out as well as a substantial barrier. The rest to me our sticky, but no more uniquely sticky than any other service that retains data. Like the switching cost of email or a browser. It does stick but not insurmountable and once the switch is made, it's like why did I wait so long? (I'm a Safari user!) Anyway, thanks for the thoughtful reply. [1] https://www.reforge.com/podcast/unsolicited-feedback/the-great-ai-corporate-divide https://www.reforge.com/podcast/unsolicited-feedback/the-gre...