4 ms·

An acquaintance of mine has a start-up in this space and uses OpenAI to do essentially the same thing. This must look like, and may well be, the guillotine for

An acquaintance of mine has a start-up in this space and uses OpenAI to do essentially the same thing. This must look like, and may well be, the guillotine for him... It's my primary fear building anything on these models, they can just come eat your lunch once it looks yummy enough. Tread carefully

potatolicious 1y ago
> "they can just come eat your lunch once it looks yummy enough. Tread carefully" True, and worse, they're hungry because it's increasingly seeming like "hosting LLMs and charging by the token" is not terribly profitable. I don't really see a path for the major players that isn't "Sherlock everything that achieves traction".
- thimabi 1y ago
  But what’s the future in terms of profitability of LLM providers? As long as features like Study Mode are little more than creative prompting, any provider will eventually be able to offer them and offer token-based charging.
  - potatolicious 1y ago
    I think a few points worth making here: - From what I can see many products are rapidly getting past "just prompt engineering the base API". So even though a lot of these things were/are primitive, I don't think it's necessarily a good bet that they will remain so. Though agree in principle - thin API wrappers will be out-competed both by cheaper thin wrappers, or products that are more sophisticated/better than thin wrappers. - This is, oddly enough, a scenario that is way easier to navigate than the rest of the LLM industry. We know consumer apps, we know consumer apps that do relatively basic (or at least, well understood) things. Success/failure then is way less about technical prowess and more about classical factors like distribution, marketing, integrations, etc. A good example here is the lasting success of paid email providers. Multiple vendors (MSFT, GOOG, etc.) make huge amounts of money hosting people's email, despite it being a mature product that, at the basic level, is pretty solved, and where the core product can be replicated fairly easily. The presence of open source/commodity commercial offerings hasn't really driven the price of the service to the floor, though the commodity offerings do provide some pricing pressure.
    - m11a 1y ago
      Email is pretty difficult to reliably self-host though, and typically a PITA to manage. And you really don’t ever want to lose your email address or the associated data. Fewer people could say they properly secure, manage and administer a VPS on which they can host the email server they eventually setup, over say a 10yr period. Most people I saw offer self-hosted emails for groups (student groups etc), it ended up a mess. Compare all that to say ollama, which makes self-hosting LLMs trivial, and they’re stateless. So I’m not sure email is a good example of commodity not bringing price to the floor.
  - mvieira38 1y ago
    We can assume that OpenAI/Anthropic offerings are going to be better long term simply because they have more human capital, though, right? If it turns out that what really matters in the AI race is study mode, then OpenAI goes "ok let's pivot the hundreds of genius level, well-paid engineers to that issue. AND our engineers can use every tool we offer for free without limits, even experimental models". It's tough for the small AI startup to compete with that, the best hope is to be bought like Windsurf
- falcor84 1y ago
  Thanks for introducing me to the verb Sherlock! I'm one of today's lucky 10,000. > In the computing verb sense, refers to the software Sherlock, which in 2002 came to replicate some of the features of an earlier complementary program called Watson.[1] [1] https://en.wiktionary.org/wiki/Sherlock https://en.wiktionary.org/wiki/Sherlock
mvieira38 1y ago
How can't these founders see this happening, too? From the start OpenAI has been getting into more markets than just "LLM provider"
- tokioyoyo 1y ago
  There’s a case for a start up to capture enough market that LLM providers would just buy it out. Think of CharacterAI case.
  - jonny_eh 1y ago
    Character AI was never acquired, it remains independent.
- azinman2 1y ago
  They originally claimed they wouldn’t as to not compete with their API users…
  - rs186 1y ago
    [citation needed]
sebzim4500 1y ago
I'm too young to have experienced this, but I'm sure others here aren't. During the early days of tech, was there prevailing wisdom that software companies would never be able to compete with hardware companies because the hardware companies would always be able to copy them and ship the software with the hardware? Because I think it's basically the analogous situation. People assume that the foundation model providers have some massive advantage over the people building on top of them, but I don't really see any evidence for this.
- jonny_eh 1y ago
  Claude Code and Gemini-CLI are able to offer much more value compared to startups (like Cursor) that need to pay for model access, largely due to the immense costs involved.
- draebek 1y ago
  Does https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked_as_a_term https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked... count? (Edit: Missed I wasn't the first to post this in a sibling.)
jstummbillig 1y ago
Ah, I don't know. Of course there is risk involved no matter what we do (see the IDE/Cursor space), but we need to be somewhat critical of the value we add. If you want to try and make a quick buck, fine, be quick and go for whatever. If you plan on building a long term business, don't do the most obvious, low effort low hanging fruit stuff.
- chrisweekly 1y ago
  yeah, if you want to stick around you need some kind of moat
teaearlgraycold 1y ago
I used to work for copy.ai and this happened to them. Investors always asked if the founders were worried about OpenAI competing with their consumer product. Then ChatGPT released. Turns out that was a reasonable concern. These days they’ve pivoted to a more enterprise product and are still chugging along.
djeastm 1y ago
Yes, any LLM-adjacent application developer should be concerned. Even if they don't do 100% of what your product does, their market reach and capitalization is scary. Any model/tooling improvements that just happen to encroach in your domain will put you on the clock...
mpalmer 1y ago
No disrespect to your acquaintance, but when I heard about this, I didn't think "oh a lot of startups are gonna go under", I thought "OAI added an option to use a hard-coded system prompt and they're calling it a 'mode'??"
- deleted 1y ago
  [deleted]
- fny 1y ago
  This is step 1. Like they say in the post, they will learn from the data.
senko 1y ago
This is actually a public validation for your friend's startup. A proper learning tool will have history of conversation with the student, understand their knowledge level, have handcrafted curricula (to match whatever the student is supposed to learn), and be less susceptible to hallucination. OpenAI have a bunch of other things to worry about and won't just pivot to this space.
jimmydoe 1y ago
This risk should be priced in on day one. All bigco today, not only foundational model providers but also in media and other vertical, tend to be a platform for end user. They don’t want middle man. If you are trying to be a middle man, you should be prepared.

potatolicious 1y ago

> "they can just come eat your lunch once it looks yummy enough. Tread carefully" True, and worse, they're hungry because it's increasingly seeming like "hosting LLMs and charging by the token" is not terribly profitable. I don't really see a path for the major players that isn't "Sherlock everything that achieves traction".

thimabi 1y ago
But what’s the future in terms of profitability of LLM providers? As long as features like Study Mode are little more than creative prompting, any provider will eventually be able to offer them and offer token-based charging.
- potatolicious 1y ago
  I think a few points worth making here: - From what I can see many products are rapidly getting past "just prompt engineering the base API". So even though a lot of these things were/are primitive, I don't think it's necessarily a good bet that they will remain so. Though agree in principle - thin API wrappers will be out-competed both by cheaper thin wrappers, or products that are more sophisticated/better than thin wrappers. - This is, oddly enough, a scenario that is way easier to navigate than the rest of the LLM industry. We know consumer apps, we know consumer apps that do relatively basic (or at least, well understood) things. Success/failure then is way less about technical prowess and more about classical factors like distribution, marketing, integrations, etc. A good example here is the lasting success of paid email providers. Multiple vendors (MSFT, GOOG, etc.) make huge amounts of money hosting people's email, despite it being a mature product that, at the basic level, is pretty solved, and where the core product can be replicated fairly easily. The presence of open source/commodity commercial offerings hasn't really driven the price of the service to the floor, though the commodity offerings do provide some pricing pressure.
  - m11a 1y ago
    Email is pretty difficult to reliably self-host though, and typically a PITA to manage. And you really don’t ever want to lose your email address or the associated data. Fewer people could say they properly secure, manage and administer a VPS on which they can host the email server they eventually setup, over say a 10yr period. Most people I saw offer self-hosted emails for groups (student groups etc), it ended up a mess. Compare all that to say ollama, which makes self-hosting LLMs trivial, and they’re stateless. So I’m not sure email is a good example of commodity not bringing price to the floor.
- mvieira38 1y ago
  We can assume that OpenAI/Anthropic offerings are going to be better long term simply because they have more human capital, though, right? If it turns out that what really matters in the AI race is study mode, then OpenAI goes "ok let's pivot the hundreds of genius level, well-paid engineers to that issue. AND our engineers can use every tool we offer for free without limits, even experimental models". It's tough for the small AI startup to compete with that, the best hope is to be bought like Windsurf
falcor84 1y ago
Thanks for introducing me to the verb Sherlock! I'm one of today's lucky 10,000. > In the computing verb sense, refers to the software Sherlock, which in 2002 came to replicate some of the features of an earlier complementary program called Watson.[1] [1] https://en.wiktionary.org/wiki/Sherlock https://en.wiktionary.org/wiki/Sherlock

mvieira38 1y ago

How can't these founders see this happening, too? From the start OpenAI has been getting into more markets than just "LLM provider"

tokioyoyo 1y ago
There’s a case for a start up to capture enough market that LLM providers would just buy it out. Think of CharacterAI case.
- jonny_eh 1y ago
  Character AI was never acquired, it remains independent.
azinman2 1y ago
They originally claimed they wouldn’t as to not compete with their API users…
- rs186 1y ago
  [citation needed]

sebzim4500 1y ago

I'm too young to have experienced this, but I'm sure others here aren't. During the early days of tech, was there prevailing wisdom that software companies would never be able to compete with hardware companies because the hardware companies would always be able to copy them and ship the software with the hardware? Because I think it's basically the analogous situation. People assume that the foundation model providers have some massive advantage over the people building on top of them, but I don't really see any evidence for this.

jonny_eh 1y ago
Claude Code and Gemini-CLI are able to offer much more value compared to startups (like Cursor) that need to pay for model access, largely due to the immense costs involved.
draebek 1y ago
Does https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked_as_a_term https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked... count? (Edit: Missed I wasn't the first to post this in a sibling.)

jstummbillig 1y ago

Ah, I don't know. Of course there is risk involved no matter what we do (see the IDE/Cursor space), but we need to be somewhat critical of the value we add. If you want to try and make a quick buck, fine, be quick and go for whatever. If you plan on building a long term business, don't do the most obvious, low effort low hanging fruit stuff.

chrisweekly 1y ago
yeah, if you want to stick around you need some kind of moat

teaearlgraycold 1y ago

I used to work for copy.ai and this happened to them. Investors always asked if the founders were worried about OpenAI competing with their consumer product. Then ChatGPT released. Turns out that was a reasonable concern. These days they’ve pivoted to a more enterprise product and are still chugging along.

djeastm 1y ago

Yes, any LLM-adjacent application developer should be concerned. Even if they don't do 100% of what your product does, their market reach and capitalization is scary. Any model/tooling improvements that just happen to encroach in your domain will put you on the clock...

mpalmer 1y ago

No disrespect to your acquaintance, but when I heard about this, I didn't think "oh a lot of startups are gonna go under", I thought "OAI added an option to use a hard-coded system prompt and they're calling it a 'mode'??"

deleted 1y ago
[deleted]
fny 1y ago
This is step 1. Like they say in the post, they will learn from the data.

senko 1y ago

This is actually a public validation for your friend's startup. A proper learning tool will have history of conversation with the student, understand their knowledge level, have handcrafted curricula (to match whatever the student is supposed to learn), and be less susceptible to hallucination. OpenAI have a bunch of other things to worry about and won't just pivot to this space.

jimmydoe 1y ago

This risk should be priced in on day one. All bigco today, not only foundational model providers but also in media and other vertical, tend to be a platform for end user. They don’t want middle man. If you are trying to be a middle man, you should be prepared.