5 ms·

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and saf

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.

dvt 2y ago
In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product.
- siquick 2y ago
  Uploading a a photo/file to a server does not a product make yet you referenced Dropbox and Instagram and that’s what they started out as. The UX of the AI applications is the moat and the infrastructure providers behind those applications is pretty much always OpenAI and Anthropic at the moment because running your own open source LLMs (which are inferior out of the box) at scale is not cheap or easy to do it right - same reason most companies use cloud. Agree that once you hit super scale then you can run your own infrastructure but there are thousand of companies who won’t get that far and still need an LLM.
  - iknownthing 2y ago
    Agreed. The LLMs have effectively been democratized. As much as people deride "LLM wrappers", the quality of the wrapper and how creatively they use the LLM api is the differentiator.
- fragmede 2y ago
  That's one opinion. the number off people who get value out of ChatGPT is known only to them, but anecdotally it's pretty high. a lot of people I know are actively using it on a daily basis and paying them $20/month.
  - walthamstow 2y ago
    Anecdote: me (SWE) and my wife (executive) each have a $20 subscription.
    - IanCal 2y ago
      n=3 now so it's full on anecdata, I've got a $20 sub professionally (swe). It has to save me so little time to be worth it it's easily great. Might add claude, though at this point probably better to find a nice interface and use the APIs. These things are products.
      - hiAndrewQuinn 2y ago
        n=5, my wife and I split a subscription between the two of us. While our needs are usually pretty minor, we both work in the computer science space and GPT-4o's ability to e.g. generate good example sentences for my Finnish vocabulary learning is astonishingly good. I'm building a wrapper around it so I can generate them en masse at [1], but it's still quite early days and very obviously not software I intend to sell https://GitHub.com/hiAndrewQuinn/wordmeat https://GitHub.com/hiAndrewQuinn/wordmeat
        deleted 2y ago
        [deleted]
- mike_hearn 2y ago
  LLMs are clearly products! The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product, it just means that they're competing against a company funding open source stuff for (presumably) strategic reasons, like in many other markets. And like in many other markets over time it's possible the proprietary versions will establish permanent competitive advantage. Meta may not keep releasing Llamas forever. > you would just eventually train and deploy your own model Just train an LLM? It's really not that easy! Even if it was, it'd be like how people can "just" run their own email service. Hosted email API is not rocket science but in practice companies all choose to pay Microsoft or Google to do it. Doing these things isn't a core competitive advantage so it gets outsourced.
  - throwaway2037 2y ago
    > The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product Real question: Why are so many LLMs given away for free? Are they hoping to crush non-free alternatives? EDIT Your last paragraph makes an excellent point. In the near future, I could see big corps paying OpenAI (or a competitor) to train a private LLM on their squillion internal documents and build a very good helpdesk agent. (Legal and compliance would love it.)
    - mike_hearn 2y ago
      Paying OpenAI for fine tunes on internal docs is already happening. Giving expensive things away for free is a great marketing technique that has been used since time immemorial, so why startups like Stability do it is somewhat understandable. And OpenAI uses free API access as a loss leader for their API product so that's understandable too. Why Meta/Google/others do open weight releases is a bit less clear. Recall though that the first Llama wasn't really an open source release. You had to sign a document saying you were a researcher to get the weights, and that document was an agreement to keep the weights secret. Two people signed the documents, anonymously compared their weights, discovered they weren't watermarked (i.e. Meta didn't take this seriously, it was a sop to their AI politics/safety people) and promptly leaked them. Presumably this was useful for the more libertarian wing of Meta as they could then prove the sky wouldn't fall, and so the influence shifted towards those arguing for more openness in research in general. With that Rubicon crossed other companies didn't see competitive advantage in withholding their similar sized models anymore and followed the leader, so to speak. Sometimes it also feels like Meta may have over-purchased GPUs and - lacking a public cloud - have just decided to let their researchers do what they wanted. Which is great for the public! But we mustn't be too overconfident. This is really only possible because of Zuckerberg's unique corporate structure that makes him unfirable, combined with Meta being a big data company. It's really benefiting all of humanity here because he's invulnerable to board action so doesn't have to worry about heat from shareholders over 'wasting' money like this. There's a lot of R&D being done right now on shrinking models whilst preserving quality, so hopefully the Zuck's generosity is enough to ride the open AI research community through the hard times when you needed billions to train LLMs.
- pmontra 2y ago
  If I had to use Llama or Mistral for free (money) I'd have to buy that RTX 3080 card plus the computer to fit it into. It doesn't work with my laptop. So I use ChatGPT for free instead, on OpenAI site with the don't archive option or whatever they call it. I think many people use the free (money) and hosted option.
- jokethrowaway 2y ago
  Llama and Mistral are not really competitive but they can be used as a base to finetune for a very specific usecase. Then they don't suck as much. With OpenAI and Claude, you throw some text instructions and you get back the answers which are surprisingly correct (minus a few exceptions). In order to replicate that with Llama you'd probably need N-hundreds finetunes and a model to decide which finetunes to use.
- meiraleal 2y ago
  > In my humble opinion, a chat interface (API or not) does not a product make. Well, you are humbly wrong then. > You would just eventually train and deploy your own model because an LLM is not a product. Hallucinations aren't exclusive to LLMs it seems.
  - williamcotton 2y ago
    We’re already seeing a lot of competition between LLMs. They are quickly becoming commodities. Margins will approach zero and the real value proposition will be with consumer products that extend beyond an <input type=“text” />.
    - VBprogrammer 2y ago
      I agree with you. I'm using a couple of different LLMs depending on what I'm doing and what happens to be easiest but the difference between them is marginal in my experience. The only play for OpenAI et al in my opinion is to try to pull up the draw bridge behind them by getting legislation passed which makes compliance prohibitively difficult if that's not your core business.
      - throwaway2037 2y ago
        I think OpenAI has a good future by having much better training data. That is a super hard problem that requires thousands of low cost workers in developing countries to tag and arrange training data.
    - TimPC 2y ago
      I disagree they will become commodities because most of my use cases are more sensitive to accuracy than cost. We typically have usage volume that isn't absurd and for large enterprise customers our LLM budget is a rounding error. Meanwhile our product saves them many hours of a data engineer. If we can pay double to get a 10% performance boost we will do so gladly. You can already see this in LLM pricing where they have cheap models that deliver low performance a My bet is that 80% of profits will be made on the workloads that are sensitive to accuracy and workloads where running an LLM at all gets you most of the benefit will become very commoditized.
- manishsharan 2y ago
  >> LLM is not a product Would you consider Database to be a product ? SQLLite , PostGres etc. are free and yet we have Oracle , Mongodb and MS SQL doing billions in revenues.
- marcosdumay 2y ago
  > an LLM is not a product I imagine you meant to say that LLMs are comoditized. Getting the correct words here is important, as you can see by all the people disagreeing on the literal interpretation of your post. And yeah. LLMs have got the fastest transition from highly innovative singular product to plain commodity I've ever seen or read about. BSD licensed software libraries do not move that quickly. They were mostly not even adopted yet, and have a huge barrier to entry, what makes it much more of a feat.
  - dvt 2y ago
    > I imagine you meant to say that LLMs are comoditized. Yep, much better way of putting it!
mlsu 2y ago
there is a lot of consumer surplus, but it’s very unclear if any one of these companies will be able to capture it. Especially if mostly good enough commodity LLMs like Llama 70b are in the mix. Where is the differentiation?
- physicsguy 2y ago
  Not to mention that fine tuned smaller parameter models are much cheaper to run. See: Google embedding a model into the latest version of Chrome, accessible through dev tools.

dvt 2y ago

In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product.

siquick 2y ago
Uploading a a photo/file to a server does not a product make yet you referenced Dropbox and Instagram and that’s what they started out as. The UX of the AI applications is the moat and the infrastructure providers behind those applications is pretty much always OpenAI and Anthropic at the moment because running your own open source LLMs (which are inferior out of the box) at scale is not cheap or easy to do it right - same reason most companies use cloud. Agree that once you hit super scale then you can run your own infrastructure but there are thousand of companies who won’t get that far and still need an LLM.
- iknownthing 2y ago
  Agreed. The LLMs have effectively been democratized. As much as people deride "LLM wrappers", the quality of the wrapper and how creatively they use the LLM api is the differentiator.
fragmede 2y ago
That's one opinion. the number off people who get value out of ChatGPT is known only to them, but anecdotally it's pretty high. a lot of people I know are actively using it on a daily basis and paying them $20/month.
- walthamstow 2y ago
  Anecdote: me (SWE) and my wife (executive) each have a $20 subscription.
  - IanCal 2y ago
    n=3 now so it's full on anecdata, I've got a $20 sub professionally (swe). It has to save me so little time to be worth it it's easily great. Might add claude, though at this point probably better to find a nice interface and use the APIs. These things are products.
    - hiAndrewQuinn 2y ago
      n=5, my wife and I split a subscription between the two of us. While our needs are usually pretty minor, we both work in the computer science space and GPT-4o's ability to e.g. generate good example sentences for my Finnish vocabulary learning is astonishingly good. I'm building a wrapper around it so I can generate them en masse at [1], but it's still quite early days and very obviously not software I intend to sell https://GitHub.com/hiAndrewQuinn/wordmeat https://GitHub.com/hiAndrewQuinn/wordmeat
      - deleted 2y ago
        [deleted]
mike_hearn 2y ago
LLMs are clearly products! The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product, it just means that they're competing against a company funding open source stuff for (presumably) strategic reasons, like in many other markets. And like in many other markets over time it's possible the proprietary versions will establish permanent competitive advantage. Meta may not keep releasing Llamas forever. > you would just eventually train and deploy your own model Just train an LLM? It's really not that easy! Even if it was, it'd be like how people can "just" run their own email service. Hosted email API is not rocket science but in practice companies all choose to pay Microsoft or Google to do it. Doing these things isn't a core competitive advantage so it gets outsourced.
- throwaway2037 2y ago
  > The fact that Meta rather inexplicably chooses to give away assets that cost millions or billions to create doesn't mean LLMs aren't a product Real question: Why are so many LLMs given away for free? Are they hoping to crush non-free alternatives? EDIT Your last paragraph makes an excellent point. In the near future, I could see big corps paying OpenAI (or a competitor) to train a private LLM on their squillion internal documents and build a very good helpdesk agent. (Legal and compliance would love it.)
  - mike_hearn 2y ago
    Paying OpenAI for fine tunes on internal docs is already happening. Giving expensive things away for free is a great marketing technique that has been used since time immemorial, so why startups like Stability do it is somewhat understandable. And OpenAI uses free API access as a loss leader for their API product so that's understandable too. Why Meta/Google/others do open weight releases is a bit less clear. Recall though that the first Llama wasn't really an open source release. You had to sign a document saying you were a researcher to get the weights, and that document was an agreement to keep the weights secret. Two people signed the documents, anonymously compared their weights, discovered they weren't watermarked (i.e. Meta didn't take this seriously, it was a sop to their AI politics/safety people) and promptly leaked them. Presumably this was useful for the more libertarian wing of Meta as they could then prove the sky wouldn't fall, and so the influence shifted towards those arguing for more openness in research in general. With that Rubicon crossed other companies didn't see competitive advantage in withholding their similar sized models anymore and followed the leader, so to speak. Sometimes it also feels like Meta may have over-purchased GPUs and - lacking a public cloud - have just decided to let their researchers do what they wanted. Which is great for the public! But we mustn't be too overconfident. This is really only possible because of Zuckerberg's unique corporate structure that makes him unfirable, combined with Meta being a big data company. It's really benefiting all of humanity here because he's invulnerable to board action so doesn't have to worry about heat from shareholders over 'wasting' money like this. There's a lot of R&D being done right now on shrinking models whilst preserving quality, so hopefully the Zuck's generosity is enough to ride the open AI research community through the hard times when you needed billions to train LLMs.
pmontra 2y ago
If I had to use Llama or Mistral for free (money) I'd have to buy that RTX 3080 card plus the computer to fit it into. It doesn't work with my laptop. So I use ChatGPT for free instead, on OpenAI site with the don't archive option or whatever they call it. I think many people use the free (money) and hosted option.
jokethrowaway 2y ago
Llama and Mistral are not really competitive but they can be used as a base to finetune for a very specific usecase. Then they don't suck as much. With OpenAI and Claude, you throw some text instructions and you get back the answers which are surprisingly correct (minus a few exceptions). In order to replicate that with Llama you'd probably need N-hundreds finetunes and a model to decide which finetunes to use.
meiraleal 2y ago
> In my humble opinion, a chat interface (API or not) does not a product make. Well, you are humbly wrong then. > You would just eventually train and deploy your own model because an LLM is not a product. Hallucinations aren't exclusive to LLMs it seems.
- williamcotton 2y ago
  We’re already seeing a lot of competition between LLMs. They are quickly becoming commodities. Margins will approach zero and the real value proposition will be with consumer products that extend beyond an <input type=“text” />.
  - VBprogrammer 2y ago
    I agree with you. I'm using a couple of different LLMs depending on what I'm doing and what happens to be easiest but the difference between them is marginal in my experience. The only play for OpenAI et al in my opinion is to try to pull up the draw bridge behind them by getting legislation passed which makes compliance prohibitively difficult if that's not your core business.
    - throwaway2037 2y ago
      I think OpenAI has a good future by having much better training data. That is a super hard problem that requires thousands of low cost workers in developing countries to tag and arrange training data.
  - TimPC 2y ago
    I disagree they will become commodities because most of my use cases are more sensitive to accuracy than cost. We typically have usage volume that isn't absurd and for large enterprise customers our LLM budget is a rounding error. Meanwhile our product saves them many hours of a data engineer. If we can pay double to get a 10% performance boost we will do so gladly. You can already see this in LLM pricing where they have cheap models that deliver low performance a My bet is that 80% of profits will be made on the workloads that are sensitive to accuracy and workloads where running an LLM at all gets you most of the benefit will become very commoditized.
manishsharan 2y ago
>> LLM is not a product Would you consider Database to be a product ? SQLLite , PostGres etc. are free and yet we have Oracle , Mongodb and MS SQL doing billions in revenues.
marcosdumay 2y ago
> an LLM is not a product I imagine you meant to say that LLMs are comoditized. Getting the correct words here is important, as you can see by all the people disagreeing on the literal interpretation of your post. And yeah. LLMs have got the fastest transition from highly innovative singular product to plain commodity I've ever seen or read about. BSD licensed software libraries do not move that quickly. They were mostly not even adopted yet, and have a huge barrier to entry, what makes it much more of a feat.
- dvt 2y ago
  > I imagine you meant to say that LLMs are comoditized. Yep, much better way of putting it!

mlsu 2y ago

there is a lot of consumer surplus, but it’s very unclear if any one of these companies will be able to capture it. Especially if mostly good enough commodity LLMs like Llama 70b are in the mix. Where is the differentiation?

physicsguy 2y ago
Not to mention that fine tuned smaller parameter models are much cheaper to run. See: Google embedding a model into the latest version of Chrome, accessible through dev tools.