7 ms·

The winner is going to be the consumers of AI. It's a race to the bottom on pricing on the provider/infra side. It seems very unlikely that any single LLM prov

The winner is going to be the consumers of AI. It's a race to the bottom on pricing on the provider/infra side. It seems very unlikely that any single LLM provider will achieve a sustained and durable advantage enough to achieve large margins on their product. Consumers can swap between providers with relative ease, and there is very little stickiness to these LLM APIs, since the interfaces are general and operate on natural language. Versus something like building out a Salesforce integration and then trying to update that to a competitor. Or migrating from Mongo to DynamoDB. Building the LLMs is where the cool tech lives, but surprised so many are seeing that as a compelling investment opportunity. But, let's see!

msikora 3y ago
Yep. I wonder why there is not more features that would differentiate your LLM API, like for example the Functions in OpenAI LLM APIs. While not perfect it is extremely useful and not aware of a similar offering (I do know that there are Python packages offering similar functionality but it is my understanding that they don't work as well as the OpenAI Functions).
tomrod 3y ago
Aye, open models from Meta are tearing at the moat.
user3939382 3y ago
> The winner is going to be the consumers Cloud infra may be a comparable market, since computation is a big share of AI costs. Did consumers win big from competition between AWS, Azure, and GCP? Not sure. I see an uptick in write ups saying “We switched off cloud and reduced costs by 2/3rds.” Not a scientific sample but may leave the question open.
- adam_arthur 3y ago
  Much harder to switch cloud providers than to switch LLM models. How much time would it take most companies to move their product from AWS to GCP, for example? What if you use a cloud specific tool like DynamoDB? Margin is a function of stickiness/cost of switching (among other things). I suspect eventually we will enter a world where migrating cloud providers is mostly a click of a button, but we're a long ways off from that. Requires vendor agnostic and portable apis/containers/WASM runtimes everywhere. Swapping an LLM, at least in the current state, is about as close to updating a pointer as you can get.
  - throwaway50182 3y ago
    Well I'd have to change my Terraform provider and the managed Kubernetes resource... Other than that, it'd be the same. So half an hour of coding + half an hour of reconfiguring CI secrets?
    - adam_arthur 3y ago
      Pretty obvious to most that switching cloud providers is not quick or painless for the majority of orgs. There's not really an argument in good faith to suggest otherwise. Especially given that many orgs use managed or cloud specific solutions that have no 1:1 mapping between vendors
      - throwaway50182 3y ago
        I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more.
        atmosx 3y ago
        > I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more. As someone who worked on a similar project recently, I'm getting that idea that you obviously don't know what you're talking about.
        motoxpro 3y ago
        Didn't you know? It's only one DB ENV variable and you're swapped over from GCP to AWS. /s
        throwaway50182 3y ago
        I'm running the tech for a global startup with 100M EUR turnover. It's still pretty small, but it's something. You need to plan ahead, sure. Portability was one of my main concerns (including the possibility to go self-hosted). But it's definitely not impossible, nor too hard to do.
- vikramkr 3y ago
  There are a HUGE number of startups that would have had a massively harder time if they had to roll their own infra. Who cares if the companies eventually have to move off? (Though even Netflix seems ok with it overall for now). There are a ton of services that just wouldn't exist otherwise
  - jokethrowaway 3y ago
    I disagree, at early startup scale you don't need much, you just buy a better VPS when you need to scale up Learning aws vs learning how to operate a vps are of comparable complexity It took me less then a day to setup infra for my startup (more than 10 years ago)
    - throwaway50182 3y ago
      If you're just running VPS on the big clouds, you're not going to get much advantage out of it, indeed. But tell me what third tier cheap provider has managed scale-to-zero-or-infinity functions? Managed storage with S3-like API? Where can I get an API gateway cheaper than Amazon? What about managed databases? These tools allow me to develop insanely scalable software incredibly easily. Agreed - all the big clouds are very expensive VPS hostings. Don't use it for that.
      - pleoxy 3y ago
        You get portability. Which the functions do not provide. Open source solutions have a longer career utility than proprietary offerings. I remember when NetWare certs were all the rage. Useless now. I remember msce. But if you learned open tools 35 years ago instead... You get the picture. You can scale from 1 thread 512MB RAM, to 500 threads and 12TB of RAM (off the shelf). Which is good enough for almost everyone who isn't planet scale. Auto scaling also comes with auto billing. Oops, your accidental infinite loop spawning functions has bankrupted your company. You don't have that risk starting with a VPS.
        throwaway50182 3y ago
- peteradio 3y ago
  It's not the tech it's the data. As far as I know the data is not freely shared or at the very least there will be custom built models from data you can't get anywhere else.
- eek2121 3y ago
  Can confirm: The cloud computing fad is well underway to dying...that is why AI is booming. One need only to follow the wall street dollars to figure that one out. Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Cloud computing never was good in terms of value, however, it was only good in terms of scalability. AI solutions built on top of 'the cloud' will always be even worse. Note that I pay a certain "third tier" cloud provider less than $100/mo total for hosting large websites that would cost me more than 10 grand a month on AWS/Azure/Google...while having better uptime. (the biggest differences? the complete lack of IO and bandwidth charges, and much lower storage charges) That should tell you all you need about these types of bubbles, but then again, most of us that watched the entire tech field unfold since the pre-internet phase already knew this.
  - borg16 3y ago
    > Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Could you provide some examples?
    - garciasn 3y ago
      Dropbox has been the biggest name who did the whole “cloud repatriation” thing, which was all the rage at the beginning of 2023, with claims that the cloud was soon to be dead—but cloud revenues are supposedly expected to be in excess of $1T by 2026, so whatever. Some random survey from ESG found 60% of respondents repatriated at least some workloads. Who knows what the N was though. Source: https://www.sdxcentral.com/articles/thinking-about-cloud-repatriation-four-factors-to-consider/2023/07/ https://www.sdxcentral.com/articles/thinking-about-cloud-rep... Frankly, it makes sense that companies are deciding what works best and where. But the death knell of cloud providers is simply not happening.
  - SupremumLimit 3y ago
    Which cloud provider is it, if you don’t mind sharing?
    - 3y ago
- chii 3y ago
  But you also have to account for the counterfactuals. If the cloud had not existed, those that claim they saved money switching away from cloud might never have been in business in the first place.
agentcoops 3y ago
Certainly the undisputed winners will be the very few firms with enough engineering resources and GPUs to train their own models (not just fine-tune) where the models in question increase the productivity of workers in their non-ai-related profit centers. After that we have the real question of what the future will be of open source LLMs, on the one hand, and the question most relevant to this article of what sort and whether profitable “AI businesses” can be sustained over time. As Stratechery has analyzed, it is very possible that OpenAI turns out to be a very profitable B2C company with ad revenue in ChatGPT not concerned with their B2B sales or even the objective quality of their AI. Right now is an incredible time for AI: cheap Uber rides never qualitatively changed my life, but the current consumer access to AI models is truly incredible and I hope that only improves. However, even ignoring whatever happens on the regulatory front, I don’t think that is guaranteed at all.
- herval 3y ago
  This was definitely a theory that made people burn tons of money on the past couple of years, but I don’t think it holds water. These models are getting obsolete so fast, and there’s so many open ones, I doubt any one’s privately trained model can stay relevant for long
  - flir 3y ago
    The data is the moat. (If you can train your internally deployed LLM on data none of your competitors have, that's an advantage).
    - shon 3y ago
      In that case, X.AI, powered by X/Twitter/Tesla data and possibly Facebook (both closed, and somewhat hard to crawl inside) have the largest moat.
      - edmundsauto 3y ago
        I don’t think they necessarily will be allowed to train on their data unless they get explicit permission. They will try, but the way I see privacy revaluations is that users will have to authorize specific uses of their data and not be surprised by any application. This could be one of the more interesting privacy fights of the next decade. I’m sure there are easy cynical takes about how they will just shrink wrap the EULA, and maybe they will. But in a good privacy environment, users should never be surprised and have control over how their data is used. And I think we’ve made some progress there.
ChatGTP 3y ago
It seems to be compelling because a genie has been the promise of technological progress forever and let's be honest, it's been marketed as such. Why would people not invest in that unless you were a technological savvy skeptic like yourself.
0x008 3y ago
Because the cost of entry to the market is so absurdly high right now, it is seen as a good investment opportunity. If you throw enough money at it you can make your place no nice and early and then win later by sheer experience in the field. That is the idea.

msikora 3y ago

Yep. I wonder why there is not more features that would differentiate your LLM API, like for example the Functions in OpenAI LLM APIs. While not perfect it is extremely useful and not aware of a similar offering (I do know that there are Python packages offering similar functionality but it is my understanding that they don't work as well as the OpenAI Functions).

tomrod 3y ago

Aye, open models from Meta are tearing at the moat.

user3939382 3y ago

> The winner is going to be the consumers Cloud infra may be a comparable market, since computation is a big share of AI costs. Did consumers win big from competition between AWS, Azure, and GCP? Not sure. I see an uptick in write ups saying “We switched off cloud and reduced costs by 2/3rds.” Not a scientific sample but may leave the question open.

adam_arthur 3y ago
Much harder to switch cloud providers than to switch LLM models. How much time would it take most companies to move their product from AWS to GCP, for example? What if you use a cloud specific tool like DynamoDB? Margin is a function of stickiness/cost of switching (among other things). I suspect eventually we will enter a world where migrating cloud providers is mostly a click of a button, but we're a long ways off from that. Requires vendor agnostic and portable apis/containers/WASM runtimes everywhere. Swapping an LLM, at least in the current state, is about as close to updating a pointer as you can get.
- throwaway50182 3y ago
  Well I'd have to change my Terraform provider and the managed Kubernetes resource... Other than that, it'd be the same. So half an hour of coding + half an hour of reconfiguring CI secrets?
  - adam_arthur 3y ago
    Pretty obvious to most that switching cloud providers is not quick or painless for the majority of orgs. There's not really an argument in good faith to suggest otherwise. Especially given that many orgs use managed or cloud specific solutions that have no 1:1 mapping between vendors
    - throwaway50182 3y ago
      I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more.
      - atmosx 3y ago
        > I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more. As someone who worked on a similar project recently, I'm getting that idea that you obviously don't know what you're talking about.
        motoxpro 3y ago
        Didn't you know? It's only one DB ENV variable and you're swapped over from GCP to AWS. /s
        throwaway50182 3y ago
        I'm running the tech for a global startup with 100M EUR turnover. It's still pretty small, but it's something. You need to plan ahead, sure. Portability was one of my main concerns (including the possibility to go self-hosted). But it's definitely not impossible, nor too hard to do.
vikramkr 3y ago
There are a HUGE number of startups that would have had a massively harder time if they had to roll their own infra. Who cares if the companies eventually have to move off? (Though even Netflix seems ok with it overall for now). There are a ton of services that just wouldn't exist otherwise
- jokethrowaway 3y ago
  I disagree, at early startup scale you don't need much, you just buy a better VPS when you need to scale up Learning aws vs learning how to operate a vps are of comparable complexity It took me less then a day to setup infra for my startup (more than 10 years ago)
  - throwaway50182 3y ago
    If you're just running VPS on the big clouds, you're not going to get much advantage out of it, indeed. But tell me what third tier cheap provider has managed scale-to-zero-or-infinity functions? Managed storage with S3-like API? Where can I get an API gateway cheaper than Amazon? What about managed databases? These tools allow me to develop insanely scalable software incredibly easily. Agreed - all the big clouds are very expensive VPS hostings. Don't use it for that.
    - pleoxy 3y ago
      You get portability. Which the functions do not provide. Open source solutions have a longer career utility than proprietary offerings. I remember when NetWare certs were all the rage. Useless now. I remember msce. But if you learned open tools 35 years ago instead... You get the picture. You can scale from 1 thread 512MB RAM, to 500 threads and 12TB of RAM (off the shelf). Which is good enough for almost everyone who isn't planet scale. Auto scaling also comes with auto billing. Oops, your accidental infinite loop spawning functions has bankrupted your company. You don't have that risk starting with a VPS.
      - throwaway50182 3y ago
peteradio 3y ago
It's not the tech it's the data. As far as I know the data is not freely shared or at the very least there will be custom built models from data you can't get anywhere else.
eek2121 3y ago
Can confirm: The cloud computing fad is well underway to dying...that is why AI is booming. One need only to follow the wall street dollars to figure that one out. Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Cloud computing never was good in terms of value, however, it was only good in terms of scalability. AI solutions built on top of 'the cloud' will always be even worse. Note that I pay a certain "third tier" cloud provider less than $100/mo total for hosting large websites that would cost me more than 10 grand a month on AWS/Azure/Google...while having better uptime. (the biggest differences? the complete lack of IO and bandwidth charges, and much lower storage charges) That should tell you all you need about these types of bubbles, but then again, most of us that watched the entire tech field unfold since the pre-internet phase already knew this.
- borg16 3y ago
  > Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Could you provide some examples?
  - garciasn 3y ago
    Dropbox has been the biggest name who did the whole “cloud repatriation” thing, which was all the rage at the beginning of 2023, with claims that the cloud was soon to be dead—but cloud revenues are supposedly expected to be in excess of $1T by 2026, so whatever. Some random survey from ESG found 60% of respondents repatriated at least some workloads. Who knows what the N was though. Source: https://www.sdxcentral.com/articles/thinking-about-cloud-repatriation-four-factors-to-consider/2023/07/ https://www.sdxcentral.com/articles/thinking-about-cloud-rep... Frankly, it makes sense that companies are deciding what works best and where. But the death knell of cloud providers is simply not happening.
- SupremumLimit 3y ago
  Which cloud provider is it, if you don’t mind sharing?
  - 3y ago
chii 3y ago
But you also have to account for the counterfactuals. If the cloud had not existed, those that claim they saved money switching away from cloud might never have been in business in the first place.

agentcoops 3y ago

Certainly the undisputed winners will be the very few firms with enough engineering resources and GPUs to train their own models (not just fine-tune) where the models in question increase the productivity of workers in their non-ai-related profit centers. After that we have the real question of what the future will be of open source LLMs, on the one hand, and the question most relevant to this article of what sort and whether profitable “AI businesses” can be sustained over time. As Stratechery has analyzed, it is very possible that OpenAI turns out to be a very profitable B2C company with ad revenue in ChatGPT not concerned with their B2B sales or even the objective quality of their AI. Right now is an incredible time for AI: cheap Uber rides never qualitatively changed my life, but the current consumer access to AI models is truly incredible and I hope that only improves. However, even ignoring whatever happens on the regulatory front, I don’t think that is guaranteed at all.

herval 3y ago
This was definitely a theory that made people burn tons of money on the past couple of years, but I don’t think it holds water. These models are getting obsolete so fast, and there’s so many open ones, I doubt any one’s privately trained model can stay relevant for long
- flir 3y ago
  The data is the moat. (If you can train your internally deployed LLM on data none of your competitors have, that's an advantage).
  - shon 3y ago
    In that case, X.AI, powered by X/Twitter/Tesla data and possibly Facebook (both closed, and somewhat hard to crawl inside) have the largest moat.
    - edmundsauto 3y ago
      I don’t think they necessarily will be allowed to train on their data unless they get explicit permission. They will try, but the way I see privacy revaluations is that users will have to authorize specific uses of their data and not be surprised by any application. This could be one of the more interesting privacy fights of the next decade. I’m sure there are easy cynical takes about how they will just shrink wrap the EULA, and maybe they will. But in a good privacy environment, users should never be surprised and have control over how their data is used. And I think we’ve made some progress there.

ChatGTP 3y ago

It seems to be compelling because a genie has been the promise of technological progress forever and let's be honest, it's been marketed as such. Why would people not invest in that unless you were a technological savvy skeptic like yourself.

0x008 3y ago

Because the cost of entry to the market is so absurdly high right now, it is seen as a good investment opportunity. If you throw enough money at it you can make your place no nice and early and then win later by sheer experience in the field. That is the idea.