4 ms·

> The winner is going to be the consumers Cloud infra may be a comparable market, since computation is a big share of AI costs. Did consumers win big from comp

> The winner is going to be the consumers Cloud infra may be a comparable market, since computation is a big share of AI costs. Did consumers win big from competition between AWS, Azure, and GCP? Not sure. I see an uptick in write ups saying “We switched off cloud and reduced costs by 2/3rds.” Not a scientific sample but may leave the question open.

adam_arthur 3y ago
Much harder to switch cloud providers than to switch LLM models. How much time would it take most companies to move their product from AWS to GCP, for example? What if you use a cloud specific tool like DynamoDB? Margin is a function of stickiness/cost of switching (among other things). I suspect eventually we will enter a world where migrating cloud providers is mostly a click of a button, but we're a long ways off from that. Requires vendor agnostic and portable apis/containers/WASM runtimes everywhere. Swapping an LLM, at least in the current state, is about as close to updating a pointer as you can get.
- throwaway50182 3y ago
  Well I'd have to change my Terraform provider and the managed Kubernetes resource... Other than that, it'd be the same. So half an hour of coding + half an hour of reconfiguring CI secrets?
  - adam_arthur 3y ago
    Pretty obvious to most that switching cloud providers is not quick or painless for the majority of orgs. There's not really an argument in good faith to suggest otherwise. Especially given that many orgs use managed or cloud specific solutions that have no 1:1 mapping between vendors
    - throwaway50182 3y ago
      I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more.
      - atmosx 3y ago
        > I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more. As someone who worked on a similar project recently, I'm getting that idea that you obviously don't know what you're talking about.
        motoxpro 3y ago
        Didn't you know? It's only one DB ENV variable and you're swapped over from GCP to AWS. /s
        throwaway50182 3y ago
        I'm running the tech for a global startup with 100M EUR turnover. It's still pretty small, but it's something. You need to plan ahead, sure. Portability was one of my main concerns (including the possibility to go self-hosted). But it's definitely not impossible, nor too hard to do.
  - altdataseller 3y ago
    You lost me at Terraform and Kubernetes..
    - throwaway50182 3y ago
      What's a serious cloud provider-portable replacement for Terraform? And Kubernetes? It goes way beyond containers, you can replace many cloud provider-specific resources with Kubernetes resources. What alternative gives you that?
- intended 3y ago
  If you swap an LLM, at the very least, you have to run your entire Eval set again. This will almost certainly lead to a prompt rebuild, to better accommodate new model idiosyncrasies. If you are unlucky- your use case may be one where Evals require human review. Unless you are YOLOing it without evals. In that case this is relevant.
  - matusp 3y ago
    You need to rerun eval after each LLM update. GPTs have a new version every couple of months and their capabilities can change quite drastically pretty much randomly. Maybe they will make it more robust in time, but I think this is the feature of the technology and people will have to adapt to these quirks
    - intended 3y ago
      Yup. Constant Eval.
vikramkr 3y ago
There are a HUGE number of startups that would have had a massively harder time if they had to roll their own infra. Who cares if the companies eventually have to move off? (Though even Netflix seems ok with it overall for now). There are a ton of services that just wouldn't exist otherwise
- jokethrowaway 3y ago
  I disagree, at early startup scale you don't need much, you just buy a better VPS when you need to scale up Learning aws vs learning how to operate a vps are of comparable complexity It took me less then a day to setup infra for my startup (more than 10 years ago)
  - throwaway50182 3y ago
    If you're just running VPS on the big clouds, you're not going to get much advantage out of it, indeed. But tell me what third tier cheap provider has managed scale-to-zero-or-infinity functions? Managed storage with S3-like API? Where can I get an API gateway cheaper than Amazon? What about managed databases? These tools allow me to develop insanely scalable software incredibly easily. Agreed - all the big clouds are very expensive VPS hostings. Don't use it for that.
    - pleoxy 3y ago
      You get portability. Which the functions do not provide. Open source solutions have a longer career utility than proprietary offerings. I remember when NetWare certs were all the rage. Useless now. I remember msce. But if you learned open tools 35 years ago instead... You get the picture. You can scale from 1 thread 512MB RAM, to 500 threads and 12TB of RAM (off the shelf). Which is good enough for almost everyone who isn't planet scale. Auto scaling also comes with auto billing. Oops, your accidental infinite loop spawning functions has bankrupted your company. You don't have that risk starting with a VPS.
      - throwaway50182 3y ago
peteradio 3y ago
It's not the tech it's the data. As far as I know the data is not freely shared or at the very least there will be custom built models from data you can't get anywhere else.
eek2121 3y ago
Can confirm: The cloud computing fad is well underway to dying...that is why AI is booming. One need only to follow the wall street dollars to figure that one out. Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Cloud computing never was good in terms of value, however, it was only good in terms of scalability. AI solutions built on top of 'the cloud' will always be even worse. Note that I pay a certain "third tier" cloud provider less than $100/mo total for hosting large websites that would cost me more than 10 grand a month on AWS/Azure/Google...while having better uptime. (the biggest differences? the complete lack of IO and bandwidth charges, and much lower storage charges) That should tell you all you need about these types of bubbles, but then again, most of us that watched the entire tech field unfold since the pre-internet phase already knew this.
- borg16 3y ago
  > Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Could you provide some examples?
  - garciasn 3y ago
    Dropbox has been the biggest name who did the whole “cloud repatriation” thing, which was all the rage at the beginning of 2023, with claims that the cloud was soon to be dead—but cloud revenues are supposedly expected to be in excess of $1T by 2026, so whatever. Some random survey from ESG found 60% of respondents repatriated at least some workloads. Who knows what the N was though. Source: https://www.sdxcentral.com/articles/thinking-about-cloud-repatriation-four-factors-to-consider/2023/07/ https://www.sdxcentral.com/articles/thinking-about-cloud-rep... Frankly, it makes sense that companies are deciding what works best and where. But the death knell of cloud providers is simply not happening.
- SupremumLimit 3y ago
  Which cloud provider is it, if you don’t mind sharing?
  - 3y ago
chii 3y ago
But you also have to account for the counterfactuals. If the cloud had not existed, those that claim they saved money switching away from cloud might never have been in business in the first place.

adam_arthur 3y ago

Much harder to switch cloud providers than to switch LLM models. How much time would it take most companies to move their product from AWS to GCP, for example? What if you use a cloud specific tool like DynamoDB? Margin is a function of stickiness/cost of switching (among other things). I suspect eventually we will enter a world where migrating cloud providers is mostly a click of a button, but we're a long ways off from that. Requires vendor agnostic and portable apis/containers/WASM runtimes everywhere. Swapping an LLM, at least in the current state, is about as close to updating a pointer as you can get.

throwaway50182 3y ago
Well I'd have to change my Terraform provider and the managed Kubernetes resource... Other than that, it'd be the same. So half an hour of coding + half an hour of reconfiguring CI secrets?
- adam_arthur 3y ago
  Pretty obvious to most that switching cloud providers is not quick or painless for the majority of orgs. There's not really an argument in good faith to suggest otherwise. Especially given that many orgs use managed or cloud specific solutions that have no 1:1 mapping between vendors
  - throwaway50182 3y ago
    I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more.
    - atmosx 3y ago
      > I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more. As someone who worked on a similar project recently, I'm getting that idea that you obviously don't know what you're talking about.
      - motoxpro 3y ago
        Didn't you know? It's only one DB ENV variable and you're swapped over from GCP to AWS. /s
      - throwaway50182 3y ago
        I'm running the tech for a global startup with 100M EUR turnover. It's still pretty small, but it's something. You need to plan ahead, sure. Portability was one of my main concerns (including the possibility to go self-hosted). But it's definitely not impossible, nor too hard to do.
- altdataseller 3y ago
  You lost me at Terraform and Kubernetes..
  - throwaway50182 3y ago
    What's a serious cloud provider-portable replacement for Terraform? And Kubernetes? It goes way beyond containers, you can replace many cloud provider-specific resources with Kubernetes resources. What alternative gives you that?
intended 3y ago
If you swap an LLM, at the very least, you have to run your entire Eval set again. This will almost certainly lead to a prompt rebuild, to better accommodate new model idiosyncrasies. If you are unlucky- your use case may be one where Evals require human review. Unless you are YOLOing it without evals. In that case this is relevant.
- matusp 3y ago
  You need to rerun eval after each LLM update. GPTs have a new version every couple of months and their capabilities can change quite drastically pretty much randomly. Maybe they will make it more robust in time, but I think this is the feature of the technology and people will have to adapt to these quirks
  - intended 3y ago
    Yup. Constant Eval.

vikramkr 3y ago

There are a HUGE number of startups that would have had a massively harder time if they had to roll their own infra. Who cares if the companies eventually have to move off? (Though even Netflix seems ok with it overall for now). There are a ton of services that just wouldn't exist otherwise

jokethrowaway 3y ago
I disagree, at early startup scale you don't need much, you just buy a better VPS when you need to scale up Learning aws vs learning how to operate a vps are of comparable complexity It took me less then a day to setup infra for my startup (more than 10 years ago)
- throwaway50182 3y ago
  If you're just running VPS on the big clouds, you're not going to get much advantage out of it, indeed. But tell me what third tier cheap provider has managed scale-to-zero-or-infinity functions? Managed storage with S3-like API? Where can I get an API gateway cheaper than Amazon? What about managed databases? These tools allow me to develop insanely scalable software incredibly easily. Agreed - all the big clouds are very expensive VPS hostings. Don't use it for that.
  - pleoxy 3y ago
    You get portability. Which the functions do not provide. Open source solutions have a longer career utility than proprietary offerings. I remember when NetWare certs were all the rage. Useless now. I remember msce. But if you learned open tools 35 years ago instead... You get the picture. You can scale from 1 thread 512MB RAM, to 500 threads and 12TB of RAM (off the shelf). Which is good enough for almost everyone who isn't planet scale. Auto scaling also comes with auto billing. Oops, your accidental infinite loop spawning functions has bankrupted your company. You don't have that risk starting with a VPS.
    - throwaway50182 3y ago

peteradio 3y ago

It's not the tech it's the data. As far as I know the data is not freely shared or at the very least there will be custom built models from data you can't get anywhere else.

eek2121 3y ago

Can confirm: The cloud computing fad is well underway to dying...that is why AI is booming. One need only to follow the wall street dollars to figure that one out. Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Cloud computing never was good in terms of value, however, it was only good in terms of scalability. AI solutions built on top of 'the cloud' will always be even worse. Note that I pay a certain "third tier" cloud provider less than $100/mo total for hosting large websites that would cost me more than 10 grand a month on AWS/Azure/Google...while having better uptime. (the biggest differences? the complete lack of IO and bandwidth charges, and much lower storage charges) That should tell you all you need about these types of bubbles, but then again, most of us that watched the entire tech field unfold since the pre-internet phase already knew this.

borg16 3y ago
> Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Could you provide some examples?
- garciasn 3y ago
  Dropbox has been the biggest name who did the whole “cloud repatriation” thing, which was all the rage at the beginning of 2023, with claims that the cloud was soon to be dead—but cloud revenues are supposedly expected to be in excess of $1T by 2026, so whatever. Some random survey from ESG found 60% of respondents repatriated at least some workloads. Who knows what the N was though. Source: https://www.sdxcentral.com/articles/thinking-about-cloud-repatriation-four-factors-to-consider/2023/07/ https://www.sdxcentral.com/articles/thinking-about-cloud-rep... Frankly, it makes sense that companies are deciding what works best and where. But the death knell of cloud providers is simply not happening.
SupremumLimit 3y ago
Which cloud provider is it, if you don’t mind sharing?
- 3y ago

chii 3y ago

But you also have to account for the counterfactuals. If the cloud had not existed, those that claim they saved money switching away from cloud might never have been in business in the first place.