4 ms·

This is not as rare as public clouds may lead people to believe. I have had to move workloads around since AWS began (even between public clouds on occasion).

This is not as rare as public clouds may lead people to believe. I have had to move workloads around since AWS began (even between public clouds on occasion). In particular, GPU availability has been a continuing problem. Unlike interchangeable x64 / arm64 instances with some adjustments based on the new core and ram count... if no GPU instances are available then I simply cannot run the job. AMD's improved support has increasingly provided an alternative in some situations but the problem persists. I recommend doing the work to make the business somewhat cloud agnostic, or at the very least multi-region capable. I realize this is not an option for some services that have no equivalent on other clouds but you mentioned databases and k8s clusters which are both supported elsewhere.

andrewstuart 4y ago
GPUs are better run in your own office. All cloud providers charge much, much more for GPUs than if you run a local machine. Cloud GPUs are also a lot slower than state of the art consumer GPUs. Cloud GPUs: much slower, less available, much more expensive.
- bushbaba 4y ago
  Say you want 100 GPUs all inter connected to your multi petabyte data lake that’s being fed by your production workload. Sure you could buy all that equipment but I’d wager it’s cheaper, more agile, and greater velocity from it being in the cloud
  - cosmic_quanta 4y ago
    I would argue that the cost profile is different. Local GPUs are a big up-front cost. But assuming that your workload is stable, in the long run I think local GPUs ends up being cheaper per-hour than cloud. For startups, it doesn't make sense to make the up-front purchase, fine. But if you're optimizing for long-term (amortized) costs, I'd be curious if cloud is cost-effective.
    - macrolime 4y ago
      The long run being three months
      - bushbaba 4y ago
        An nvidia dgx box is roughly 40k. And that’s not including power/storage/rack Space. But yes, If a single workstation can meet your gpu training needs then it’ll be cheaper with sufficient usage
- pclmulqdq 4y ago
  This is generally true for all accelerators (I work with cloud and on-prem FPGAs for my startup, Arbitrand). However, lots of people only need those accelerators once in a while, so time sharing (aka cloud computing) makes a lot of sense and saves a ton of money overall. For FPGAs and some compute GPU applications, not having to handle support for your accelerators is also nice.
- discordance 4y ago
  For small orgs this makes sense, but this really depends on how big your data sets are that you're training against and how your ML Ops / Data Ops is set up. GPUs are better run close to your data. If you're training on-prem then your data needs to be on-prem too.
dehrmann 4y ago
You want to be in a position where you can spin up in a nearby region and pretend it's local and have things be good enough for a while. Properly building out multi-region is hard, and multi-cloud isn't worth it because it improves how you handle rare events (where half the internet is already down) with ongoing operational toil.

andrewstuart 4y ago

GPUs are better run in your own office. All cloud providers charge much, much more for GPUs than if you run a local machine. Cloud GPUs are also a lot slower than state of the art consumer GPUs. Cloud GPUs: much slower, less available, much more expensive.

bushbaba 4y ago
Say you want 100 GPUs all inter connected to your multi petabyte data lake that’s being fed by your production workload. Sure you could buy all that equipment but I’d wager it’s cheaper, more agile, and greater velocity from it being in the cloud
- cosmic_quanta 4y ago
  I would argue that the cost profile is different. Local GPUs are a big up-front cost. But assuming that your workload is stable, in the long run I think local GPUs ends up being cheaper per-hour than cloud. For startups, it doesn't make sense to make the up-front purchase, fine. But if you're optimizing for long-term (amortized) costs, I'd be curious if cloud is cost-effective.
  - macrolime 4y ago
    The long run being three months
    - bushbaba 4y ago
      An nvidia dgx box is roughly 40k. And that’s not including power/storage/rack Space. But yes, If a single workstation can meet your gpu training needs then it’ll be cheaper with sufficient usage
pclmulqdq 4y ago
This is generally true for all accelerators (I work with cloud and on-prem FPGAs for my startup, Arbitrand). However, lots of people only need those accelerators once in a while, so time sharing (aka cloud computing) makes a lot of sense and saves a ton of money overall. For FPGAs and some compute GPU applications, not having to handle support for your accelerators is also nice.
discordance 4y ago
For small orgs this makes sense, but this really depends on how big your data sets are that you're training against and how your ML Ops / Data Ops is set up. GPUs are better run close to your data. If you're training on-prem then your data needs to be on-prem too.

dehrmann 4y ago

You want to be in a position where you can spin up in a nearby region and pretend it's local and have things be good enough for a while. Properly building out multi-region is hard, and multi-cloud isn't worth it because it improves how you handle rare events (where half the internet is already down) with ongoing operational toil.