4 ms·

GPUs are better run in your own office. All cloud providers charge much, much more for GPUs than if you run a local machine. Cloud GPUs are also a lot slower

GPUs are better run in your own office. All cloud providers charge much, much more for GPUs than if you run a local machine. Cloud GPUs are also a lot slower than state of the art consumer GPUs. Cloud GPUs: much slower, less available, much more expensive.

bushbaba 4y ago
Say you want 100 GPUs all inter connected to your multi petabyte data lake that’s being fed by your production workload. Sure you could buy all that equipment but I’d wager it’s cheaper, more agile, and greater velocity from it being in the cloud
- cosmic_quanta 4y ago
  I would argue that the cost profile is different. Local GPUs are a big up-front cost. But assuming that your workload is stable, in the long run I think local GPUs ends up being cheaper per-hour than cloud. For startups, it doesn't make sense to make the up-front purchase, fine. But if you're optimizing for long-term (amortized) costs, I'd be curious if cloud is cost-effective.
  - macrolime 4y ago
    The long run being three months
    - bushbaba 4y ago
      An nvidia dgx box is roughly 40k. And that’s not including power/storage/rack Space. But yes, If a single workstation can meet your gpu training needs then it’ll be cheaper with sufficient usage
pclmulqdq 4y ago
This is generally true for all accelerators (I work with cloud and on-prem FPGAs for my startup, Arbitrand). However, lots of people only need those accelerators once in a while, so time sharing (aka cloud computing) makes a lot of sense and saves a ton of money overall. For FPGAs and some compute GPU applications, not having to handle support for your accelerators is also nice.
discordance 4y ago
For small orgs this makes sense, but this really depends on how big your data sets are that you're training against and how your ML Ops / Data Ops is set up. GPUs are better run close to your data. If you're training on-prem then your data needs to be on-prem too.

bushbaba 4y ago

Say you want 100 GPUs all inter connected to your multi petabyte data lake that’s being fed by your production workload. Sure you could buy all that equipment but I’d wager it’s cheaper, more agile, and greater velocity from it being in the cloud

cosmic_quanta 4y ago
I would argue that the cost profile is different. Local GPUs are a big up-front cost. But assuming that your workload is stable, in the long run I think local GPUs ends up being cheaper per-hour than cloud. For startups, it doesn't make sense to make the up-front purchase, fine. But if you're optimizing for long-term (amortized) costs, I'd be curious if cloud is cost-effective.
- macrolime 4y ago
  The long run being three months
  - bushbaba 4y ago
    An nvidia dgx box is roughly 40k. And that’s not including power/storage/rack Space. But yes, If a single workstation can meet your gpu training needs then it’ll be cheaper with sufficient usage

pclmulqdq 4y ago

This is generally true for all accelerators (I work with cloud and on-prem FPGAs for my startup, Arbitrand). However, lots of people only need those accelerators once in a while, so time sharing (aka cloud computing) makes a lot of sense and saves a ton of money overall. For FPGAs and some compute GPU applications, not having to handle support for your accelerators is also nice.

discordance 4y ago

For small orgs this makes sense, but this really depends on how big your data sets are that you're training against and how your ML Ops / Data Ops is set up. GPUs are better run close to your data. If you're training on-prem then your data needs to be on-prem too.