9 ms·
The cloud is staggeringly more expensive than building your own workstation. Like a couple of weeks of on-demand GPU instances on AWS Virginia costs about as m
by CoolGuySteve 7y ago
The cloud is staggeringly more expensive than building your own workstation. Like a couple of weeks of on-demand GPU instances on AWS Virginia costs about as much as a machine with 2 RTX 2080 cards in it that will stay performant for at least a couple years.
But coming back to the Mac Pro, unless you need Apple-specific software, the same is true compared to Ryzen/ThreadRipper + NVidia cards. The Mac costs 2-3x as much while delivering less performance.
At least the cloud can scale horizontally, the Mac Pro is just slow and inefficient.
- GenerocUsername 7y agoWhy is that? Shouldn't competition drive cloud prices closer to ownership prices?
- gbear605 7y agoThere’s a high cost of entry (onboarding lots of customers, building data centers, writing the management software, etc.), so there isn’t much competition. The market just isn’t efficient in this case.
- magashna 7y agoI think it does for stuff like storage, but transactional and heavy data processing are still cheaper to do on-prem from what I've seen.
- sseveran 7y agoIts because NVIDIA requires Teslas in the cloud, which while more powerful than 2080 ti's, cost roughly 6-8x as much.
- Analemma_ 7y agoHow does that work? If I buy a 2080 Ti and then decide to use it in an internet-hosted box that I rent out to other people, the first-sale doctrine should tell Nvidia to piss off if they have a problem with it.
- ptomato 7y agoBecause nvidia's driver licensing terms require you not to be using the consumer stuff in a datacenter.
- lliamander 7y agoWhat if you're running Linux with the nouveau driver?
- sseveran 7y agoIt's irrelevant. You need the CUDA SDK and the CuDNN SDK both of which have license agreements. Edit: You may also need the official Nvidia driver. I have never run anything with the OSS drivers.
- lliamander 7y agoGotcha
- angry_octet 7y agoWhen you are operating at any kind of scale you need the Telsa packaging (airflow, connectors, power) and testing (running many in a system). The consumer desktop cards cause lots of trouble. Also the consumer cards have artificially small memory sizes (e.g. 11GB max) which painfully constrains DL jobs. No manufacturer is allowed to build Tesla-like cards. Theoretically AMD could crush the nVidia profit margins by releasing cheap data center boards, but their developer support is rubbish and they want that high margin cash too.
- fit2rule 7y ago.. what this means, is that Apple is competing against the Titan/NVIDIA hardware designers to make better compute power available at scale, and in a way which makes sense across consumer/pro boundaries. In that context, the Mac Pro doesn't sound too bad a proposition. I say that as someone who recently built a dual-Titan/AMD Ryzen system, and while the pain of the build is almost gone away .. I do lust after that sexy Apple box, being plug 'n play and all ..
- angry_octet 7y agoThe average corporate IT department is rubbish at doing computing. Easier and quicker to outsource than doing it in house. But nVidia sell GPU appliances (DGX) for these on-prem customers. Basically, expertise is expensive and comes in $200k/year increments. You can use a lot of cloud GPU for that much money.
- PaulHoule 7y agoIf you are doing serious training with on-demand instances you are doing it wrong. If your training takes more than an hour, you really need to checkpoint it periodically. If you have periodic checkpoints you can use spot instances and pay 10% or less of what on-demand would cost.
- CoolGuySteve 7y ago1) Even with checkpoints and spot instances, the initial REPL development process needs to happen somewhere. 2) Where are you seeing 10%? More realistically spot instances are 25-30% of the cost. And if so fine, 6 weeks of machine time == dedicated workstation.
- phendrenad2 7y agoYeah, one workstation. If you need to build out 1,000x as many GPUs, the cloud becomes more economical. And that's where the money is, in large customers, not someone who needs 1 or 2 cloud GPUs.
- m463 7y agoYou forget that using the cloud you can spin up 9 ladies to have a baby in a month, or 270 ladies to have a baby in a day.