3 ms·

Sorry for the tangential post here but the cost difference between hosting the GPUs and using AWS is jarring to me. The whole value proposition is to do things

Sorry for the tangential post here but the cost difference between hosting the GPUs and using AWS is jarring to me. The whole value proposition is to do things cheaper. Now I can’t understand how this can be correct anymore. Basit ever been the case at all?

nknealk 5y ago
If I recall correctly Nvidia does not allow you to put consumer cards into data centers. They force you to buy a different line of cards which are like 10x more expensive. That may be a driver of the AWS price. Update — I do recall correctly. Source: https://www.digitaltrends.com/computing/nvidia-bans-consumer-gpus-in-data-centers/ https://www.digitaltrends.com/computing/nvidia-bans-consumer...
kube-system 5y ago
One of the reasons the cost looks so radically different is also because it is not comparing the same things. AWS is not just GPU + electricity. It's GPU + a server + a rack + electricity + network + a physical data center + real estate + labor + reserve capacity + automation, etc...
dotnet00 5y ago
AWS GPUs (or ones offered by any of the big cloud providers for that matter) are not really all that cost effective. They also have to use datacenter GPUs, which are a lot more expensive than consumer ones. For cost effectiveness, one route (mostly in the ML community) is to rent people's machines via services like vast.ai, who can get around NVIDIA preventing use of consumer cards in data centers by making it so they're only selling/buying time on people's computers. These end up being a lot more affordable (IIRC ~$20/day for a 2x 3090 system) but in exchange have the unreliability that comes with renting time on someone else's computer, like tasks randomly getting killed (which is pretty frustrating to wake up to, as you get charged for time rather than usage), some people having poor upload/download speeds or other gpu performance issues.
Sohcahtoa82 5y ago
> The whole value proposition is to do things cheaper. I don't think this was ever the case, at least, not in the short term or at smaller scales. I always thought the value proposition for AWS was the fact that you could scale up and down nearly instantly and that you didn't have to run your own data center, which was a monster of an expense. AWS made it so a single person could be a startup. With just a couple clicks, you've got a server running, and you only pay for the time you use. Now, maybe you're asking about GPUs specifically...in that case, yeah, I don't know why people would use AWS unless they really only need a couple hours of GPU time per month or something. GPU instances are so expensive that the ROI for buying them yourself hits pretty quickly. You have to REALLY need the rest of the AWS infrastructure to be directly connected to your GPU to make it worthwhile.

nknealk 5y ago

If I recall correctly Nvidia does not allow you to put consumer cards into data centers. They force you to buy a different line of cards which are like 10x more expensive. That may be a driver of the AWS price. Update — I do recall correctly. Source: https://www.digitaltrends.com/computing/nvidia-bans-consumer-gpus-in-data-centers/ https://www.digitaltrends.com/computing/nvidia-bans-consumer...

kube-system 5y ago

One of the reasons the cost looks so radically different is also because it is not comparing the same things. AWS is not just GPU + electricity. It's GPU + a server + a rack + electricity + network + a physical data center + real estate + labor + reserve capacity + automation, etc...

dotnet00 5y ago

AWS GPUs (or ones offered by any of the big cloud providers for that matter) are not really all that cost effective. They also have to use datacenter GPUs, which are a lot more expensive than consumer ones. For cost effectiveness, one route (mostly in the ML community) is to rent people's machines via services like vast.ai, who can get around NVIDIA preventing use of consumer cards in data centers by making it so they're only selling/buying time on people's computers. These end up being a lot more affordable (IIRC ~$20/day for a 2x 3090 system) but in exchange have the unreliability that comes with renting time on someone else's computer, like tasks randomly getting killed (which is pretty frustrating to wake up to, as you get charged for time rather than usage), some people having poor upload/download speeds or other gpu performance issues.

Sohcahtoa82 5y ago

> The whole value proposition is to do things cheaper. I don't think this was ever the case, at least, not in the short term or at smaller scales. I always thought the value proposition for AWS was the fact that you could scale up and down nearly instantly and that you didn't have to run your own data center, which was a monster of an expense. AWS made it so a single person could be a startup. With just a couple clicks, you've got a server running, and you only pay for the time you use. Now, maybe you're asking about GPUs specifically...in that case, yeah, I don't know why people would use AWS unless they really only need a couple hours of GPU time per month or something. GPU instances are so expensive that the ROI for buying them yourself hits pretty quickly. You have to REALLY need the rest of the AWS infrastructure to be directly connected to your GPU to make it worthwhile.