4 ms·

And why Pascal based Titan? Is it the best investment in terms of performance per $ spent? Also, how cost effective is it to use cloud GPUs for real world mach

And why Pascal based Titan? Is it the best investment in terms of performance per $ spent? Also, how cost effective is it to use cloud GPUs for real world machine learning?

general_ai 10y ago
Cloud GPUs are not economical if you use them 24x7x365 (which for any serious deep learning researcher or engineer is usually the case). The only scenario I can think of in which they'd be more economical than something under your desk is when you need to run a massive and embarrassingly parallel workload. I.e. try training dozens of models at the same time with different hyperparameters, and run that for a few days. You could do it cheaper, but it would take a long time and it would be a massive pain in the ass, so you pay the pretty penny and get it done in a week. For my needs I have a machine with a 2011-v3 socket, and four GTX1080 GPUs. Warms up my man cave pretty nicely in winter. I also have access to about a hundred GPUs (older Titans, Teslas, newer 1080s and Pascal Titans) at work that I share with others. Now, regarding Titan. Titan is actually not that much faster than GTX1080, so in terms of raw speed there's no reason to pay twice as much. BUT, it has 4GB more RAM, which lets you run larger models. NVIDIA rightly decided that for a $100+/hr deep learning researcher $600 is not going to be that big of a deal, and priced the card accordingly. If your models fit into 8GB, you'll be better off buying two 1080's instead. As to me, I'm thinking of replacing at least one of my 1080s with a Titan, to be able to train larger models. On a purely TFLOP or even TFLOP/watt basis, it doesn't currently make sense to buy anything that doesn't run Pascal.
modeless 10y ago
Titan XP is the maximum single chip performance you can buy right now. $1200 is well worth it if ML is part of your career. The time saved will pay for it.
nl 10y ago
Cloud GPUs are cost effective if you need to either fine-tune a pretrained network (eg, use pretrained ResNet/VGG/AlexNet for custom classes, ie[1]) or for inference, or if you don't want the upfront costs. A 4GB GTX1050 is ~$180. A p2 instance on Amazon is $0.9/hour. The cost effectiveness depends on if you have a PC already. [1] https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html https://blog.keras.io/building-powerful-image-classification...
- dTal 10y ago
  And the cost of your electricity, don't forget.
  - robotresearcher 10y ago
    Current mean cost to US domestic consumers of $0.12 kW/h, says Google. TPD Specs: GTX 1050 75W GTX 1080 180W Titan 250W Triple these numbers for the overall machine's PSU spec. So even the Titan costs $0.03 per hour to run, or maybe $0.10 if the rest of the machine is flat-out. edit: $0.19 kWh for San Francisco residents. Domestic rates.

And why Pascal based Titan? Is it the best investment in terms of performance per $ spent? Also, how cost effective is it to use cloud GPUs for real world machine learning?

general_ai 10y ago

Cloud GPUs are not economical if you use them 24x7x365 (which for any serious deep learning researcher or engineer is usually the case). The only scenario I can think of in which they'd be more economical than something under your desk is when you need to run a massive and embarrassingly parallel workload. I.e. try training dozens of models at the same time with different hyperparameters, and run that for a few days. You could do it cheaper, but it would take a long time and it would be a massive pain in the ass, so you pay the pretty penny and get it done in a week. For my needs I have a machine with a 2011-v3 socket, and four GTX1080 GPUs. Warms up my man cave pretty nicely in winter. I also have access to about a hundred GPUs (older Titans, Teslas, newer 1080s and Pascal Titans) at work that I share with others. Now, regarding Titan. Titan is actually not that much faster than GTX1080, so in terms of raw speed there's no reason to pay twice as much. BUT, it has 4GB more RAM, which lets you run larger models. NVIDIA rightly decided that for a $100+/hr deep learning researcher $600 is not going to be that big of a deal, and priced the card accordingly. If your models fit into 8GB, you'll be better off buying two 1080's instead. As to me, I'm thinking of replacing at least one of my 1080s with a Titan, to be able to train larger models. On a purely TFLOP or even TFLOP/watt basis, it doesn't currently make sense to buy anything that doesn't run Pascal.

modeless 10y ago

Titan XP is the maximum single chip performance you can buy right now. $1200 is well worth it if ML is part of your career. The time saved will pay for it.

nl 10y ago

Cloud GPUs are cost effective if you need to either fine-tune a pretrained network (eg, use pretrained ResNet/VGG/AlexNet for custom classes, ie[1]) or for inference, or if you don't want the upfront costs. A 4GB GTX1050 is ~$180. A p2 instance on Amazon is $0.9/hour. The cost effectiveness depends on if you have a PC already. [1] https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html https://blog.keras.io/building-powerful-image-classification...

dTal 10y ago
And the cost of your electricity, don't forget.
- robotresearcher 10y ago
  Current mean cost to US domestic consumers of $0.12 kW/h, says Google. TPD Specs: GTX 1050 75W GTX 1080 180W Titan 250W Triple these numbers for the overall machine's PSU spec. So even the Titan costs $0.03 per hour to run, or maybe $0.10 if the rest of the machine is flat-out. edit: $0.19 kWh for San Francisco residents. Domestic rates.