4 ms·

Thanks. I was glossing over the details. The point I was trying to emphasize is that the Azure instance gives you the whole K80 board (i.e. two GK210 processors

by n00b101 10y ago

Thanks. I was glossing over the details. The point I was trying to emphasize is that the Azure instance gives you the whole K80 board (i.e. two GK210 processors), unlike the AWS G2 instance which effectively gives you a virtualized half of a K10 board (i.e. one GK104 processor)

dogma1138 10y ago
Oh yeah the Kepler 2 K80's blow the K10 out of the water. I wonder just how expensive the enterprise grade Pascal boards are that even Microsoft doesn't offer them as an option. It's really amazing just how much money NVIDIA seems to be squeezing out of the compute market (almost makes you feel bad for AMD) especially considering that Kepler is a 4 almost 5 year old architecture by now. The K80 is still going for like 4000-5000$ a pop I would love to know just how many GPU's Microsoft ended up buying.
- dharma1 10y ago
  Is there any reason cloud providers don't offer virtualised Titan X, or GTX 1080?
  - dogma1138 10y ago
    "Gaming" GPU's have reduced compute capabilities (pretty horrible DP/HP compute performance compared to Tesla parts), and their SKU's are not really designed for enterprise integration (power requirements, thermals etc.). As far as "virtualization" goes NVIDIA actually restricts that in the driver, including PCIE passthrough especially to not allow cheaper desktop parts being used for low end computing as the margins per GPU are considerably higher with their enterprise parts.
    - dharma1 10y ago
      ""Gaming" GPU's have reduced compute capabilities" Not so for machine learning, at least in my experience. http://pjreddie.com/darknet/hardware-guide/ http://pjreddie.com/darknet/hardware-guide/ Passthrough: https://www.evonide.com/non-root-gpu-passthrough-setup/ https://www.evonide.com/non-root-gpu-passthrough-setup/
      - dogma1138 10y ago
        You missed the end of the sentence: "....pretty horrible DP/HP compute performance compared to Tesla parts." Might not be applicable in your case, but NVIDIA hasn't released a desktop part with FP64 capabilities even remotely comparable to their Tesla's since the original Titan. Both Titan X's (Maxwell and Pascal) and the [Insert Physicist's Name Here] Quadros are pretty gimped as far as it comes to HP/DP operations. If anything between generations DP performance across desktop parts have been severely degraded from 3:1 for Kepler to 16:1 for Maxwell to bloody 32:1 for Pascal desktop GPU's. As for the passthrough it is possible but it requires a few hacks and is technically in violations of NVIDIA's policies, and unlike the Tesla parts there is no full virtualization once the a device has been initialized it can only be assigned to 1 host/guest and a full host reboot is required to reinitialize the device again. You also need to do a few UEFI hacks to prevent the UEFI from initializing the GPU before your host OS loads and it can be passed through to the guest. This isn't something that cloud providers would dealing with, NVIDIA will not sell them anything but Tesla/Grid parts and buying from AIB's/OEM's and hacking your way through it isn't an option, you'll get zero support from NVIDIA, you could not thin provision your GPU's and you'll need to do a full host reboot every time you want to reassign the GPU or try to figure out if you can write a custom BIOS for your card that would somehow allow you to reinitialize it. P.S. I've actually built a KVM multi-user gaming rig with NVIDIA cards, I would not recommend anyone to try that, AMD cards are more pleasant to work with, too bad the performance are utter garbage for high end gaming.
        pjietr 10y ago
        Agree with dharma1 that the FP32-based cloud services could be a lot useful for deep learning than the Tesla-based Clouds. Baidu has actually built Titan X -based servers for themselves as they saw it as the most cost-efficient way to build the system: http://www.nextplatform.com/2016/04/22/baidu-eyes-deep-learning-strategy-wake-new-gpu-options/ http://www.nextplatform.com/2016/04/22/baidu-eyes-deep-learn... Same logic would probably work in "deep learning" clouds if there would not be the licensing issue. And this conversation got immortalized to that slideshow as well for selecting hardware parts for deep learning systems: http://tinyurl.com/zzhj9vr http://tinyurl.com/zzhj9vr

by n00b101 10y ago

dogma1138 10y ago

Oh yeah the Kepler 2 K80's blow the K10 out of the water. I wonder just how expensive the enterprise grade Pascal boards are that even Microsoft doesn't offer them as an option. It's really amazing just how much money NVIDIA seems to be squeezing out of the compute market (almost makes you feel bad for AMD) especially considering that Kepler is a 4 almost 5 year old architecture by now. The K80 is still going for like 4000-5000$ a pop I would love to know just how many GPU's Microsoft ended up buying.

dharma1 10y ago
Is there any reason cloud providers don't offer virtualised Titan X, or GTX 1080?
- dogma1138 10y ago
  "Gaming" GPU's have reduced compute capabilities (pretty horrible DP/HP compute performance compared to Tesla parts), and their SKU's are not really designed for enterprise integration (power requirements, thermals etc.). As far as "virtualization" goes NVIDIA actually restricts that in the driver, including PCIE passthrough especially to not allow cheaper desktop parts being used for low end computing as the margins per GPU are considerably higher with their enterprise parts.
  - dharma1 10y ago
    ""Gaming" GPU's have reduced compute capabilities" Not so for machine learning, at least in my experience. http://pjreddie.com/darknet/hardware-guide/ http://pjreddie.com/darknet/hardware-guide/ Passthrough: https://www.evonide.com/non-root-gpu-passthrough-setup/ https://www.evonide.com/non-root-gpu-passthrough-setup/
    - dogma1138 10y ago
      You missed the end of the sentence: "....pretty horrible DP/HP compute performance compared to Tesla parts." Might not be applicable in your case, but NVIDIA hasn't released a desktop part with FP64 capabilities even remotely comparable to their Tesla's since the original Titan. Both Titan X's (Maxwell and Pascal) and the [Insert Physicist's Name Here] Quadros are pretty gimped as far as it comes to HP/DP operations. If anything between generations DP performance across desktop parts have been severely degraded from 3:1 for Kepler to 16:1 for Maxwell to bloody 32:1 for Pascal desktop GPU's. As for the passthrough it is possible but it requires a few hacks and is technically in violations of NVIDIA's policies, and unlike the Tesla parts there is no full virtualization once the a device has been initialized it can only be assigned to 1 host/guest and a full host reboot is required to reinitialize the device again. You also need to do a few UEFI hacks to prevent the UEFI from initializing the GPU before your host OS loads and it can be passed through to the guest. This isn't something that cloud providers would dealing with, NVIDIA will not sell them anything but Tesla/Grid parts and buying from AIB's/OEM's and hacking your way through it isn't an option, you'll get zero support from NVIDIA, you could not thin provision your GPU's and you'll need to do a full host reboot every time you want to reassign the GPU or try to figure out if you can write a custom BIOS for your card that would somehow allow you to reinitialize it. P.S. I've actually built a KVM multi-user gaming rig with NVIDIA cards, I would not recommend anyone to try that, AMD cards are more pleasant to work with, too bad the performance are utter garbage for high end gaming.
      - pjietr 10y ago
        Agree with dharma1 that the FP32-based cloud services could be a lot useful for deep learning than the Tesla-based Clouds. Baidu has actually built Titan X -based servers for themselves as they saw it as the most cost-efficient way to build the system: http://www.nextplatform.com/2016/04/22/baidu-eyes-deep-learning-strategy-wake-new-gpu-options/ http://www.nextplatform.com/2016/04/22/baidu-eyes-deep-learn... Same logic would probably work in "deep learning" clouds if there would not be the licensing issue. And this conversation got immortalized to that slideshow as well for selecting hardware parts for deep learning systems: http://tinyurl.com/zzhj9vr http://tinyurl.com/zzhj9vr