4 ms·

Nvidia already has high speed GPU-GPU links: http://www.nvidia.com/object/nvlink.html http://www.nvidia.com/object/nvlink.html

jamesblonde 10y ago
They aren't widely deployed yet - the DGX-1 uses it, though. I assume it will be infiniband that will be the commodity hard adopted to distribute Deep Learning. However, Nvidia are currently trying to segment the market by providing GPU->GPU direct access via infiniband on only the Tesla cards - not the commodity ones (titanx, 1080 gtx). That strategy will probably be the death of them in Deep Learning.
- mattnewton 10y ago
  Or it will make buttloads of money until they have to move it downmarket when serious competitors arrive. Money they can use to lap competition by investing in better cudnn and hardware. Full disclosure: I am (and have been since I found about about cudnn) long nvidia.
  - spangry 10y ago
    That's how I see it playing out too, assuming Nvidia pull it off. By my reckoning, it roughly goes: HPC -> Mainstream servers (and niches like ML) -> 'Professional workstations' -> Consumer desktops Intel's response (at least in part) seems fairly sensible: throw money and developers at open-source and open-standardisation efforts to try and rally the rest of the market (e.g around stuff like OpenCL). Otherwise network effects from stuff like widespread CUDA usage will become too strong to overcome, at which point Nvidia will have a monopoly. However, I think they're missing an opportunity when it comes to high-speed networking. Specifically, I think they should be looking to release cheap, simple and open-standards based networking gear without trying to cross-sell Xeons or whatever at the same time. I realise that this means cannibalising revenue from their CPU market, but at this point someone is going to do it so it may as well be them. Hell, they could probably even get away with tying the functionality in to some fairly widespread Intel CPU instruction set (say, from Ivybridge onward). However, anything past that and it's a non-starter for consumer segments (which, in my mind, is also the developer segment). 10gbit RDMA is actually sort of affordable nowadays (at least if you pick up gear from eBay). But it's currently a complicated mess of competing and incompatible standards. Some SFP+ modules are even vendor-locked; if they sense there's a different SFP+ branded module on the other end they stop functioning due to an 'unsupported configuration'. These kinds of things make it impossible for a decent sized consumer market to form. If Intel could just accept that the world is moving on with or without them and get a proper consumer market going, at the very least they'd keep the market open (and at the very best the might even tilt the odds in their favour). AMD, on the other hand, I have no idea wtf they are doing (and neither do they, by the looks of it). It could just be my biases, but I really think the best strategy for them is to basically bet the house on open-source and really up their engagement with Linux core dev. While I get it's attractive to target the gaming market because it's profitable in the present, it probably means death in the future. Actually their interests are pretty aligned with Intel's, and they have highly complementary work forces. It sounds crazy, but I wouldn't be too surprised if Intel were to acquire AMD at some point, or at least buy out their GPU division...
- greglindahl 10y ago
  Why are you assuming Infiniband is the winner? There's Intel Omni-Path and there's Ethernet. (Disclaimer: I'm the system architect of InfiniPath, which is one of the things that evolved into Omni-Path.)
  - spangry 10y ago
    Ah, just the man I've been wanting to talk to then :) As I mentioned in another post, I really think Intel are missing an opportunity when it comes to high speed networking. Why don't they drop all the crazy server market lock-in vendor shenanigans (e.g. brand to brand SFP incompatibility, 'call us' pricing etc.) and just jump straight to the consumer market? I know this means cannibalising CPU revenue, but that's going to happen anyway. Developers are consumers. In many cases the direction that their skills develop in is largely determined by the consumer hardware they're running at home. The same seems to be true for Ops folks, who all seems to be running vSphere homelabs atm. Based on what others have said here, it sounds like NVIDIA have chosen to cross-sell, and work their way down from HPC -> ... -> Consumer, presumably to extract maximum rents, despite not yet having strong enough network effects o lock out competition. This seems like the perfect opportunity for Intel to cut the legs out from under that strategy, by aggressively pushing widespread consumer (or at least 'power-user'/developer) take-up of high speed, low-latency networking. And hasn't iWARP been endorsed as an IETF standard (or is on its way to be)? It seems like there's a window of opportunity here, albeit one that gets smaller and smaller as Nvidia slowly work their way down market.
    - greglindahl 10y ago
      Well, Omni-Path has a 48 port switch ASIC. So that's it for consumer accessibility. But if you want to buy the PCI Express cards, Newegg has 'em. Intel has a great channel organization.
      - spangry 10y ago
        It's certainly impressive tech. But, and bear in mind this is just from a consumer perspective, here are the reasons I wouldn't buy it: - ~$8000 USD for a 24 port switch is way beyond my price range. I doubt any consumer market could support this (http://www.newegg.com/Product/Product.aspx?Item=9SIA6ZP53G3510 http://www.newegg.com/Product/Product.aspx?Item=9SIA6ZP53G35...) - ~$550 USD for a single port adapter is also way outside of my budget, especially since I'd need to buy at least half a dozen of these for my home lab (http://www.newegg.com/Product/Product.aspx?Item=N82E16833106290 http://www.newegg.com/Product/Product.aspx?Item=N82E16833106...) - It's very difficult to find solid information on the product. The Ark page doesn't tell me anything (http://ark.intel.com/products/92007/Intel-Omni-Path-Host-Fabric-Interface-Adapter-100-Series-1-Port-PCIe-x16 http://ark.intel.com/products/92007/Intel-Omni-Path-Host-Fab...). The link to the whitepaper (which I'm assuming is actually a marketing brochure) 404s. - So even if I could afford this as a consumer/dev, I probably wouldn't buy it due to the uncertainty around total cost of ownership. Will it accept non-Intel branded QSFP transceivers? Same question with optical cabling. - Given the product brief mentions 'fabric performance [will] scale automatically with ongoing advances in Intel Xeon processors...', it leaves me wondering if this is somehow dependent on some specialised Xeon CPU instruction sets, making it useless for most home labs (which, at best, will be running older generation Xeons from second-hand servers, but more likely CPUs from Intel's consumer line-up). On costs: If these are just reflective of the cost of production, then fair enough, I guess I'll just have to wait (for either Intel, or another manufacturer, to devise a cheaper production process). If it's not, then I think you're missing the opportunity outlined above, and you'll lose the HPC fight given Nvidia occupy the high ground here. On compatibility: If this is just some idiot 'multi-channel cross sales' strategy that some genius with an MBA has cooked up, you're going to have a bad time. It falls apart the second someone releases a commodity adapter (which may already exist, for all I know).

jamesblonde 10y ago

They aren't widely deployed yet - the DGX-1 uses it, though. I assume it will be infiniband that will be the commodity hard adopted to distribute Deep Learning. However, Nvidia are currently trying to segment the market by providing GPU->GPU direct access via infiniband on only the Tesla cards - not the commodity ones (titanx, 1080 gtx). That strategy will probably be the death of them in Deep Learning.

mattnewton 10y ago
Or it will make buttloads of money until they have to move it downmarket when serious competitors arrive. Money they can use to lap competition by investing in better cudnn and hardware. Full disclosure: I am (and have been since I found about about cudnn) long nvidia.
- spangry 10y ago
  That's how I see it playing out too, assuming Nvidia pull it off. By my reckoning, it roughly goes: HPC -> Mainstream servers (and niches like ML) -> 'Professional workstations' -> Consumer desktops Intel's response (at least in part) seems fairly sensible: throw money and developers at open-source and open-standardisation efforts to try and rally the rest of the market (e.g around stuff like OpenCL). Otherwise network effects from stuff like widespread CUDA usage will become too strong to overcome, at which point Nvidia will have a monopoly. However, I think they're missing an opportunity when it comes to high-speed networking. Specifically, I think they should be looking to release cheap, simple and open-standards based networking gear without trying to cross-sell Xeons or whatever at the same time. I realise that this means cannibalising revenue from their CPU market, but at this point someone is going to do it so it may as well be them. Hell, they could probably even get away with tying the functionality in to some fairly widespread Intel CPU instruction set (say, from Ivybridge onward). However, anything past that and it's a non-starter for consumer segments (which, in my mind, is also the developer segment). 10gbit RDMA is actually sort of affordable nowadays (at least if you pick up gear from eBay). But it's currently a complicated mess of competing and incompatible standards. Some SFP+ modules are even vendor-locked; if they sense there's a different SFP+ branded module on the other end they stop functioning due to an 'unsupported configuration'. These kinds of things make it impossible for a decent sized consumer market to form. If Intel could just accept that the world is moving on with or without them and get a proper consumer market going, at the very least they'd keep the market open (and at the very best the might even tilt the odds in their favour). AMD, on the other hand, I have no idea wtf they are doing (and neither do they, by the looks of it). It could just be my biases, but I really think the best strategy for them is to basically bet the house on open-source and really up their engagement with Linux core dev. While I get it's attractive to target the gaming market because it's profitable in the present, it probably means death in the future. Actually their interests are pretty aligned with Intel's, and they have highly complementary work forces. It sounds crazy, but I wouldn't be too surprised if Intel were to acquire AMD at some point, or at least buy out their GPU division...
greglindahl 10y ago
Why are you assuming Infiniband is the winner? There's Intel Omni-Path and there's Ethernet. (Disclaimer: I'm the system architect of InfiniPath, which is one of the things that evolved into Omni-Path.)
- spangry 10y ago
  Ah, just the man I've been wanting to talk to then :) As I mentioned in another post, I really think Intel are missing an opportunity when it comes to high speed networking. Why don't they drop all the crazy server market lock-in vendor shenanigans (e.g. brand to brand SFP incompatibility, 'call us' pricing etc.) and just jump straight to the consumer market? I know this means cannibalising CPU revenue, but that's going to happen anyway. Developers are consumers. In many cases the direction that their skills develop in is largely determined by the consumer hardware they're running at home. The same seems to be true for Ops folks, who all seems to be running vSphere homelabs atm. Based on what others have said here, it sounds like NVIDIA have chosen to cross-sell, and work their way down from HPC -> ... -> Consumer, presumably to extract maximum rents, despite not yet having strong enough network effects o lock out competition. This seems like the perfect opportunity for Intel to cut the legs out from under that strategy, by aggressively pushing widespread consumer (or at least 'power-user'/developer) take-up of high speed, low-latency networking. And hasn't iWARP been endorsed as an IETF standard (or is on its way to be)? It seems like there's a window of opportunity here, albeit one that gets smaller and smaller as Nvidia slowly work their way down market.
  - greglindahl 10y ago
    Well, Omni-Path has a 48 port switch ASIC. So that's it for consumer accessibility. But if you want to buy the PCI Express cards, Newegg has 'em. Intel has a great channel organization.
    - spangry 10y ago
      It's certainly impressive tech. But, and bear in mind this is just from a consumer perspective, here are the reasons I wouldn't buy it: - ~$8000 USD for a 24 port switch is way beyond my price range. I doubt any consumer market could support this (http://www.newegg.com/Product/Product.aspx?Item=9SIA6ZP53G3510 http://www.newegg.com/Product/Product.aspx?Item=9SIA6ZP53G35...) - ~$550 USD for a single port adapter is also way outside of my budget, especially since I'd need to buy at least half a dozen of these for my home lab (http://www.newegg.com/Product/Product.aspx?Item=N82E16833106290 http://www.newegg.com/Product/Product.aspx?Item=N82E16833106...) - It's very difficult to find solid information on the product. The Ark page doesn't tell me anything (http://ark.intel.com/products/92007/Intel-Omni-Path-Host-Fabric-Interface-Adapter-100-Series-1-Port-PCIe-x16 http://ark.intel.com/products/92007/Intel-Omni-Path-Host-Fab...). The link to the whitepaper (which I'm assuming is actually a marketing brochure) 404s. - So even if I could afford this as a consumer/dev, I probably wouldn't buy it due to the uncertainty around total cost of ownership. Will it accept non-Intel branded QSFP transceivers? Same question with optical cabling. - Given the product brief mentions 'fabric performance [will] scale automatically with ongoing advances in Intel Xeon processors...', it leaves me wondering if this is somehow dependent on some specialised Xeon CPU instruction sets, making it useless for most home labs (which, at best, will be running older generation Xeons from second-hand servers, but more likely CPUs from Intel's consumer line-up). On costs: If these are just reflective of the cost of production, then fair enough, I guess I'll just have to wait (for either Intel, or another manufacturer, to devise a cheaper production process). If it's not, then I think you're missing the opportunity outlined above, and you'll lose the HPC fight given Nvidia occupy the high ground here. On compatibility: If this is just some idiot 'multi-channel cross sales' strategy that some genius with an MBA has cooked up, you're going to have a bad time. It falls apart the second someone releases a commodity adapter (which may already exist, for all I know).