30 ms·

If Nvidia are looking to supplant Intel, I think they key, weirdly, is i/o. In a sense, GPU acceleration of general purpose computation is just a specific case

by spangry 10y ago

If Nvidia are looking to supplant Intel, I think they key, weirdly, is i/o. In a sense, GPU acceleration of general purpose computation is just a specific case of the more general case of distributed computing (albeit the computers are extremely 'close'). For embarrassingly parallel workloads, i/o is not an issue; the data and computations can be batched, distributed and no further communication required (until you collect the results). But for mixed, semi-sequential/semi-parallel workloads, I/O becomes the bottleneck. Having to retrieve intermediate results, combine and then redistribute can eat up your gains from parallelisation. It will be interesting to see if Nvidia start pushing for a new standard to replace PCIe, and maybe invest in low-latency, high throughput networking R&D. Who knows? Maybe they'll just build some kind of networking functionality in to their GPUs directly, so data can be transferred NVRAM -> NVRAM without having to travel up and down the stack (until they need to interact with userspace; once on the way in and once on the way out). Given there are RDMA adapters that can manage ~20ns latency already, it would be interesting to see what could be achieved before we start pushing up against the laws of physics. Even at the physical layer, my understanding is that most networking fibre optics manage about 60% the speed of light. So, just to make a wild prediction: within the next year, Nvidia will acquire a HPC network adapter company (e.g. Mellanox).

vomitcuddle 10y ago
Does Nvidia already have NVLink for this purpose?
- jsheard 10y ago
  They do, but you can't run NVLink to the CPU unless you're using exotic IBM POWER8 systems. x86 systems like the DGX-1 can only run NVLink between the GPUs, and still rely on PCI-E for the CPU connection.
jburgess777 10y ago
Nvidia already has high speed GPU-GPU links: http://www.nvidia.com/object/nvlink.html http://www.nvidia.com/object/nvlink.html
- jamesblonde 10y ago
  They aren't widely deployed yet - the DGX-1 uses it, though. I assume it will be infiniband that will be the commodity hard adopted to distribute Deep Learning. However, Nvidia are currently trying to segment the market by providing GPU->GPU direct access via infiniband on only the Tesla cards - not the commodity ones (titanx, 1080 gtx). That strategy will probably be the death of them in Deep Learning.
  - mattnewton 10y ago
    Or it will make buttloads of money until they have to move it downmarket when serious competitors arrive. Money they can use to lap competition by investing in better cudnn and hardware. Full disclosure: I am (and have been since I found about about cudnn) long nvidia.
    - spangry 10y ago
      That's how I see it playing out too, assuming Nvidia pull it off. By my reckoning, it roughly goes: HPC -> Mainstream servers (and niches like ML) -> 'Professional workstations' -> Consumer desktops Intel's response (at least in part) seems fairly sensible: throw money and developers at open-source and open-standardisation efforts to try and rally the rest of the market (e.g around stuff like OpenCL). Otherwise network effects from stuff like widespread CUDA usage will become too strong to overcome, at which point Nvidia will have a monopoly. However, I think they're missing an opportunity when it comes to high-speed networking. Specifically, I think they should be looking to release cheap, simple and open-standards based networking gear without trying to cross-sell Xeons or whatever at the same time. I realise that this means cannibalising revenue from their CPU market, but at this point someone is going to do it so it may as well be them. Hell, they could probably even get away with tying the functionality in to some fairly widespread Intel CPU instruction set (say, from Ivybridge onward). However, anything past that and it's a non-starter for consumer segments (which, in my mind, is also the developer segment). 10gbit RDMA is actually sort of affordable nowadays (at least if you pick up gear from eBay). But it's currently a complicated mess of competing and incompatible standards. Some SFP+ modules are even vendor-locked; if they sense there's a different SFP+ branded module on the other end they stop functioning due to an 'unsupported configuration'. These kinds of things make it impossible for a decent sized consumer market to form. If Intel could just accept that the world is moving on with or without them and get a proper consumer market going, at the very least they'd keep the market open (and at the very best the might even tilt the odds in their favour). AMD, on the other hand, I have no idea wtf they are doing (and neither do they, by the looks of it). It could just be my biases, but I really think the best strategy for them is to basically bet the house on open-source and really up their engagement with Linux core dev. While I get it's attractive to target the gaming market because it's profitable in the present, it probably means death in the future. Actually their interests are pretty aligned with Intel's, and they have highly complementary work forces. It sounds crazy, but I wouldn't be too surprised if Intel were to acquire AMD at some point, or at least buy out their GPU division...

by spangry 10y ago

vomitcuddle 10y ago

Does Nvidia already have NVLink for this purpose?

jsheard 10y ago
They do, but you can't run NVLink to the CPU unless you're using exotic IBM POWER8 systems. x86 systems like the DGX-1 can only run NVLink between the GPUs, and still rely on PCI-E for the CPU connection.

jburgess777 10y ago

Nvidia already has high speed GPU-GPU links: http://www.nvidia.com/object/nvlink.html http://www.nvidia.com/object/nvlink.html

jamesblonde 10y ago
They aren't widely deployed yet - the DGX-1 uses it, though. I assume it will be infiniband that will be the commodity hard adopted to distribute Deep Learning. However, Nvidia are currently trying to segment the market by providing GPU->GPU direct access via infiniband on only the Tesla cards - not the commodity ones (titanx, 1080 gtx). That strategy will probably be the death of them in Deep Learning.
- mattnewton 10y ago
  Or it will make buttloads of money until they have to move it downmarket when serious competitors arrive. Money they can use to lap competition by investing in better cudnn and hardware. Full disclosure: I am (and have been since I found about about cudnn) long nvidia.
  - spangry 10y ago
    That's how I see it playing out too, assuming Nvidia pull it off. By my reckoning, it roughly goes: HPC -> Mainstream servers (and niches like ML) -> 'Professional workstations' -> Consumer desktops Intel's response (at least in part) seems fairly sensible: throw money and developers at open-source and open-standardisation efforts to try and rally the rest of the market (e.g around stuff like OpenCL). Otherwise network effects from stuff like widespread CUDA usage will become too strong to overcome, at which point Nvidia will have a monopoly. However, I think they're missing an opportunity when it comes to high-speed networking. Specifically, I think they should be looking to release cheap, simple and open-standards based networking gear without trying to cross-sell Xeons or whatever at the same time. I realise that this means cannibalising revenue from their CPU market, but at this point someone is going to do it so it may as well be them. Hell, they could probably even get away with tying the functionality in to some fairly widespread Intel CPU instruction set (say, from Ivybridge onward). However, anything past that and it's a non-starter for consumer segments (which, in my mind, is also the developer segment). 10gbit RDMA is actually sort of affordable nowadays (at least if you pick up gear from eBay). But it's currently a complicated mess of competing and incompatible standards. Some SFP+ modules are even vendor-locked; if they sense there's a different SFP+ branded module on the other end they stop functioning due to an 'unsupported configuration'. These kinds of things make it impossible for a decent sized consumer market to form. If Intel could just accept that the world is moving on with or without them and get a proper consumer market going, at the very least they'd keep the market open (and at the very best the might even tilt the odds in their favour). AMD, on the other hand, I have no idea wtf they are doing (and neither do they, by the looks of it). It could just be my biases, but I really think the best strategy for them is to basically bet the house on open-source and really up their engagement with Linux core dev. While I get it's attractive to target the gaming market because it's profitable in the present, it probably means death in the future. Actually their interests are pretty aligned with Intel's, and they have highly complementary work forces. It sounds crazy, but I wouldn't be too surprised if Intel were to acquire AMD at some point, or at least buy out their GPU division...