7 ms·

Raja, if you are reading this make sure your Intel GPU has two things that competition doesn't: 1) FP8 half-precision training: NVidia is artificially disablin

Raja, if you are reading this make sure your Intel GPU has two things that competition doesn't: 1) FP8 half-precision training: NVidia is artificially disabling this feature in consumer GPUs to charge more for Tesla / Volta. 2) A licensed / clone of AMD SSG technology to give massive on-GPU memory: NVidia's 12 GB memory is not sufficient for anything beyond thumbnail or VGA sized images. My experience with Intel Phi KNL has been miserable so far, I hope Raja has better luck with GPU line.

userbinator 9y ago
Freely available full documentation, from package pinouts to ISA and programming guides (at least as extensive as the x86 SDM[1]), would also be met with great praise, especially in the open-source community. For a recent GPU to have such open documentation would be, AFAIK, a first. [1] https://software.intel.com/en-us/articles/intel-sdm https://software.intel.com/en-us/articles/intel-sdm
ndesaulniers 9y ago
> 1) FP8 half-precision half precision is FP16.
dogma1138 9y ago
1) is not true, in the current generation FP16 is not artificially disabled on consumer GPUs but rather does not exists. And it’s not just consumer GPUs out of 4 Pascal Tesla’s only 1 has support for FP16 and FP64 the GP100 based P100. The GP102 and 104 Cards which include the consumer cards and Tesla’s like the P4 and P40 are inference focused and support Int 16 and Int 8 dot products while the first generation Pascal GP100 doesn’t. If anyone artificially locked down FP16 support it’s AMD as consumer VEGA doesn’t support it for compute. 2) NVIDIA already has a competing solution, Pascal has had unified memory support form day one, 49 bit address space, AST and paging they already partner with SSD makers to have an addon card which is mapped to VRAM.
rbanffy 9y ago
> My experience with Intel Phi KNL has been miserable so far, I hope Raja has better luck with GPU line. I'd love to see the Phi approach taken further. I'm not a huge fan of having different ISAs, one for my CPU, one for the compute engines of the GPU (to say nothing about the blobs on my GPU, network controller, ME). I'd prefer a more general approach where I could easily spread the various workloads running on my CPU to other, perhaps more specialized but still binary-compatible, cores. Heck... Even my phone has 8 cores (4 fast, 4 power-efficient, running the same ISA).
- Symmetry 9y ago
  When you've got a huge out of order engine the extra effort it takes to decode x86 instructions is lost in the noise. When you're going with the flock of chickens approach and you have a huge number of very small cores then the overhead is killer. Intel tried to solve this by using medium cores with big SIMD units but SIMD is just less flexible than a GPU's SIMT is. Power and area generally scale as the square of the single threaded performance of a core. The huge number of "cores"/lanes in a GPU are much smaller and more efficient individually than even your phone's smaller cores. And the x86 tax gets worse and worse the smaller you try to make a core with the same ISA. Intel wasn't even able to succeed in competing with the Atom against medium sized cellphone chips.
  - rbanffy 9y ago
    > but SIMD is just less flexible than a GPU's SIMT is There is nothing preventing the x86 ISA to be extended in that direction. As long as all cores (oxen and chickens, as Seymour Cray would say) can shift binaries around according to desired performance/power, I don't care. Binary compatibility is awesome for software that has already been written and for which we don't have the source code. Pretty much everything on my machines has source available. The OS may need to be more aware of performance characteristics of software it's running on the slightly different cores so it can be better allocated, but, apart from that, it's a more or less solved problem. Atoms didn't perform that much worse than ARMs on phones. What killed them is that they didn't run our desktop software all that well (even though one of my favorite laptops was an Atom netbook).
- paulmd 9y ago
  That's a nice goal on paper but in the real world a "generic" architecture is never going to approach the performance of a specialized one with bare-metal optimizations. If your program is computationally intensive enough that you're abandoning general-purpose processors and moving to a specialized co-processor card, you should really just go whole-hog on it and bare-metal optimize to get some performance out of it. It doesn't make sense to do this half-way - which is why Xeon Phi has always faced such an uphill battle for adoption.
paulmd 9y ago
> 1) FP8 half-precision training: NVidia is artificially disabling this feature in consumer GPUs to charge more for Tesla / Volta. No, this physically is not present on consumer chips. You can't subdivide the ALUs like that even on Tesla P5000 cards. Of course you can promote FP8 to FP32 without an issue, on any card, but you don't gain any performance either. At the time Pascal was designed it didn't make any sense to waste die space on FP16 support let alone FP8, since games are purely FP32. This is changing now that Vega has FP16 capability ("Rapid Packed Math") and titles may be using this capability where appropriate. I would not be surprised to see it in Volta gaming cards at all. It's funny, everything old is new again. Someone comes up with this idea about once every 10 years. Using FP16 or FP24 used to be big back in the DX9 days. > 2) A licensed / clone of AMD SSG technology to give massive on-GPU memory: NVidia's 12 GB memory is not sufficient for anything beyond thumbnail or VGA sized images. You're looking for NVIDIA GPUDirect Peer-to-Peer, which has existed since like 2011. https://developer.nvidia.com/gpudirect https://developer.nvidia.com/gpudirect AMD's product is actually purely marketing hype, it's simply a card that contains a PLX chip to interface a NVMe SSD. It is the same technology that is used for multi-GPU cards like the Titan Z or 295x2, and it offers no performance advantages vs a regular NVMe SSD sitting in the next PCIe slot over. This is something that people didn't know they wanted until AMD told them they wanted it. But you can do this on any GeForce card even, no need to shell out $7000 for some crazy custom card that doesn't even run CUDA. The bigger problem is that there really isn't much of a use-case for it. NVMe runs at 4 GB/s, which is painfully short of the ~500 GB/s that the GPU normally runs at. That is even significantly less bandwidth than host memory can provide (a 3.0x16 PCIe bus limits you to 16 GB/s of transfers regardless of whether that's coming from NVMe or host memory).
- ActsJuvenile 9y ago
  AMD SSG supports up to quad-SSD built right on the graphics card, presumably to improve bandwidth.

userbinator 9y ago

Freely available full documentation, from package pinouts to ISA and programming guides (at least as extensive as the x86 SDM[1]), would also be met with great praise, especially in the open-source community. For a recent GPU to have such open documentation would be, AFAIK, a first. [1] https://software.intel.com/en-us/articles/intel-sdm https://software.intel.com/en-us/articles/intel-sdm

ndesaulniers 9y ago

> 1) FP8 half-precision half precision is FP16.

dogma1138 9y ago

1) is not true, in the current generation FP16 is not artificially disabled on consumer GPUs but rather does not exists. And it’s not just consumer GPUs out of 4 Pascal Tesla’s only 1 has support for FP16 and FP64 the GP100 based P100. The GP102 and 104 Cards which include the consumer cards and Tesla’s like the P4 and P40 are inference focused and support Int 16 and Int 8 dot products while the first generation Pascal GP100 doesn’t. If anyone artificially locked down FP16 support it’s AMD as consumer VEGA doesn’t support it for compute. 2) NVIDIA already has a competing solution, Pascal has had unified memory support form day one, 49 bit address space, AST and paging they already partner with SSD makers to have an addon card which is mapped to VRAM.

rbanffy 9y ago

> My experience with Intel Phi KNL has been miserable so far, I hope Raja has better luck with GPU line. I'd love to see the Phi approach taken further. I'm not a huge fan of having different ISAs, one for my CPU, one for the compute engines of the GPU (to say nothing about the blobs on my GPU, network controller, ME). I'd prefer a more general approach where I could easily spread the various workloads running on my CPU to other, perhaps more specialized but still binary-compatible, cores. Heck... Even my phone has 8 cores (4 fast, 4 power-efficient, running the same ISA).

Symmetry 9y ago
When you've got a huge out of order engine the extra effort it takes to decode x86 instructions is lost in the noise. When you're going with the flock of chickens approach and you have a huge number of very small cores then the overhead is killer. Intel tried to solve this by using medium cores with big SIMD units but SIMD is just less flexible than a GPU's SIMT is. Power and area generally scale as the square of the single threaded performance of a core. The huge number of "cores"/lanes in a GPU are much smaller and more efficient individually than even your phone's smaller cores. And the x86 tax gets worse and worse the smaller you try to make a core with the same ISA. Intel wasn't even able to succeed in competing with the Atom against medium sized cellphone chips.
- rbanffy 9y ago
  > but SIMD is just less flexible than a GPU's SIMT is There is nothing preventing the x86 ISA to be extended in that direction. As long as all cores (oxen and chickens, as Seymour Cray would say) can shift binaries around according to desired performance/power, I don't care. Binary compatibility is awesome for software that has already been written and for which we don't have the source code. Pretty much everything on my machines has source available. The OS may need to be more aware of performance characteristics of software it's running on the slightly different cores so it can be better allocated, but, apart from that, it's a more or less solved problem. Atoms didn't perform that much worse than ARMs on phones. What killed them is that they didn't run our desktop software all that well (even though one of my favorite laptops was an Atom netbook).
paulmd 9y ago
That's a nice goal on paper but in the real world a "generic" architecture is never going to approach the performance of a specialized one with bare-metal optimizations. If your program is computationally intensive enough that you're abandoning general-purpose processors and moving to a specialized co-processor card, you should really just go whole-hog on it and bare-metal optimize to get some performance out of it. It doesn't make sense to do this half-way - which is why Xeon Phi has always faced such an uphill battle for adoption.

paulmd 9y ago

> 1) FP8 half-precision training: NVidia is artificially disabling this feature in consumer GPUs to charge more for Tesla / Volta. No, this physically is not present on consumer chips. You can't subdivide the ALUs like that even on Tesla P5000 cards. Of course you can promote FP8 to FP32 without an issue, on any card, but you don't gain any performance either. At the time Pascal was designed it didn't make any sense to waste die space on FP16 support let alone FP8, since games are purely FP32. This is changing now that Vega has FP16 capability ("Rapid Packed Math") and titles may be using this capability where appropriate. I would not be surprised to see it in Volta gaming cards at all. It's funny, everything old is new again. Someone comes up with this idea about once every 10 years. Using FP16 or FP24 used to be big back in the DX9 days. > 2) A licensed / clone of AMD SSG technology to give massive on-GPU memory: NVidia's 12 GB memory is not sufficient for anything beyond thumbnail or VGA sized images. You're looking for NVIDIA GPUDirect Peer-to-Peer, which has existed since like 2011. https://developer.nvidia.com/gpudirect https://developer.nvidia.com/gpudirect AMD's product is actually purely marketing hype, it's simply a card that contains a PLX chip to interface a NVMe SSD. It is the same technology that is used for multi-GPU cards like the Titan Z or 295x2, and it offers no performance advantages vs a regular NVMe SSD sitting in the next PCIe slot over. This is something that people didn't know they wanted until AMD told them they wanted it. But you can do this on any GeForce card even, no need to shell out $7000 for some crazy custom card that doesn't even run CUDA. The bigger problem is that there really isn't much of a use-case for it. NVMe runs at 4 GB/s, which is painfully short of the ~500 GB/s that the GPU normally runs at. That is even significantly less bandwidth than host memory can provide (a 3.0x16 PCIe bus limits you to 16 GB/s of transfers regardless of whether that's coming from NVMe or host memory).

ActsJuvenile 9y ago
AMD SSG supports up to quad-SSD built right on the graphics card, presumably to improve bandwidth.