5 ms·
For large data and trivial algorithms, such as multiplying matrices (which means then any problem you can express as a set of operations on large matrices) gpu'
by alkonaut 11y ago
For large data and trivial algorithms, such as multiplying matrices (which means then any problem you can express as a set of operations on large matrices) gpu's do really well, so it's hard to compete with something that has 1000 cores (Edit: "compute units", not "cores"). Neural nets is essentially matrix multiplication.
However a lot of interesting problems are seemingly parallel but highly branching and nonlinear. Take path tracing as an example: it's very little code and highly parallel as each Ray/pixel is independent, yet it's not an easy problem for a GPU: each time a ray bounces it will disperse and not do whatever the Ray next to it was doing in terms of which geometry it will hit etc.
It might seem like today if a problem can benefit from 8 CPU cores then it benefits 100x more from being run on a GPU but this is far from true. A great machine for general computing could do well with a board with 100 x86 CPUs apart from having a big gpu with a thousand cores for brute forcing the "simpler" problems.
- vardump 11y ago> ... so it's hard to compete with something that has 1000 cores. Any references to what has "1000 cores"? Nvidia GPUs usually have about 12 or so cores that can be compared to x86 cores, meaning they can independently branch. For example high end Nvidia 980 GTX GPU has only 16 of such comparable SIMD execution cores. SMXs or whatever Nvidia calls them. GPU marketing materials confusingly refer as cores to something like x86 CPU SIMD lanes (and that's being very generous to GPUs), that artificially inflates the numbers. Or put differently, one CUDA core can compute up to 1 FMA per cycle @1196-1300 (?) MHz. One recent Intel X86 core can compute at least up to 16 FMAs per cycle @2800-4000 Mhz.
- pandaman 11y agoI am not very familiar with NVidia hardware but I imagine an SMX is not the smallest unit, which can branch. A "warp" can branch independently and it's 32 lanes wide so I figure an SMX core with 192 "CUDA cores" can run 6 warps. It's still hundreds of cores and not thousands but much more than a dozen.
- vardump 11y agoI think WARP is more like hardware thread, and one SMX is processing one particular WARP per clock cycle. So on any given clock cycle you still have just as many independent simultaneous control paths as you have SMX units.
- pandaman 11y agoNot quite. All warps are running in parallel (otherwise you won't get the performance numbers) and each has its own control path (actually each has its own code) but, indeed, only one can execute control flow instructions at a time since the control unit is shared in the SMX.
- vardump 11y agoWell, GPUs don't have any branch prediction or out of order capabilities, so you need to have a way to keep execution units (mainly floating point units) busy. A WARP is really nothing more than a way to have work for SMXs (and computational units it controls) at as many clock cycles as possible. You need some way for masking FPU pipeline and memory latency. > All warps are running in parallel (otherwise you won't get the performance numbers) and each has its own control path (actually each has its own code) It's not that different from x86 hyperthreading, just with more hardware threads. Pipelined execution units are fed each clock cycle by the core. Multiple FP operations are in flight in parallel, otherwise CPUs won't get the performance numbers either.
- pandaman 11y agoSure, an SMX can also switch between warps in the manner similar to hyperthreading on x86 but it does not mean it executes a single warp at a time. Consider Tesla K40, a GK110 with 15 SMXs. It runs 750Mhz and has peak performance of 4.29 Tflops. If each SMX could only execute a warp at a time it could get, at most, 15(number of smx) x 32(warp width) x 750M(frequency) x 2(two flops per FMA) = 720Gflops.
- 11y ago
- alkonaut 11y agoSorry, should have written "compute units" - the point is as you point out that flops is one thing but branch prediction and feeding those floating point units is another. There has been a surge in the tractability of massive but simple linear algebra problems lately, such as deep learning, which might have given the impression that GPUs are the answer to any supercomputing.
- creshal 11y ago> A great machine for general computing could do well with a board with 100 x86 CPUs apart from having a big gpu with a thousand cores for brute forcing the "simpler" problems. Which is the idea behind Intel's Xeon Phi "GPU" with 70+ Pentium/Atom cores, which this compiler specifically targets. [Long before that, Intel showcased an 80 core x86 CPU in 2007 (Polaris/Teraflops Research Chip) – and then promptly shelved it to focus on building programming languages and compilers that can actually make use of it, before introducing the Xeon Phi half a decade later.]
- semi-extrinsic 11y agoThe problem with having 100 x86 CPU on one board is NUMA becomes a bad problem. When memory accesses from one CPU to different regions of memory will have quite different bandwidths and latencies, you're much better off by acknowledging that upfront, designing a proper interconnect (Infiniband), and not sharing memory between threads but rather communicating explicitly (MPI).
- CyberDildonics 11y agoThe next Xeon Phi has an Intel designed interconnect and a layer of 16GB high bandwidth memory between the main memory and the processor.
- Symmetry 11y agoIt's true that problems with diverging control flow work better on a number of cores than on a GPU. But by the same token a GPU's SIMT execution model does better with ray tracing than the SIMD units on a CPU. And the article is about targeting SIMD units.