8 ms·
> What workloads are so massively parallel that they can use 64 cores of x86 but can’t use the thousands of CUDA cores on a Quadro card? Building software, for
by ajross 6y ago
> What workloads are so massively parallel that they can use 64 cores of x86 but can’t use the thousands of CUDA cores on a Quadro card?
Building software, for one. C compilers and python interpreters don't run on a GPU. Lots of stuff doesn't run on a GPU. In fact in practice the only things that run on a GPU are the tiny handful of known subproblems that the industry has collectively decided are "GPU problems".
Like it or not general purpose scalar software is, has always been, and will always remain the standard mechanism by which computing hardware is applied to new problems. Everything else is an optimization around the edges.
- scotty79 6y agoInteresting. Is there a compiler, that runs on GPU?
- the_pwner224 6y agoGPUs are mainly good at doing matrix multiplication and other math with little branching. Compilers have a lot of if/else statements; training neural networks does not. IIRC (and I may be wrong) the architecture is something like many GPU cores share things like the code cache and branch prediction, so if the different cores have to execute different code paths there is a big performance decrease. Edit: another reply to this top level comment: https://news.ycombinator.com/item?id=23788755 https://news.ycombinator.com/item?id=23788755
- Symmetry 6y agoAlso GPUs really like predictable memory access patterns so that they can load the data that'll be needed next ahead of time into registers or dedicated caches explicitly. They don't have cache hierarchy features a CPU has to minimize the average disruption from only partially predictable memory accesses and they don't have out of order windows to paper over the occasional low level cache miss.
- FpUser 6y agoNot to counter the main argument but CPU loves it some predictable patterns as well. pattern like the one below will execute up to 10 times faster if array is sorted due to data locality and branch prediction sum: float a: float array[some_huge_number] fill_array(a, random(1)) for f in a if a[i]>0.5 sum=sum+f
- raphlinus 6y agoA really interesting exploration is Co-dfns [1]. This is indeed a compiler that runs on the GPU, but it's also extremely out of the mainstream. IMHO this is an area ripe for more exploration. If I were working on it, I might look to linking first before compilation, because the basic link task is more similar to what GPUs are good at (advanced stuff such as LTO is a different story, though). [1]: https://github.com/Co-dfns/Co-dfns https://github.com/Co-dfns/Co-dfns
- BiasRegularizer 6y agoAlso, GPU has VERY limited memory and no virtual memory.
- rational_indian 6y ago> VERY limited memory and no virtual memory This is incorrect.
- zdw 6y agoCompared to 2-4TB max in a 2 socket server system, the 16 or 32GB/card you get on a GPU is pretty small. The other issue is loading that memory - PCIe 4 is still the transfer time bottleneck between GPU and main memory.
- atq2119 6y agoThe part about no virtual memory is kind of incorrect though. It doesn't work exactly like on the CPU, but still.
- zdw 6y agoI'm not totally up to speed on modern GPU's - do they have a MMU that handles page faults on non-resident pages like a CPU would? My understanding is that the GPU memory models are different enough that what an OS traditionally calls "virtual memory" couldn't be implemented in the same way.
- baybal2 6y ago> Building software Well, the problem with that is that no make system can utilise so much core. Even linux kernel, probably the biggest C project in mainstream use, can not consistently load even 16 cores with mostly handwritten makefiles. P.S. I do not say that building software is not CPU parallelisable in principles, I'm saying that regular make systems have trouble handing big number of parallel tasks in practice, even in big, well run projects that use handwritten makefiles.
- ajross 6y agoThat's... not true at all. A kernel build will be as parallelizable as you like. I do kernel builds fairly regularly on 56 and 64 core hosts and in fact the build time is quite linear with core count. (The ramdisk packing provided by the distro at "make install" time tends to be what hurts scaling, FWIW). And my day job involves lots of iterations over the Zephyr test suite, which builds and runs hundreds of individual test apps X dozens of platforms. This is almost a pure build-time problem and scales very linearly indeed.
- baybal2 6y agoNot in my experience, from time to time it does boost to 40+ compile tasks, especially on drivers code, but it mostly hover at around 10 tasks.
- CrazyStat 6y agoCPU-parallelizable and GPU-parallelizable are different problems. CPU cores are independent. GPU cores are SIMD--every core does the same operation, but with different data. That works really well for things like matrix multiplication where you need to multiply a million times and then add a million times, because you can have all 3000 CUDA cores multiplying and then all 3000 CUDA cores adding. It doesn't work for compiling software.
- monocasa 6y agoIt's not that SIMD for GPUs. It'll be 64 or 32 GPU threads together generally sharing the dispatch and control logic. So if you have a 4096 "CUDA core" GPU, it'll be 64 of what would be called cores on the CPU side, each with a 64 lane vector processor.
- lallysingh 6y ago.. and chrome and your IDE. They're very bursty, but those bursts can go smoothly over many cores or slowly over a few.
- waltpad 6y agoDisclaimer: I am not a HW designer, I could very well be wrong. It is true that there are tasks where threading matters, but still require a CPU rather than a GPU. I wonder however if these tasks do need full SSE/AVX etc. Couldn't these extensions be removed of the CPU cores and instead have the necessary work performed by the GPU? It would be interesting to produce statistics on how much these extensions are used in these scenario. Imagine how much space and complexity could be saved on a CPU die by making stripped down versions. That space could in turn be used for more cores! I read a little about the Xeon PHI cpus, which iirc, is a multicore CPU with a very small ISA, but I wonder why x86 makers aren't trying to go in that direction: isn't there plenty of dedicated workloads which would happily run on these (eg, web servers), or is this just a (too) simplistic view?
- dragontamer 6y ago> It is true that there are tasks where threading matters, but still require a CPU rather than a GPU. I wonder however if these tasks do need full SSE/AVX etc. Couldn't these extensions be removed of the CPU cores and instead have the necessary work performed by the GPU? SSE/AVX shares an L1 cache that's damn near instantaneous to access for the CPU core. Total L1 bandwidth is on the scale of TB/s. PCIe -> GPU takes 1-microsecond to 10-microseconds per access, and operates only at 50GB/s (or 1/20th the speed of L1 bandwidths). ------------ Case in point: Memset is very commonly AVX'd to clear out L1 cache and initialize ~1kb to 32kb of data to 0 as quickly as possible. There's no way for "memset" to move from CPU to GPU unless you feel like obliterating the entire point of L1, L2, and L3 cache. If you moved a "memset" to GPU, it'd operate only at 15GB/s (the speed of PCIe 3.0 x16 lanes), far, far slower than L1 cache AVX-loads/stores. SIMD units, like SSE and AVX, are highly "local" and have huge advantages.
- TinkersW 6y agoI think the opposite is where things need to go. Having a wide SIMD ALU quickly accessible from your CPU core is very useful, especially as it shares the same memory system and a much more flexible programming model that allows you to do everything in a single source.
- 6y ago