9 ms·
Implementing a GPU's programming model on a CPU
- saagarjha 3y ago(Is this available anywhere?)
- hguj 3y ago[flagged]
- fulafel 3y agoSee also: https://ispc.github.io/ https://ispc.github.io/
- hkgjjgjfjfjfjf 3y ago[dead]
- ckyngcybra 3y ago[flagged]
- fancyfredbot 3y agoISPC is great software. It's a shame in many ways Intel didn't put more resources behind it.
- boulos 3y agoAs Bryan mentions in https://news.ycombinator.com/item?id=37880669 https://news.ycombinator.com/item?id=37880669, I would read Matt's post-Intel and post-Google write up of why Intel didn't really invest. https://pharr.org/matt/blog/2018/04/30/ispc-all https://pharr.org/matt/blog/2018/04/30/ispc-all if you don't want to follow the comment link again :).
- kristjansson 3y agoI naively thought ISPC turned into the new DPCPP compiler they’re leaning into with their oneAPI marketing. Are those really different lineages?
- ckyngcybra 3y ago[flagged]
- adrian_b 3y agoThis so-called GPU programming model has existed many decades before the appearance of the first GPUs, but at that time the compilers were not so good like the CUDA compilers, so the burden for a programmer was greater. As another poster has already mentioned, there exists a compiler for CPUs which has been inspired by CUDA and which has been available for many years: ISPC (Implicit SPMD Program Compiler), at https://github.com/ispc/ispc https://github.com/ispc/ispc . NVIDIA has the very annoying habit of using a lot of terms that are different from those that have been previously used in computer science for decades. The worst is that NVIDIA has not invented new words, but they have frequently reused words that have been widely used with other meanings. SIMT (Single-Instruction Multiple Thread) is not the worst term coined by NVIDIA, but there was no need for yet another acronym. For instance they could have used SPMD (Single Program, Multiple Data Stream), which dates from 1988, two decades before CUDA. Moreover, SIMT is the same thing that was called "array of processes" by C.A.R. Hoare in August 1978 (in "Communicating Sequential Processes"), or "replicated parallel" by Occam in 1985 or "PARALLEL DO" by "OpenMP Fortran" in 1997-10 or "parallel for" by "OpenMP C and C++" in 1998-10. Each so-called CUDA kernel is just the body of a "parallel for" (which is multi-dimensional, like in Fortran). The only (but extremely important) innovation brought by CUDA is that the compiler is smart enough so that the programmer does not need to know the structure of the processor, i.e. how many cores it has and how many SIMD lanes each core has. The CUDA compiler distributes automatically the work over the available SIMD lanes and available cores and in most cases the programmer does not care whether two executions of the function that must be executed for each data item are done on two different cores or on two different SIMD lanes of the same core. This distribution of the work over SIMD lanes and over cores is simple when the SIMD operations are maskable, like in GPUs or in AVX-512 a.k.a. AVX10 or in ARM SVE. When masking is not available, like in AVX2 or Armv8-A, the implementation of conditional statements and expressions is more complicated.
- mathisfun123 3y ago[flagged]
- tovej 3y agoAre you even reading the text you cite? That compares SIMD to SIMT. SIMD, not SPMD. SIMD is a more restrictive hardware paradigm, SPMD is just a mutiple copies of a single program running on multiple processes/threads, i.e., exactly what CUDA does. Most MPI-programs are SPMD, and really, the only conceptual difference between MPI and CUDA is that a thread's id is a point in a grid space that's relevant for scheduling and memory locality. Oh wait, that's true for MPI as well, only the scheduling and memory locality relates to CPU cores and nodes, instead of CUDAs SMPs and blocks. The only real justification for a different programming model is that SIMT creates opportunities for implicit SIMD on a thread scheduling level if you've written your program in a way that the warps don't diverge.
- amelius 3y agoIsn't this already implemented in QEMU?
- hguj 3y ago[flagged]
- mikepurvis 3y agoI don't think so. I was quite interested in getting emulated GPUs to work for CI workers to be able to test CUDA stuff without needing actual GPU passthrough, but all I could find in this space were abandoned, early stage hobby projects on GitHub.
- bonzini 3y agoNo, QEMU only supports up to AVX2. Based on the time it took to implement AVX and AVX2, I expect it would be about a month of full time work to implement AVX512F, AVX512BW and AVX512VL (it would not be much simpler to implement only one of them).
- imtringued 3y agoYou mean llvmpipe. You can run OpenCL programs with it.
- phdelightful 3y agoOne of my colleague's Ph.D. thesis was on how to achieve high-performance CPU implementations for bulk-synchronous programming models ("GPU programming") http://impact.crhc.illinois.edu/shared/Thesis/dissertation-hee-seok_kim.pdf http://impact.crhc.illinois.edu/shared/Thesis/dissertation-h...
- bcatanzaro 3y agoMatt Pharr’s series of blogs on ISPC are worth reading: https://pharr.org/matt/blog/2018/04/30/ispc-all https://pharr.org/matt/blog/2018/04/30/ispc-all
- raphlinus 3y agoIn addition to ISPC, some of this is also done in software fallback implementations of GPU APIs. In the open source world we have SwiftShader and Lavapipe, and on Windows we have WARP[1]. It's sad to me that Larrabee didn't catch on, as that might have been a path to a good parallel computer, one that has efficient parallel throughput like a GPU, but also agility more like a CPU, so you don't need to batch things into huge dispatches and wait RPC-like latencies for them to complete. Apparently the main thing that sunk it was power consumption. [1]: https://learn.microsoft.com/en-us/windows/win32/direct3darticles/directx-warp https://learn.microsoft.com/en-us/windows/win32/direct3darti...
- zozbot234 3y agoThe recent "AI chip" proposals (Tenstorrent, Esperanto Technologies etc.) seem to be quite similar to the old Larrabee design, except based on RISC-V as opposed to x86. So we might see that happen after all.
- raphlinus 3y agoYes, I've got my eye on those and am hopeful. Do you know of any meaty technical description of the programming model? All I've been able to find so far is fairly high level marketing material. At least for Tenstorrent Jim Keller has promised that the software stack will be open sourced, something I'm looking forward to.
- samsartor 3y agoI helped make a really cursed RISC-V version of this for a class project last year! The idea was to first compile each program to WASM using clang, and lower the WASM back to C but this time with all opcodes implemented in terms of the RISC-V vector intrinsics. That was a hack to be sure, but a surprisingly elegant one since 1. WASM's structured control flow maps really well to lane masking 2. Stack and local values easily use "structure of arrays" layout 3. Heap values easily use "array of structures" layout It never went anywhere but the code is still online if anyone wants to stare directly at the madness: https://gitlab.com/samsartor/wasm2simt https://gitlab.com/samsartor/wasm2simt
- TechnicolorByte 3y ago> This is in contrast to SIMD, or "single instruction multiple data," where the programmer explicitly uses vector types and operations in their program. The SIMD approach is suited for when you have a single program that has to process a lot of data, whereas SIMT is suited for when you have many programs and each one operates on its own data This statement is comparing the SIMT model to SIMD. Can anyone explain the last part about SIMT being better for many programs operating on its own data? Are they just saying you can have individual “threads” executing independently (via predication/masks and such)?
- mhh__ 3y agoSIMT let's a scheduler get clever about memory accesses, SIMD can practically only access memory linearly (scatter gather can do better but it's still usually quite linear) whereas SIMT can be much smarter in terms of having lots of similar bits of work going on in ways that use the bandwidth maximally and don't overlap.
- kllrnohj 3y agohttps://developer.nvidia.com/blog/how-access-global-memory-efficiently-cuda-c-kernels/ https://developer.nvidia.com/blog/how-access-global-memory-e... SIMT still expects coalesced memory access that's close together otherwise performance falls off a cliff
- the_svd_doctor 3y agoYes, but not all thread in the block need to. As long as you fill a cache line you’re good.
- westurner 3y agoHey, AVX-512 again! "Show HN: SimSIMD vs SciPy: How AVX-512 and SVE make SIMD nicer and ML 10x faster" (2023-10) https://news.ycombinator.com/item?id=37805810 https://news.ycombinator.com/item?id=37805810
- pjmlp 3y agoWhich is to be expected, given that AVX is what survived from Larrabe.
- vient 3y agoSeems to be the same concept as in https://gamozolabs.github.io/fuzzing/2018/10/14/vectorized_emulation.html https://gamozolabs.github.io/fuzzing/2018/10/14/vectorized_e..., cool!