5 ms·
What Every Developer Should Know About GPU Computing (2023)
- cgreerrun 2y agoThis video is a great explainer too: How do Graphics Cards Work? Exploring GPU Architecture (https://youtu.be/h9Z4oGN89MU?si=EPPO0kny-gN0zLeC https://youtu.be/h9Z4oGN89MU?si=EPPO0kny-gN0zLeC)
- penr0se 2y agoWow, this is one of the best videos I've ever watched. Thanks for sharing
- ferbivore 2y agoThese videos always give me an unpleasant feeling that I don't know how to express in a useful way. Almost everything in this one is oversimplified, misleading, or wrong, yet I feel like any attempt to argue for this would come off as pedantic; any individual complaint could be countered by either "it's technically true" or "it's just entertainment". Like, the "cowboy hat" example is wrong on multiple levels - GPUs are not SIMD machines, model-to-world translation doesn't work like that in practice - but you can maybe excuse it as a useful educational lie, and he does kind-of explain SIMT later, so is objecting to it valid? Or: the video claims to be about "GPUs", but is in fact exclusively about Nvidia's architecture and the GA102 in particular; is this a valid complaint, or is the lie excusable because it's just a YouTube video? Or: it overemphasizes the memory chips because of who's sponsoring it; does this compromise the message? Or: it plays fast-and-loose with die shots and floorplans; is a viewer expected to understand that it's impossible to tell where the FMA units really are? Or: it spends a lot of time on relatively unimportant topics while neglecting things like instruction dispatch, registers, dedicated graphics hardware, etc.; but is it really fair to complain, considering the target audience doesn't seem to be programmers? And so on. Did you actually get anything out of this video? Any new knowledge? The article seems like a much more useful intro, even if it is also specific to Nvidia and CUDA.
- cruffle_duffle 2y agoHonestly, I think Branch Education does a solid job with topics like this. Yes, it glosses over details and sometimes simplifies to the point of oversights, but that’s often necessary in educational content to avoid getting bogged down. It’s a balance: if you dive too deep, you risk losing the main points. Branch Education is designed to introduce complex concepts, often for high schoolers or newcomers to the subject. Even my first grader finds it interesting because it’s visually engaging and conveys a general understanding, even if much of the terminology goes over their head. Their video on how computer chips are made, for example, managed to hold the whole family’s attention. That is hard to do for most of the nerdy shit I watch on YouTube! It’s not meant to be a deep dive—Ben Eater is better suited for that. His work on instruction counters, registers, and the intricacies of “how CPUs work” is incredible, but it’s for a different audience. You need a fair amount of computer science and electrical engineering knowledge to get the most out of his content. Good luck getting my family to watch him breadboard an entire graphics system; it’s fascinating but requires a serious commitment to appreciate fully.
- cgreerrun 2y agoThe only time I really feel like I understand something is by building something with it. So actually writing a CUDA kernel to do grayscale conversion and then tweaking the code. BUT... both the video and the article are useful before you do that. They both allow you to build a mental model of how GPUs work that you can test later on.
- ChrisArchitect 2y ago(2023) Some discussion then: https://news.ycombinator.com/item?id=37967126 https://news.ycombinator.com/item?id=37967126
- winwang 2y agoMakes me consider writing a post on misconceptions of GPU computing, such as requiring the problem to be fully data-parallel.
- etdeagle2 2y agoPlease do! I would love to read about it. I have been playing with GPU hash tables shared between all the threads using locking (Interlocked.CompareExchange) and such in a Compute Shader. I have been wondering if there are better ways than locking.
- jms55 2y agoHash tables on GPUs are cool. You can use them for meshless radiance caches, that automatically (for better or for worse) adapt to the surrounding geometry.
- winwang 2y agoUntil I get around to it, there's this blog post I wrote: https://wiwa.substack.com/p/can-we-10x-rust-hashmap-throughput https://wiwa.substack.com/p/can-we-10x-rust-hashmap-throughp... Not an "expert" on GPU hash tables, but I'm mildly aware of other hashmap things like a novel(?) hash-indexed sorted array (HISA): https://arxiv.org/html/2311.02206v3 https://arxiv.org/html/2311.02206v3
- jms55 2y agoIn my opinion, the biggest misconception around GPUs I see people have is that they don't realize it's an entirely separate device with it's own memory, compiler, scheduler, etc. You don't call functions to tell the GPU what to do - you record commands to a buffer, that the GPU later executes at some indeterminate point. When you call dispatch/draw(), nothing actually happens yet. Another kind of misconception: data transfer is a _really_ overlooked issue. People think "oh this is a parallel problem, I can have the GPU do it" and completely discount the cost to send the data to the GPU, and then get it back. If you want to write 20mb of data to a buffer, that's not just a memcpy, all that data has to go over the PCIe buss to the GPU (which again, is a completely separate device unless you're using an iGPU), and that's going to be expensive (in real time contexts). Similarly if you want to read a whole large buffer of results back from the GPU, that's going to take some time.
- Miniminix 2y agoIf you are not writing the GPU kernel, just use a high level language which wraps up the CUDA, Metal, or whatever. https://julialang.org https://julialang.org https://juliagpu.org https://juliagpu.org
- schmidtleonard 2y agoThe big problem I've had historically with non-native CUDA wrappers is that they always seem to omit or bug some feature that is critical for my application, and the amount of debugging pain and implementation or bugfix work to get around this problem exceeds the effort "savings" of a high level interface by an order of magnitude or three.
- esperent 2y agoUnrelated but I absolutely love this reply from the previous time this was posted and someone complained about the line "most programmmers ...": > Try this on: "A non-trivial number of Computer Scientists, Computer Engineers, Electrical Engineers, and hobbyists have ..." > Took some philosophy courses for fun in college. I developed a reading skill there that lets me forgive certain statements by improving them instead of dismissing them. My brain now automatically translates over-generalizations and even outright falsehoods into rationally-nearby true statements. As the argument unfolds, those ideas are reconfigured until the entire piece can be evaluated as logically coherent. > The upshot is that any time I read a crappy article, I'm left with a new batch of true and false premises or claims about topics I'm interested in. And thus my mental world expands. https://news.ycombinator.com/item?id=37969305 https://news.ycombinator.com/item?id=37969305
- ImHereToVote 2y agoThat's beautiful
- dmichulke 2y agoI know this process as steelmanning. https://umbrex.com/resources/tools-for-thinking/what-is-steelmanning/ https://umbrex.com/resources/tools-for-thinking/what-is-stee... If you see people doing this, befriend them because it means they're valuing knowledge higher than their ego.
- aquariusDue 2y agoKinda reminds me of Rogerian rhetoric: https://en.wikipedia.org/wiki/Rogerian_argument https://en.wikipedia.org/wiki/Rogerian_argument I suppose it always depends on the goals of the conversation and the participants' greater view of the world. Your advice also seems similar to the sage wisdom along the lines of "It's the mark of an educated mind to entertain a thought without accepting it" but goes one step further. I generally agree with you, it truly is rare to find people who can put aside their ego for the pursuit of a higher (or common) goal.
- amelius 2y ago
- BillLucky 2y agogreat
- amelius 2y agoTitle reminds of: https://people.freebsd.org/~lstewart/articles/cpumemory.pdf https://people.freebsd.org/~lstewart/articles/cpumemory.pdf
- hermitcrab 2y agoGPU are optimized for number crunching. Do they get used at all for string processing? I ask because I develop data wrangling software and most of it is string processing (joins, concatenations, aggregations, filtering etc), rather than numerical.
- TheDudeMan 2y agoDo you have millions of strings that need to be manipulated in the same way at the same time?
- hermitcrab 2y agoYes. For example, you might want to change a column of 10 million strings from upper case to lower case. Or concatenate 2 columns to create a third column. It is not clear to me this would be any faster on a GPU.
- hermitcrab 2y agoAlso, you might want to create a hash table from a million values in a column, so you can use this for a join.
- deleted 2y ago[deleted]