3 ms·

I am pretty sure this is just a gimmick. I would not call libc code in a GPU kernel. It would mean dragging in a whole bunch of stuff I don't want, and cant con

I am pretty sure this is just a gimmick. I would not call libc code in a GPU kernel. It would mean dragging in a whole bunch of stuff I don't want, and cant control or pick-and-choose. That makes sense for regular processes on the CPU; it does _not_ make sense in code you run millions of times in GPU threads. I see people saying they've "dreamed" of this or have waited so long for this to happen... well, my friends, you should not have; and I'm afraid you're in for a disappointment.

JonChesterfield 2y ago
It uses a few mb of contiguous shared memory and periodically calling a function from a host thread. Unless you only want sprintf or similar in which case neither is needed. The unused code deadstrips pretty well. Won't help compilation time. Generally you don't want libc calls in numerical kernels doing useful stuff - the most common request was for printf as a debugging crutch, I mostly wanted mmap.
- einpoklum 2y ago
  > It uses a few mb of contiguous shared memory and periodically calling a function from a host thread. Well, if someone wants to shoot themselves in the head, then by all means... > Unless you only want sprintf or similar in which case neither is needed. > ... Generally you don't want libc calls in numerical kernels doing useful > stuff - the most common request was for printf as a debugging crutch I have actually adapted a library for that particular case: https://github.com/eyalroz/printf/ https://github.com/eyalroz/printf/ I started with a standalone printf-family implementation targetting embedded devices, and (among other things) adapted it for use also with CUDA. > I mostly wanted mmap. Does it really make sense to make a gazillion mmap calls from the threads of your GPU kernel? I mean, is it really not always better to mmap on the CPU side? At most, I might do it asynchronously using a CUDA callback or some other mechanism. But I will admit I've not had that use-case.
  - rowanG077 2y ago
    There is a ton of code that is sections of massive parallelism interspersed with sections of serial non-hot code. It makes total sense to do some syscall then and continue to the next section.
imtringued 2y ago
I'm not sure what disappointment you're predicting. Unless your GPU is connected through a cache coherent protocol like CXL to your CPU, you are unlikely to make your code run faster by transferring data back to the CPU and back again to the GPU. You have 128 compute units on the 4090, even at a lower frequency and higher memory latency, you will probably not end up too far away from the performance of an 8 core CPU running at 4.5GHz. Nobody is running millions of CPU threads in the first place, so you seem to be completely misunderstanding the workload here. Nobody wants to speed up their CPU code by running it on the GPU, they want to stop slowing down their GPU code by waiting for data transfers to and from the CPU.

JonChesterfield 2y ago

It uses a few mb of contiguous shared memory and periodically calling a function from a host thread. Unless you only want sprintf or similar in which case neither is needed. The unused code deadstrips pretty well. Won't help compilation time. Generally you don't want libc calls in numerical kernels doing useful stuff - the most common request was for printf as a debugging crutch, I mostly wanted mmap.

einpoklum 2y ago
> It uses a few mb of contiguous shared memory and periodically calling a function from a host thread. Well, if someone wants to shoot themselves in the head, then by all means... > Unless you only want sprintf or similar in which case neither is needed. > ... Generally you don't want libc calls in numerical kernels doing useful > stuff - the most common request was for printf as a debugging crutch I have actually adapted a library for that particular case: https://github.com/eyalroz/printf/ https://github.com/eyalroz/printf/ I started with a standalone printf-family implementation targetting embedded devices, and (among other things) adapted it for use also with CUDA. > I mostly wanted mmap. Does it really make sense to make a gazillion mmap calls from the threads of your GPU kernel? I mean, is it really not always better to mmap on the CPU side? At most, I might do it asynchronously using a CUDA callback or some other mechanism. But I will admit I've not had that use-case.
- rowanG077 2y ago
  There is a ton of code that is sections of massive parallelism interspersed with sections of serial non-hot code. It makes total sense to do some syscall then and continue to the next section.

imtringued 2y ago

I'm not sure what disappointment you're predicting. Unless your GPU is connected through a cache coherent protocol like CXL to your CPU, you are unlikely to make your code run faster by transferring data back to the CPU and back again to the GPU. You have 128 compute units on the 4090, even at a lower frequency and higher memory latency, you will probably not end up too far away from the performance of an 8 core CPU running at 4.5GHz. Nobody is running millions of CPU threads in the first place, so you seem to be completely misunderstanding the workload here. Nobody wants to speed up their CPU code by running it on the GPU, they want to stop slowing down their GPU code by waiting for data transfers to and from the CPU.