4 ms·
Very true! I'd also go for similar optimizations when processing texts or sparse linear algebra on Nvidia and AMD GPUs. You only have ~50 KB of constant memory,
by ashvardanian 2y ago
Very true! I'd also go for similar optimizations when processing texts or sparse linear algebra on Nvidia and AMD GPUs. You only have ~50 KB of constant memory, ~50 MB of shared memory, and ~50 GB of global memory. It is BIG compared to microcontrollers but very little compared to the scope of problems often solved on GPUs. So many optimizations revolve around compressed representations and coalesced memory accesses.
I am still looking for a short example of such CUDA kernels, and I would love to see more embedded examples if you have thoughts ;)
- boricj 2y agoI haven't had to reach for them so far either professionally or personally, but custom memory allocators (slab allocation, bump allocator...) and allocation strategies is something I've been meaning to look into. Too bad that the one game I've done reverse-engineering on used dynamic memory allocation for just about everything, with an allocator that uses a singly-linked list of used/free chunks that wouldn't look out of place in the 1980s. I'm aware that the C++ standard library has polymorphic allocators alongside a couple of memory resource implementations. I've also heard that the dynamic dispatch for the polymorphic allocators could bring some optimization or speed penalties compared to a statically dispatched allocator or the standard std::allocator that uses operator new(), but I have no concrete data to judge either way.