3 ms·

>That's simply never going to fit on a 8GB, 40GB, or even 80GB high-end GPU. The only way to work with that kind of data is to think about how to split it up, a

>That's simply never going to fit on a 8GB, 40GB, or even 80GB high-end GPU. The only way to work with that kind of data is to think about how to split it up, and have the CPU store lots of the data, while the GPU processes pieces of the data in parallel. There is always a problem size that does not fit into memory. Whether that memory is the GPU memory, or the CPU memory, doesn't really matter. We have been solving this problem for 60 years already. It isn't rocket science. --- The CPU doesn't have to do anything. The GPU can map a file stored on hard disk to VRAM memory, do random access into it, process chunks of it, write the results into network sockets and send them over the network, etc. The only thing the CPU has to do is launch a kernel: int main(args...) { main_kernel<<<...>>>(args...); synchronize(); return 0; } and this is a relatively accurate depiction of how the "main function" of the two latests apps I've worked on look like: the GPU does everything. --- > However, the famous "Moana" scene for Disney-level productions is a 93GB (!!!!) scene statically, with another 131GBs [...] That's simply never going to fit on a high-end GPU. LOL. V100 with 32Gbs and 8x per rack gave you 256 Gb of VRAM addressable from any GPU in the rack. A100 with 80GB and 16x per rack give you 1.3 TB of VRAM addressable from any GPU in the rack. You can fit Moana in GPU VRAM in a now old DGX-2. If you are willing to bet cash on Moana never fitting on a single GPU, I'd take you on that bet. Sounds like free money to me.

dragontamer 6y ago
I'll post the link again: https://www.render-blog.com/2020/10/03/gpu-motunui/ https://www.render-blog.com/2020/10/03/gpu-motunui/ This person rendered the Moana scene on just 8GBs of GPU VRAM. It does this by rendering 6.7GB chunks at a time on the GPU, with the CPU keeping the RAM-heavy "big picture" in mind. (EDITED paragraph. First wording of this paragraph was poor). ------ Its not that these problems "cannot be solved", its that these problems "become grossly more complicated" when under RAM / VRAM constraints. They're still solvable, but now you have to do strange techniques. ------ With regards to a Ray-tracer, tracing the ray-of-light that's bouncing around could theoretically touch ANY of the 93GBs of static object data (which could have been shifted by any of the 131 GBs of animation data). That is to say: a ray that bounces off of a any leaf on any tree could bounce in any direction, hitting potentially any other geometry in the scene. That pretty much forces you to keep the geometry in high-speed RAM, and not do an I/O cycle between each ray-bounce. As a rough reminder of the target performance: Raytracers aim at ~30 million to 30-billion ray-bounces per second, depending on movie-grade vs video-game optimized. Either way, that level of performance is really only ever going to be solved by keeping all of the geometry data in RAM. > A100 with 80GB and 16x per rack give you 1.3 TB of VRAM addressable from any GPU in the rack. That doesn't mean it makes sense to traverse a BVH-tree across a relatively high-latency NVLink connection off-chip. I know GPUs have decent latency hiding but... that's a lot of latency to hide. Again: your CPU-renderers can hit 10s of millions of rays per second. I'm not sure if you're gonna get something pragmatic by just dropping the entire geometry into distributed NVSwitch'd memory and hoping for the best. Honestly, that's where the 8GB CPU+GPU team becomes interesting to me. A methodology for clearly separating the geometry and splitting up which local compute-devices are responsible for handling which rays is going to scale better than a naive dump reliant on remote-connections pretending to be RAM. Video games hit Billions of rays/second. The promise of GPU-compute is on that order, and I just doubt that remote RAM accesses over NVLink will get you there. > If you are willing to bet cash on Moana never fitting on a single GPU, I'd take you on that bet. Sounds like free money to me. The issue is not Moana (or other movies from 2016), the issue are the movies that will be made in 2022 and into the future. Especially if they're near photorealistic like Marvel-movies or Star Wars. ---------- The other problem is: what's cheaper? A DGX-system could very well be faster than one CPU system. But would it be faster than a cluster of Ice-Lake Xeons with AVX512 each with the precise amount of RAM needed for the problem? (Ex: 512GBs in some hypothetical future movie?) A team probably would be better: CPUs have expandable RAM, that's their biggest advantage. GPUs have fixed RAM. Slicing the problem up so that pieces of Raytracing fits on GPUs, while the other, "bulkier" bits fit on CPU DDR4 (or DDR5), would probably be the most cost-efficient way at solving the raytracing problem. The GPU-Moana experiment showed that "collecting rays that bounce outside of RAM" is an efficient methodology. Slice the scene into 8GB chunks, process the rays that are within that chunk, and the collate the rays together to find where the rays go.
- 4paceman 6y ago
  These convos are super interesting to me as a guy building his own poor-man's render farm, and validate some of what I'd heard elsewhere. You do need a huge amount of memory for some scenes, and while the raytracing GPU in my notebook is great for modeling, it doesn't take many 4k/8k textures to run out/crash. I'm not savy enough to work only in GPU and like you say we don't know where a light bounce is headed anyway right. Even with sufficiently small scenes, GPUs and systems to run them are expensive when you can buy 10 year old rackmount servers with 12 cores @ 3GHz and loaded with cheap old memory for less than $200 a pop total. At this point the shipping might cost more than the shipped. I just bought a lot of old servers from a local university for a better deal than that, half junkyard bound they are so ancient.

dragontamer 6y ago

I'll post the link again: https://www.render-blog.com/2020/10/03/gpu-motunui/ https://www.render-blog.com/2020/10/03/gpu-motunui/ This person rendered the Moana scene on just 8GBs of GPU VRAM. It does this by rendering 6.7GB chunks at a time on the GPU, with the CPU keeping the RAM-heavy "big picture" in mind. (EDITED paragraph. First wording of this paragraph was poor). ------ Its not that these problems "cannot be solved", its that these problems "become grossly more complicated" when under RAM / VRAM constraints. They're still solvable, but now you have to do strange techniques. ------ With regards to a Ray-tracer, tracing the ray-of-light that's bouncing around could theoretically touch ANY of the 93GBs of static object data (which could have been shifted by any of the 131 GBs of animation data). That is to say: a ray that bounces off of a any leaf on any tree could bounce in any direction, hitting potentially any other geometry in the scene. That pretty much forces you to keep the geometry in high-speed RAM, and not do an I/O cycle between each ray-bounce. As a rough reminder of the target performance: Raytracers aim at ~30 million to 30-billion ray-bounces per second, depending on movie-grade vs video-game optimized. Either way, that level of performance is really only ever going to be solved by keeping all of the geometry data in RAM. > A100 with 80GB and 16x per rack give you 1.3 TB of VRAM addressable from any GPU in the rack. That doesn't mean it makes sense to traverse a BVH-tree across a relatively high-latency NVLink connection off-chip. I know GPUs have decent latency hiding but... that's a lot of latency to hide. Again: your CPU-renderers can hit 10s of millions of rays per second. I'm not sure if you're gonna get something pragmatic by just dropping the entire geometry into distributed NVSwitch'd memory and hoping for the best. Honestly, that's where the 8GB CPU+GPU team becomes interesting to me. A methodology for clearly separating the geometry and splitting up which local compute-devices are responsible for handling which rays is going to scale better than a naive dump reliant on remote-connections pretending to be RAM. Video games hit Billions of rays/second. The promise of GPU-compute is on that order, and I just doubt that remote RAM accesses over NVLink will get you there. > If you are willing to bet cash on Moana never fitting on a single GPU, I'd take you on that bet. Sounds like free money to me. The issue is not Moana (or other movies from 2016), the issue are the movies that will be made in 2022 and into the future. Especially if they're near photorealistic like Marvel-movies or Star Wars. ---------- The other problem is: what's cheaper? A DGX-system could very well be faster than one CPU system. But would it be faster than a cluster of Ice-Lake Xeons with AVX512 each with the precise amount of RAM needed for the problem? (Ex: 512GBs in some hypothetical future movie?) A team probably would be better: CPUs have expandable RAM, that's their biggest advantage. GPUs have fixed RAM. Slicing the problem up so that pieces of Raytracing fits on GPUs, while the other, "bulkier" bits fit on CPU DDR4 (or DDR5), would probably be the most cost-efficient way at solving the raytracing problem. The GPU-Moana experiment showed that "collecting rays that bounce outside of RAM" is an efficient methodology. Slice the scene into 8GB chunks, process the rays that are within that chunk, and the collate the rays together to find where the rays go.

4paceman 6y ago
These convos are super interesting to me as a guy building his own poor-man's render farm, and validate some of what I'd heard elsewhere. You do need a huge amount of memory for some scenes, and while the raytracing GPU in my notebook is great for modeling, it doesn't take many 4k/8k textures to run out/crash. I'm not savy enough to work only in GPU and like you say we don't know where a light bounce is headed anyway right. Even with sufficiently small scenes, GPUs and systems to run them are expensive when you can buy 10 year old rackmount servers with 12 cores @ 3GHz and loaded with cheap old memory for less than $200 a pop total. At this point the shipping might cost more than the shipped. I just bought a lot of old servers from a local university for a better deal than that, half junkyard bound they are so ancient.