4 ms·

Isn't it more efficient to use their GPU instances for rendering?

dragontamer 11y ago
Depends. If it costs you 15 full time developers 6 months to transfer your x86 rendering code to port to OpenCL (or CUDA), then maybe spending $100,000 on hardware is cheaper than spending $700,000+ on developers.
- lsadam0 11y ago
  Let me qualify, this isn't really my area, so I'm a bit out of my element. If we are talking rasterization, isn't an OpenCL/CUDA version pretty much always going to stomp an x86 implementation? While on the ray tracing side, there's little difference? I'm just curious because my first reaction was that "sure the dev cost is initially much higher, but how much time is saved on rendering in the long run?".
  - bhouston 11y ago
    GPU is maturing as we speak. It isn't yet used on the high end though because CPU-based render machines often have +32GB of ram to store the scenes, where as high end GPUs may have 12GB of ram. But you are correct that GPUs is the future of rendering. :)
  - dragontamer 11y ago
    > If we are talking rasterization, isn't an OpenCL/CUDA version pretty much always going to stomp an x86 implementation? While on the ray tracing side, there's little difference? GPU will stomp CPU on memory-bandwidth alone (GDDR5 RAM is much much faster than DDR3... and when DDR4 catches up in the coming years all the GPUs will switch to HBM and still retain a massive memory bandwidth advantage). CPU wins due to legacy code and access to huge memory. In practice, RAM Requirements mean at best, it'd be a OpenCL/CUDA version PLUS CPU to handle the large scenes. I'm sure the OpenCL / CUDA version will be cheaper from a hardware perspective, but once again... developers are very expensive. A decent software development team runs in the millions of $$ / year in costs, let alone a team of GPU-specialist high-performance programmers. Its things like this why Intel's Knight's Landing GPU chip (which runs x86) actually have a valid market case. Sure, Knights Landing is slower, weaker, and more expensive than the competition. But that x86 compatibility is a major feature that saves developer's time. Also, when you start doing GPU + CPU computations, the PCIe latency starts to become a major issue. Tasks may complete faster on the CPU than CPU+GPU, because the task is in CPU L1 or L2 Cache. While if you do CPU+GPU, you push the code out the PCIe and then synchronize the CPU+GPU... then the GPU starts pulling the data into its cache. When the GPU is done the task, its pushed over PCIe AGAIN. So its a bit tricky task to balance the memory issues and cooperation across cores.
    - bhouston 11y ago
      Just be careful about the Knights Landing GPU, it could be cancelled at any moment as I think it never got adoption outside of a few supercomputer installations.
      - berkut 11y ago
        It hasn't been released yet has it? Take up should be much better, as it supports much more memory and full x86 instructions.
    - scott_s 11y ago
      I just want to support one of your points: for some applications, the PCIe latency kills the potential of using GPUs. It doesn't matter how fast the GPU is at a computation if the time to transfer the data is longer than it would take to do the calculation on the host CPU. I published a paper with this conclusion a while back ("Evaluation of Streaming Aggregation on Parallel Hardware Architectures", http://www.scott-a-s.com/files/debs2010.pdf http://www.scott-a-s.com/files/debs2010.pdf). My paper was basically, "GPUs are awful for our problem, so let's compare to CPUs and Cell". A more recent paper from NSDI 2015 from a group at CMU, with Anuj Kalia as the first author, that also looks into it, "Raising the Bar for Using GPUs in Software Packet Processing", http://www.cs.cmu.edu/~akalia/doc/nsdi15/gopt_nsdi.pdf http://www.cs.cmu.edu/~akalia/doc/nsdi15/gopt_nsdi.pdf. They look at just the GPU and CPU part of the spectrum - which is probably more relevant to most, as the Cell is a dead technology now. (I think my conclusions will matter more when on-chip GPUs become more prevalent, but we're not there yet). Their paper starts with an excellent analysis of the latencies for transferring data to GPUs, and comparing end-to-end performance against several different algorithms, where they vary the amount of data reuse. My consistent gripe when talking about improving application performance by using accelerators is that one must take data copying costs into consideration, which means we must use application end-to-end performance as our metric. There's a temptation to focus on just the computational kernel performance. The Kalia et al. paper is the best treatment of this subject that I have seen.
      - dragontamer 11y ago
berkut 11y ago
For feature film, CPU still largely rules the roost, thanks to insane memory requirements and the fact that properly SSE'd/AVX'd code isn't THAT much slower than GPUs for highly parallel stuff like raytracing when you've got multiple CPU sockets.
dagw 11y ago
GPU's still only have a quite small amounts of memory, that greatly limits the type of work they can do.
- bhouston 11y ago
  High end GPUs can have up to 12GB of ram now, which I have to admit is starting to get reasonable. Just a few more doublings and you've got what high end render machines have, which I understand is in the range of 64GB to 128GB. Although I've heard that some simulation oriented machines have 256GB to 512GB.
bhouston 11y ago
A lot of the time yes, but a lot of the high-end renderers used in production are still CPU based (anything from Dreamworks, ILM, Disney or Pixar are all using CPU-based renders for the most part.) It is a lot of work to convert some of the global illumination calculations that are high quality (not special case) to run on GPUs fast. That is the main issue. We support real-time high-end GPU-based rendering as well: https://clara.io/view/b43f3215-c9ef-488f-a55f-1bf2a7d74f3f/webgl https://clara.io/view/b43f3215-c9ef-488f-a55f-1bf2a7d74f3f/w... https://clara.io/view/193070f2-e8af-4afc-a531-9d82338b5288/webgl https://clara.io/view/193070f2-e8af-4afc-a531-9d82338b5288/w... https://clara.io/view/a558dca2-8c2f-432c-ab34-9135c3066010/webgl https://clara.io/view/a558dca2-8c2f-432c-ab34-9135c3066010/w...

Depends. If it costs you 15 full time developers 6 months to transfer your x86 rendering code to port to OpenCL (or CUDA), then maybe spending $100,000 on hardware is cheaper than spending $700,000+ on developers.

lsadam0 11y ago
Let me qualify, this isn't really my area, so I'm a bit out of my element. If we are talking rasterization, isn't an OpenCL/CUDA version pretty much always going to stomp an x86 implementation? While on the ray tracing side, there's little difference? I'm just curious because my first reaction was that "sure the dev cost is initially much higher, but how much time is saved on rendering in the long run?".
- bhouston 11y ago
  GPU is maturing as we speak. It isn't yet used on the high end though because CPU-based render machines often have +32GB of ram to store the scenes, where as high end GPUs may have 12GB of ram. But you are correct that GPUs is the future of rendering. :)
- dragontamer 11y ago
  > If we are talking rasterization, isn't an OpenCL/CUDA version pretty much always going to stomp an x86 implementation? While on the ray tracing side, there's little difference? GPU will stomp CPU on memory-bandwidth alone (GDDR5 RAM is much much faster than DDR3... and when DDR4 catches up in the coming years all the GPUs will switch to HBM and still retain a massive memory bandwidth advantage). CPU wins due to legacy code and access to huge memory. In practice, RAM Requirements mean at best, it'd be a OpenCL/CUDA version PLUS CPU to handle the large scenes. I'm sure the OpenCL / CUDA version will be cheaper from a hardware perspective, but once again... developers are very expensive. A decent software development team runs in the millions of $$ / year in costs, let alone a team of GPU-specialist high-performance programmers. Its things like this why Intel's Knight's Landing GPU chip (which runs x86) actually have a valid market case. Sure, Knights Landing is slower, weaker, and more expensive than the competition. But that x86 compatibility is a major feature that saves developer's time. Also, when you start doing GPU + CPU computations, the PCIe latency starts to become a major issue. Tasks may complete faster on the CPU than CPU+GPU, because the task is in CPU L1 or L2 Cache. While if you do CPU+GPU, you push the code out the PCIe and then synchronize the CPU+GPU... then the GPU starts pulling the data into its cache. When the GPU is done the task, its pushed over PCIe AGAIN. So its a bit tricky task to balance the memory issues and cooperation across cores.
  - bhouston 11y ago
    Just be careful about the Knights Landing GPU, it could be cancelled at any moment as I think it never got adoption outside of a few supercomputer installations.
    - berkut 11y ago
      It hasn't been released yet has it? Take up should be much better, as it supports much more memory and full x86 instructions.
  - scott_s 11y ago
    I just want to support one of your points: for some applications, the PCIe latency kills the potential of using GPUs. It doesn't matter how fast the GPU is at a computation if the time to transfer the data is longer than it would take to do the calculation on the host CPU. I published a paper with this conclusion a while back ("Evaluation of Streaming Aggregation on Parallel Hardware Architectures", http://www.scott-a-s.com/files/debs2010.pdf http://www.scott-a-s.com/files/debs2010.pdf). My paper was basically, "GPUs are awful for our problem, so let's compare to CPUs and Cell". A more recent paper from NSDI 2015 from a group at CMU, with Anuj Kalia as the first author, that also looks into it, "Raising the Bar for Using GPUs in Software Packet Processing", http://www.cs.cmu.edu/~akalia/doc/nsdi15/gopt_nsdi.pdf http://www.cs.cmu.edu/~akalia/doc/nsdi15/gopt_nsdi.pdf. They look at just the GPU and CPU part of the spectrum - which is probably more relevant to most, as the Cell is a dead technology now. (I think my conclusions will matter more when on-chip GPUs become more prevalent, but we're not there yet). Their paper starts with an excellent analysis of the latencies for transferring data to GPUs, and comparing end-to-end performance against several different algorithms, where they vary the amount of data reuse. My consistent gripe when talking about improving application performance by using accelerators is that one must take data copying costs into consideration, which means we must use application end-to-end performance as our metric. There's a temptation to focus on just the computational kernel performance. The Kalia et al. paper is the best treatment of this subject that I have seen.
    - dragontamer 11y ago

berkut 11y ago

For feature film, CPU still largely rules the roost, thanks to insane memory requirements and the fact that properly SSE'd/AVX'd code isn't THAT much slower than GPUs for highly parallel stuff like raytracing when you've got multiple CPU sockets.

dagw 11y ago

GPU's still only have a quite small amounts of memory, that greatly limits the type of work they can do.

bhouston 11y ago
High end GPUs can have up to 12GB of ram now, which I have to admit is starting to get reasonable. Just a few more doublings and you've got what high end render machines have, which I understand is in the range of 64GB to 128GB. Although I've heard that some simulation oriented machines have 256GB to 512GB.

bhouston 11y ago

A lot of the time yes, but a lot of the high-end renderers used in production are still CPU based (anything from Dreamworks, ILM, Disney or Pixar are all using CPU-based renders for the most part.) It is a lot of work to convert some of the global illumination calculations that are high quality (not special case) to run on GPUs fast. That is the main issue. We support real-time high-end GPU-based rendering as well: https://clara.io/view/b43f3215-c9ef-488f-a55f-1bf2a7d74f3f/webgl https://clara.io/view/b43f3215-c9ef-488f-a55f-1bf2a7d74f3f/w... https://clara.io/view/193070f2-e8af-4afc-a531-9d82338b5288/webgl https://clara.io/view/193070f2-e8af-4afc-a531-9d82338b5288/w... https://clara.io/view/a558dca2-8c2f-432c-ab34-9135c3066010/webgl https://clara.io/view/a558dca2-8c2f-432c-ab34-9135c3066010/w...