2 ms·

From what I heard, feature films regularly spend $1 mio+ on renting CPU power alone. And yes, while caches won't help for jumping around in memory, rendering i

From what I heard, feature films regularly spend $1 mio+ on renting CPU power alone. And yes, while caches won't help for jumping around in memory, rendering is heavily optimized to not jump around. It's the same for AI convolution. While the inner loop might not fit into a 128 MB cache, it would fit into 512 MB cache. That would cause the memory bandwidth to raise from ~20 GB/s to ~500 GB/s. So if memory bandwidth is the limiting factor - and it is for many convolution and rendering workloads - then a 10x speed-up is possible. Oh and the development tools I'm referring to is V-Tune & Amplify, i.e. the Intel tools to make sure your loop is using all the tricks to avoid pipeline stalls so that RAM bandwidth becomes the limiting factor.

CyberDildonics 6y ago
> From what I heard, feature films regularly spend $1 mio+ on renting CPU power alone. Renting CPU power is typically an indication that rendering was severely underestimated and a million USD would be an extreme example. That money could buy multiple racks that would be in use for four or five years. > rendering is heavily optimized to not jump around. It's the same for AI convolution. I wish that were true, but with the exception of Disney's internal renderer only a few small parts of most renderers end up architected for this. Don't forget that most actual production renderers are multiple decades old. > While the inner loop might not fit into a 128 MB cache, it would fit into 512 MB cache. That would cause the memory bandwidth to raise from ~20 GB/s to ~500 GB/s. This doesn't really make sense. Instructions are very small and the data they operate on are very large. Each core will be doing different things to different data and the total L3 cache will be used by all of them. I'm not sure how you are getting these numbers, but they don't seem grounded in reality of real software and lots of cores. L3 cache typically has bandwidth in the same order of magnitude as memory, while Intel is claiming to increase that by 3x in the article. > So if memory bandwidth is the limiting factor - and it is for many convolution and rendering workloads - then a 10x speed-up is possible. This is very far from the truth, especially for rendering. Most renderers hop around him memory to trace rays through acceleration structures, actual geometry, geometry attributes and texture lookups. Also I'm not sure how 3x the cache bandwidth translates to 10x program speed in any scenario. > Oh and the development tools I'm referring to is V-Tune & Amplify, i.e. the Intel tools to make sure your loop is using all the tricks to avoid pipeline stalls so that RAM bandwidth becomes the limiting factor. Yes, those exist, but they don't have anything to do with the article or cache sizes.
- fxtentacle 6y ago
  > Don't forget that most actual production renderers are multiple decades old. I use V-Ray, and the memory bandwidth is one of the defining factors of my rendering speed. I saw a noticeable bump when I went from DDR4-1866 to DDR4-2666. > L3 cache typically has bandwidth in the same order of magnitude as memory, while Intel is claiming to increase that by 3x in the article. My 500 GB/s number was the estimated L2 bandwidth for a Ryzen CPU. > Most renderers hop around him memory to trace rays through acceleration structures There's an Intel library called Embree which is very cache friendly. It's used by v-Ray, the raytracer that I use. https://www.embree.org/ https://www.embree.org/ > Yes, those exist, but they don't have anything to do with the article or cache sizes. When I had to implement a C++ convolution algorithm, it was pretty slow at first. Then I used V-Tune and Amplify to re-order the loops, optimize memory access patterns, and identify places where AVX or SSE intrinsics would be helpful. After doing that, I could then measure with Amplify that my convolution loop is now limited in performance by cache misses, meaning exactly what a larger cache would improve. And if I remember correctly, I got only 10% pipeline utilization, which would suggest that it could be 10x faster with fast enough memory.
  - CyberDildonics 6y ago
    > There's an Intel library called Embree which is very cache friendly. It's used by v-Ray, the raytracer that I use. https://www.embree.org/ https://www.embree.org/ I started using it when it was released and I've looked through the source code. It does some operations in chunks of 4, 8 or 16 to try to utilize AVX registers, especially in the version written in ISPC. It isn't what I would call cache friendly, it still hops around a lot in memory. > And if I remember correctly, I got only 10% pipeline utilization, which would suggest that it could be 10x faster with fast enough memory. You misunderstood. You could likely get a 10x speedup if you restructured how you access memory so that it can prefetched. Memory with 10x better latency does not exist. What you are interested here is prefetching, not memory bandwidth or cache. It's great that you are interested in this stuff, but I think you need to draw a clearer line between your knowledge and your assumptions.

CyberDildonics 6y ago

> From what I heard, feature films regularly spend $1 mio+ on renting CPU power alone. Renting CPU power is typically an indication that rendering was severely underestimated and a million USD would be an extreme example. That money could buy multiple racks that would be in use for four or five years. > rendering is heavily optimized to not jump around. It's the same for AI convolution. I wish that were true, but with the exception of Disney's internal renderer only a few small parts of most renderers end up architected for this. Don't forget that most actual production renderers are multiple decades old. > While the inner loop might not fit into a 128 MB cache, it would fit into 512 MB cache. That would cause the memory bandwidth to raise from ~20 GB/s to ~500 GB/s. This doesn't really make sense. Instructions are very small and the data they operate on are very large. Each core will be doing different things to different data and the total L3 cache will be used by all of them. I'm not sure how you are getting these numbers, but they don't seem grounded in reality of real software and lots of cores. L3 cache typically has bandwidth in the same order of magnitude as memory, while Intel is claiming to increase that by 3x in the article. > So if memory bandwidth is the limiting factor - and it is for many convolution and rendering workloads - then a 10x speed-up is possible. This is very far from the truth, especially for rendering. Most renderers hop around him memory to trace rays through acceleration structures, actual geometry, geometry attributes and texture lookups. Also I'm not sure how 3x the cache bandwidth translates to 10x program speed in any scenario. > Oh and the development tools I'm referring to is V-Tune & Amplify, i.e. the Intel tools to make sure your loop is using all the tricks to avoid pipeline stalls so that RAM bandwidth becomes the limiting factor. Yes, those exist, but they don't have anything to do with the article or cache sizes.

fxtentacle 6y ago
> Don't forget that most actual production renderers are multiple decades old. I use V-Ray, and the memory bandwidth is one of the defining factors of my rendering speed. I saw a noticeable bump when I went from DDR4-1866 to DDR4-2666. > L3 cache typically has bandwidth in the same order of magnitude as memory, while Intel is claiming to increase that by 3x in the article. My 500 GB/s number was the estimated L2 bandwidth for a Ryzen CPU. > Most renderers hop around him memory to trace rays through acceleration structures There's an Intel library called Embree which is very cache friendly. It's used by v-Ray, the raytracer that I use. https://www.embree.org/ https://www.embree.org/ > Yes, those exist, but they don't have anything to do with the article or cache sizes. When I had to implement a C++ convolution algorithm, it was pretty slow at first. Then I used V-Tune and Amplify to re-order the loops, optimize memory access patterns, and identify places where AVX or SSE intrinsics would be helpful. After doing that, I could then measure with Amplify that my convolution loop is now limited in performance by cache misses, meaning exactly what a larger cache would improve. And if I remember correctly, I got only 10% pipeline utilization, which would suggest that it could be 10x faster with fast enough memory.
- CyberDildonics 6y ago
  > There's an Intel library called Embree which is very cache friendly. It's used by v-Ray, the raytracer that I use. https://www.embree.org/ https://www.embree.org/ I started using it when it was released and I've looked through the source code. It does some operations in chunks of 4, 8 or 16 to try to utilize AVX registers, especially in the version written in ISPC. It isn't what I would call cache friendly, it still hops around a lot in memory. > And if I remember correctly, I got only 10% pipeline utilization, which would suggest that it could be 10x faster with fast enough memory. You misunderstood. You could likely get a 10x speedup if you restructured how you access memory so that it can prefetched. Memory with 10x better latency does not exist. What you are interested here is prefetching, not memory bandwidth or cache. It's great that you are interested in this stuff, but I think you need to draw a clearer line between your knowledge and your assumptions.