5 ms·

The main advantage of this is the ability to have insane amounts of cache memory. For a benchmark like rendering, where AMD currently leads due to having more

The main advantage of this is the ability to have insane amounts of cache memory. For a benchmark like rendering, where AMD currently leads due to having more cores (each with its own cache), having 512mb cache might be a 2x to 10x in performance. Its the same for deep learning where convolutions are heavily used and cache misses and memory bandwidth become the major bottlenecks. Intel already has the world's best tools for analyzing and fixing pipeline stalls. But that doesn't help if you have to wait for the relatively slow main memory. So this technology could massively increase the value of Intel's existing development tools and leapfrog performance per watt for rendering, a highly profitable market if you look at the CGI budgets of recent movies.

CyberDildonics 6y ago
This is wild speculation from hope and hype. First, all CPUs have local L1 and L2 cache in addition to some sort of shared L3 cache. If you mean 512 megabits, that has already been done, if you mean 512 megabytes, I don't think there is any indication that expensive and power intensive cache will suddenly made a giant leap in size. Don't forget that chips already have around 40% of their transistors dedicated to cache. Intel's given numbers for this look great, but they don't add up to what you are saying. Caches for graphics running on the CPU are going to help mostly unoptimized software or very optimized software that works with contiguous memory chunks that will fit in cache, thus saving memory bandwidth (which looks like the biggest breakthrough here). Caches are only going to do so much for jumping around in memory. > So this technology could massively increase the value of Intel's existing development tools and leapfrog performance per watt for rendering, a highly profitable market if you look at the CGI budgets of recent movies. I don't think this has anything to do with Intel's development tools. There is no indication this would result in "leapfrogging performance per watt". Cache needs significant power too. Movie CG budgets are almost all based on the number of hours clocked in making them. The hardware, including power and cooling is proportionately a small part of the budget, usually around 10%-15%.
- fxtentacle 6y ago
  From what I heard, feature films regularly spend $1 mio+ on renting CPU power alone. And yes, while caches won't help for jumping around in memory, rendering is heavily optimized to not jump around. It's the same for AI convolution. While the inner loop might not fit into a 128 MB cache, it would fit into 512 MB cache. That would cause the memory bandwidth to raise from ~20 GB/s to ~500 GB/s. So if memory bandwidth is the limiting factor - and it is for many convolution and rendering workloads - then a 10x speed-up is possible. Oh and the development tools I'm referring to is V-Tune & Amplify, i.e. the Intel tools to make sure your loop is using all the tricks to avoid pipeline stalls so that RAM bandwidth becomes the limiting factor.
  - CyberDildonics 6y ago
    > From what I heard, feature films regularly spend $1 mio+ on renting CPU power alone. Renting CPU power is typically an indication that rendering was severely underestimated and a million USD would be an extreme example. That money could buy multiple racks that would be in use for four or five years. > rendering is heavily optimized to not jump around. It's the same for AI convolution. I wish that were true, but with the exception of Disney's internal renderer only a few small parts of most renderers end up architected for this. Don't forget that most actual production renderers are multiple decades old. > While the inner loop might not fit into a 128 MB cache, it would fit into 512 MB cache. That would cause the memory bandwidth to raise from ~20 GB/s to ~500 GB/s. This doesn't really make sense. Instructions are very small and the data they operate on are very large. Each core will be doing different things to different data and the total L3 cache will be used by all of them. I'm not sure how you are getting these numbers, but they don't seem grounded in reality of real software and lots of cores. L3 cache typically has bandwidth in the same order of magnitude as memory, while Intel is claiming to increase that by 3x in the article. > So if memory bandwidth is the limiting factor - and it is for many convolution and rendering workloads - then a 10x speed-up is possible. This is very far from the truth, especially for rendering. Most renderers hop around him memory to trace rays through acceleration structures, actual geometry, geometry attributes and texture lookups. Also I'm not sure how 3x the cache bandwidth translates to 10x program speed in any scenario. > Oh and the development tools I'm referring to is V-Tune & Amplify, i.e. the Intel tools to make sure your loop is using all the tricks to avoid pipeline stalls so that RAM bandwidth becomes the limiting factor. Yes, those exist, but they don't have anything to do with the article or cache sizes.

CyberDildonics 6y ago

This is wild speculation from hope and hype. First, all CPUs have local L1 and L2 cache in addition to some sort of shared L3 cache. If you mean 512 megabits, that has already been done, if you mean 512 megabytes, I don't think there is any indication that expensive and power intensive cache will suddenly made a giant leap in size. Don't forget that chips already have around 40% of their transistors dedicated to cache. Intel's given numbers for this look great, but they don't add up to what you are saying. Caches for graphics running on the CPU are going to help mostly unoptimized software or very optimized software that works with contiguous memory chunks that will fit in cache, thus saving memory bandwidth (which looks like the biggest breakthrough here). Caches are only going to do so much for jumping around in memory. > So this technology could massively increase the value of Intel's existing development tools and leapfrog performance per watt for rendering, a highly profitable market if you look at the CGI budgets of recent movies. I don't think this has anything to do with Intel's development tools. There is no indication this would result in "leapfrogging performance per watt". Cache needs significant power too. Movie CG budgets are almost all based on the number of hours clocked in making them. The hardware, including power and cooling is proportionately a small part of the budget, usually around 10%-15%.

fxtentacle 6y ago
From what I heard, feature films regularly spend $1 mio+ on renting CPU power alone. And yes, while caches won't help for jumping around in memory, rendering is heavily optimized to not jump around. It's the same for AI convolution. While the inner loop might not fit into a 128 MB cache, it would fit into 512 MB cache. That would cause the memory bandwidth to raise from ~20 GB/s to ~500 GB/s. So if memory bandwidth is the limiting factor - and it is for many convolution and rendering workloads - then a 10x speed-up is possible. Oh and the development tools I'm referring to is V-Tune & Amplify, i.e. the Intel tools to make sure your loop is using all the tricks to avoid pipeline stalls so that RAM bandwidth becomes the limiting factor.
- CyberDildonics 6y ago
  > From what I heard, feature films regularly spend $1 mio+ on renting CPU power alone. Renting CPU power is typically an indication that rendering was severely underestimated and a million USD would be an extreme example. That money could buy multiple racks that would be in use for four or five years. > rendering is heavily optimized to not jump around. It's the same for AI convolution. I wish that were true, but with the exception of Disney's internal renderer only a few small parts of most renderers end up architected for this. Don't forget that most actual production renderers are multiple decades old. > While the inner loop might not fit into a 128 MB cache, it would fit into 512 MB cache. That would cause the memory bandwidth to raise from ~20 GB/s to ~500 GB/s. This doesn't really make sense. Instructions are very small and the data they operate on are very large. Each core will be doing different things to different data and the total L3 cache will be used by all of them. I'm not sure how you are getting these numbers, but they don't seem grounded in reality of real software and lots of cores. L3 cache typically has bandwidth in the same order of magnitude as memory, while Intel is claiming to increase that by 3x in the article. > So if memory bandwidth is the limiting factor - and it is for many convolution and rendering workloads - then a 10x speed-up is possible. This is very far from the truth, especially for rendering. Most renderers hop around him memory to trace rays through acceleration structures, actual geometry, geometry attributes and texture lookups. Also I'm not sure how 3x the cache bandwidth translates to 10x program speed in any scenario. > Oh and the development tools I'm referring to is V-Tune & Amplify, i.e. the Intel tools to make sure your loop is using all the tricks to avoid pipeline stalls so that RAM bandwidth becomes the limiting factor. Yes, those exist, but they don't have anything to do with the article or cache sizes.