5 ms·

as core count goes up the number of tasks that can be sped up starts to shrink, because memory access and throughput become more and more of a limiter. That is

as core count goes up the number of tasks that can be sped up starts to shrink, because memory access and throughput become more and more of a limiter. That is why you tend to see huge l3 caches on the big core cpus, it helps but it can still be a challenge to load up all 32 cores and get any speedup, for many workloads. l3 cache is not THAT much faster than ram. that is why I tend to eye the ~6 core machines for desktop use, higher single core mhz on them

rbanffy 8y ago
If we can balance the L1 cache eviction rate for the extra cores (since L1 is not shared), memory bandwidth pressure will be lower. These monsters also have a lot of memory channels to address memory hunger. I still think we can push the envelope a bit further for most common desktop software.
- gameswithgo 8y ago
  There is definitely room to improve how often and efficiently desktop software utilizes more cores, but I don't expect it will be common that ~32 cores gets you much gain over ~8. But when you do have a workload that can benefit, like if you know GCC will compile a huge c++ project linearly faster up to 32 cores, then it is VERY welcome!
  - rbanffy 8y ago
    If you have cores to spare, you can do incremental compilation every time you save a file. You can also do a lot of speculative work that may be thrown away but, in case it's needed, it's already done. There is no such thing as too many cores ;-)
- xxs 8y ago
  L1 cannot be too big and it should not grow any larger. L2 is a better option being 'only' 3 times slower than L1 (L1 tend to be ~3 clocks to read). L3 is an interesting concept. Communication through L3 is quite beneficial in concurrent programming when done right; other than that it's there to hide the latency of accessing RAM. However, it's shared and easy to trash. The software developers could do a lot for memory throughput hungry applications (datastructure layout, not using bloated strings for everything, not using linked lists, etc.), alas memory based optimizations almost never happen and tend to require a great deal of hardware understanding + non scripting language(s).
  - rbanffy 8y ago
    > L1 cannot be too big and it should not grow any larger. L2 is a better option L1 and L2 grow linearly with the number of cores. If you have cores to burn, pinning processes makes a lot of sense.
    - xxs 8y ago
      rbanffy, it was meant per core. Counting L1/L2 across the entire die is rather pointless. Pinning makes most sense in NUMA indeed. Nowadays my impression is that often times the multicore/socket hardware tend to be chopped down by virtual machines, though.
      - rbanffy 8y ago
        If you use VMs and don't pin VMs to cores (which is pretty sensible) yes, but if you run your processes on your metal, pinning processes will reduce pressure on their L1/L2 caches if a given process is running on it constantly, the odds the caches have the "right" data increase. The same goes for the processes that are not pinned, as they don't need to compete with the pinned one (unless they have the misfortune of being scheduled on those cores). In the end, it'll look like your cache is bigger.

rbanffy 8y ago

If we can balance the L1 cache eviction rate for the extra cores (since L1 is not shared), memory bandwidth pressure will be lower. These monsters also have a lot of memory channels to address memory hunger. I still think we can push the envelope a bit further for most common desktop software.

gameswithgo 8y ago
There is definitely room to improve how often and efficiently desktop software utilizes more cores, but I don't expect it will be common that ~32 cores gets you much gain over ~8. But when you do have a workload that can benefit, like if you know GCC will compile a huge c++ project linearly faster up to 32 cores, then it is VERY welcome!
- rbanffy 8y ago
  If you have cores to spare, you can do incremental compilation every time you save a file. You can also do a lot of speculative work that may be thrown away but, in case it's needed, it's already done. There is no such thing as too many cores ;-)
xxs 8y ago
L1 cannot be too big and it should not grow any larger. L2 is a better option being 'only' 3 times slower than L1 (L1 tend to be ~3 clocks to read). L3 is an interesting concept. Communication through L3 is quite beneficial in concurrent programming when done right; other than that it's there to hide the latency of accessing RAM. However, it's shared and easy to trash. The software developers could do a lot for memory throughput hungry applications (datastructure layout, not using bloated strings for everything, not using linked lists, etc.), alas memory based optimizations almost never happen and tend to require a great deal of hardware understanding + non scripting language(s).
- rbanffy 8y ago
  > L1 cannot be too big and it should not grow any larger. L2 is a better option L1 and L2 grow linearly with the number of cores. If you have cores to burn, pinning processes makes a lot of sense.
  - xxs 8y ago
    rbanffy, it was meant per core. Counting L1/L2 across the entire die is rather pointless. Pinning makes most sense in NUMA indeed. Nowadays my impression is that often times the multicore/socket hardware tend to be chopped down by virtual machines, though.
    - rbanffy 8y ago
      If you use VMs and don't pin VMs to cores (which is pretty sensible) yes, but if you run your processes on your metal, pinning processes will reduce pressure on their L1/L2 caches if a given process is running on it constantly, the odds the caches have the "right" data increase. The same goes for the processes that are not pinned, as they don't need to compete with the pinned one (unless they have the misfortune of being scheduled on those cores). In the end, it'll look like your cache is bigger.