5 ms·

I wonder what workloads would benefit from having an L4 victim cache on another CPU, but that other CPU doesn't need its own L2 cache.

_a_a_a_ 2y ago
At a guess, a single thread which benefits from as much cache as it can get.
- tedunangst 2y ago
  Sure, but having to buy entire CPUs filled with idle cores to scale up cache seems very expensive.
  - _a_a_a_ 2y ago
    I said this is where it would help (AFAICS), not that it was the best solution.
  - kev009 2y ago
    These cores are typically licensed with class/restrictions so in absolute terms yes but in the financial engineering of how the system is delivered with excess and restricted hardware no (see core types on the prior/shipping generation here https://www.ibm.com/downloads/cas/6NW3RPQV https://www.ibm.com/downloads/cas/6NW3RPQV) There are probably design reuse and RAS considerations that make it not currently worthwhile to i.e. have a distinct physical design for SAP or whatever cores.
    - sillywalk 2y ago
      I don't know if it's still the case, but in terms of RAS, the Z/Series CPUs from ~2004 had duplicated/compared instruction-fetch/decode and execution units. https://pages.cs.wisc.edu/~remzi/Classes/838/Fall2001/Papers/tale-of-two.pdf https://pages.cs.wisc.edu/~remzi/Classes/838/Fall2001/Papers...
  - nxobject 2y ago
    I wouldn't be surprised as well if there was some binning that occurred – the dies are huge, so why not overprovision in the design? (Although erring on the side of slighlty more surprise in the case of some binning, since IBM mainframes seem to exist beyond the laws of commodity economics, and it looks like they're using a 5nm node.)
  - nullc 2y ago
    the CPU's only appear to use about 1/3rd of the die area. Most of the space is cache.
  - twic 2y ago
    People buy whole machines to run memcached!
kev009 2y ago
The claimed latency for it seems not far off from some other vendor's L3 caches which may be by virtue of rethinking where to share and therefore paying interconnection coherency taxes. The innovation here seems to be adaptive sizing so if by whatever algorithm/metric a remote core is idle, it can volunteer cache to L4. Presumably the interconnect is much richer than contemporary processors in typical IBM fashion and they can do all the control at a very low level (hw state machines&microcoding) so it is fast and transparent. It will be interesting to hear how it works in practice and if POWER12 gets a similar feature since it shares a lot of R&D.
unnah 2y ago
At a basic level, anything with a working set on the order of 360 MB should benefit from 360 MB of combined L3 with a worst-case latency of 11.5 ns, regardless of which parts end up in which L2 slice (and the cache allocation heuristics described in the article look pretty smart to me). Similarly, if you have a total working set of a couple of GB then the 2.8 GB combined L4 at 48.5 ns latency should be great. Is there any other hardware on the market that can offer so much memory at such a low latency?
- lostmsu 2y ago
  /Uneducated/ these latency numbers seem large to me. DDR5 memory sticks I browsed yesterday for a home PC listed 10ns first word latency.
  - unnah 2y ago
    If the data is not in cache, it takes quite a while longer from the time the CPU core issues a load instruction for the results to get back to the next instruction. The CPU core has to first try L1 and L2, do a TLB lookup to convert a virtual address to a physical address, send a request to L3 over an on-chip connection, then after L3 lookup fails the memory controller has to transfer a 64-byte cache line from the main memory, and the results are then sent back to the core... Have a look at the section "Cache setup" at https://chipsandcheese.com/2024/08/14/amds-ryzen-9950x-zen-5-on-desktop/ https://chipsandcheese.com/2024/08/14/amds-ryzen-9950x-zen-5... for some real-world latency values. Once we're talking about a 100+ MB working set (i.e. DDR5 instead of cache), a top-of-the-line Ryzen 9950X has an access latency of about 100 ns. There is also some older data for a wider variety of CPUs at https://chipsandcheese.com/memory-latency-data/ https://chipsandcheese.com/memory-latency-data/ - and there the older IBM z15 is in a class of its own.

_a_a_a_ 2y ago

At a guess, a single thread which benefits from as much cache as it can get.

tedunangst 2y ago
Sure, but having to buy entire CPUs filled with idle cores to scale up cache seems very expensive.
- _a_a_a_ 2y ago
  I said this is where it would help (AFAICS), not that it was the best solution.
- kev009 2y ago
  These cores are typically licensed with class/restrictions so in absolute terms yes but in the financial engineering of how the system is delivered with excess and restricted hardware no (see core types on the prior/shipping generation here https://www.ibm.com/downloads/cas/6NW3RPQV https://www.ibm.com/downloads/cas/6NW3RPQV) There are probably design reuse and RAS considerations that make it not currently worthwhile to i.e. have a distinct physical design for SAP or whatever cores.
  - sillywalk 2y ago
    I don't know if it's still the case, but in terms of RAS, the Z/Series CPUs from ~2004 had duplicated/compared instruction-fetch/decode and execution units. https://pages.cs.wisc.edu/~remzi/Classes/838/Fall2001/Papers/tale-of-two.pdf https://pages.cs.wisc.edu/~remzi/Classes/838/Fall2001/Papers...
- nxobject 2y ago
  I wouldn't be surprised as well if there was some binning that occurred – the dies are huge, so why not overprovision in the design? (Although erring on the side of slighlty more surprise in the case of some binning, since IBM mainframes seem to exist beyond the laws of commodity economics, and it looks like they're using a 5nm node.)
- nullc 2y ago
  the CPU's only appear to use about 1/3rd of the die area. Most of the space is cache.
- twic 2y ago
  People buy whole machines to run memcached!

kev009 2y ago

The claimed latency for it seems not far off from some other vendor's L3 caches which may be by virtue of rethinking where to share and therefore paying interconnection coherency taxes. The innovation here seems to be adaptive sizing so if by whatever algorithm/metric a remote core is idle, it can volunteer cache to L4. Presumably the interconnect is much richer than contemporary processors in typical IBM fashion and they can do all the control at a very low level (hw state machines&microcoding) so it is fast and transparent. It will be interesting to hear how it works in practice and if POWER12 gets a similar feature since it shares a lot of R&D.

unnah 2y ago

At a basic level, anything with a working set on the order of 360 MB should benefit from 360 MB of combined L3 with a worst-case latency of 11.5 ns, regardless of which parts end up in which L2 slice (and the cache allocation heuristics described in the article look pretty smart to me). Similarly, if you have a total working set of a couple of GB then the 2.8 GB combined L4 at 48.5 ns latency should be great. Is there any other hardware on the market that can offer so much memory at such a low latency?

lostmsu 2y ago
/Uneducated/ these latency numbers seem large to me. DDR5 memory sticks I browsed yesterday for a home PC listed 10ns first word latency.
- unnah 2y ago
  If the data is not in cache, it takes quite a while longer from the time the CPU core issues a load instruction for the results to get back to the next instruction. The CPU core has to first try L1 and L2, do a TLB lookup to convert a virtual address to a physical address, send a request to L3 over an on-chip connection, then after L3 lookup fails the memory controller has to transfer a 64-byte cache line from the main memory, and the results are then sent back to the core... Have a look at the section "Cache setup" at https://chipsandcheese.com/2024/08/14/amds-ryzen-9950x-zen-5-on-desktop/ https://chipsandcheese.com/2024/08/14/amds-ryzen-9950x-zen-5... for some real-world latency values. Once we're talking about a 100+ MB working set (i.e. DDR5 instead of cache), a top-of-the-line Ryzen 9950X has an access latency of about 100 ns. There is also some older data for a wider variety of CPUs at https://chipsandcheese.com/memory-latency-data/ https://chipsandcheese.com/memory-latency-data/ - and there the older IBM z15 is in a class of its own.