4 ms·

The biggest concern I have with chiplets is the impact on concepts like async/await. If the runtime isn't NUMA-aware, it will wind up with cross-die continuatio

The biggest concern I have with chiplets is the impact on concepts like async/await. If the runtime isn't NUMA-aware, it will wind up with cross-die continuations of logical work, increasing latency and burning up interconnect bandwidth. Even if it is aware of the architecture, you still have a messy, non-obvious situation to resolve regarding whether or not to take the latency impact. Managing your own threads and always assuming the worst case scenario (I.e., they're all a microsecond away from each other) will give you the most defensive solution as architecture changes over time. I really don't know the best answer. Another NUMA node implies some necessary additional complexity. It would seem the wonder of 192 cores in a socket has some tradeoffs the software people need to compensate for now. You can't get all that magic for free.

badgersnake 2y ago
My EPYC gen1 chip appeared as 4 NUMA nodes. It seems like they've moved away from that though.
- bradfa 2y ago
  I believe the first generation of Zen Epyc chips were designed to be scalable in the sense that each die had its own CCX, memory controllers, and serdes blocks (for PCIe, SATA, and 10GKR). A single die Epyc was a single NUMA node, dual die Epyc were 2 NUMA nodes, and quad die Epycs were 4 NUMA nodes. It really was that all the resources from the other dies were in other NUMA nodes. Accessing another die's caches, memory controllers, or serdes blocks required using the infinity fabric. In later generations of Zen Epyc CPUs, there's a central IO die which has the memory controllers and serdes blocks, then small CCX dies which effectively just had CPU cores and caches on them. This allowed for using different process nodes or manufacturing technologies to optimize for each die's specific tasks. It also allowed for much more modular configurations of CPU core counts and CPU frequencies while providing a high number of memory controllers and serdes lanes, to address different market desires.
baybal2 2y ago
[dead]
fluoridation 2y ago
Chiplets don't require NUMA. The Ryzen 9s have their cores split between two chiplets and still have UMA. Of course, if you're on over a hundred cores you will be on a NUMA platform, but NUMA isn't really new, either, and has been around since at least the first multi-socket systems. Async/await isn't more of a problem with it than regular old preemption, and that has generally been solved through virtualization and/or per-process processor affinity.
- bob1029 2y ago
  > Chiplets don't require NUMA The nuance is that even if you "hide" the extra NUMA tier and have oversized lanes connecting everything, you still have some cores that are further away than others in time. It doesn't matter how it shows up in device manager, the physical constraint is still there.
  - rowanG077 2y ago
    I don't see how the physical constraint matters of you can't observe it with a performance penalty. I don't really know if that's the case but that is what the person you are replying to is implying.
  - fluoridation 2y ago
    Ryzen chips have a different network topology than Threadripper and EPYC chips. All external communication goes through the IO die, so any given core on either processor die is the same effective distance to the memory modules. The speed of signal propagation being finite, it's possible that individual cores within the same die might have different effective latencies, depending on how they routed the bus internally (although that's a problem you have on any single die multicore design), but it's not possible when comparing cores at mirrored positions on opposite dies. Threadripper and EPYC are totally different. They actually have NUMA topologies, where different dies are wired such that they have more distance to traverse to reach a given memory module, and accessing the farther ones involves some negotiation with a routing component.

badgersnake 2y ago

My EPYC gen1 chip appeared as 4 NUMA nodes. It seems like they've moved away from that though.

bradfa 2y ago
I believe the first generation of Zen Epyc chips were designed to be scalable in the sense that each die had its own CCX, memory controllers, and serdes blocks (for PCIe, SATA, and 10GKR). A single die Epyc was a single NUMA node, dual die Epyc were 2 NUMA nodes, and quad die Epycs were 4 NUMA nodes. It really was that all the resources from the other dies were in other NUMA nodes. Accessing another die's caches, memory controllers, or serdes blocks required using the infinity fabric. In later generations of Zen Epyc CPUs, there's a central IO die which has the memory controllers and serdes blocks, then small CCX dies which effectively just had CPU cores and caches on them. This allowed for using different process nodes or manufacturing technologies to optimize for each die's specific tasks. It also allowed for much more modular configurations of CPU core counts and CPU frequencies while providing a high number of memory controllers and serdes lanes, to address different market desires.

baybal2 2y ago

[dead]

fluoridation 2y ago

Chiplets don't require NUMA. The Ryzen 9s have their cores split between two chiplets and still have UMA. Of course, if you're on over a hundred cores you will be on a NUMA platform, but NUMA isn't really new, either, and has been around since at least the first multi-socket systems. Async/await isn't more of a problem with it than regular old preemption, and that has generally been solved through virtualization and/or per-process processor affinity.

bob1029 2y ago
> Chiplets don't require NUMA The nuance is that even if you "hide" the extra NUMA tier and have oversized lanes connecting everything, you still have some cores that are further away than others in time. It doesn't matter how it shows up in device manager, the physical constraint is still there.
- rowanG077 2y ago
  I don't see how the physical constraint matters of you can't observe it with a performance penalty. I don't really know if that's the case but that is what the person you are replying to is implying.
- fluoridation 2y ago
  Ryzen chips have a different network topology than Threadripper and EPYC chips. All external communication goes through the IO die, so any given core on either processor die is the same effective distance to the memory modules. The speed of signal propagation being finite, it's possible that individual cores within the same die might have different effective latencies, depending on how they routed the bus internally (although that's a problem you have on any single die multicore design), but it's not possible when comparing cores at mirrored positions on opposite dies. Threadripper and EPYC are totally different. They actually have NUMA topologies, where different dies are wired such that they have more distance to traverse to reach a given memory module, and accessing the farther ones involves some negotiation with a routing component.