4 ms·

2/4 CCX’s lack direct memory access. This is kind of a deal breaker. Threadripper is already memory bound, so the 2 new packages will just be waiting on RAM mos

2/4 CCX’s lack direct memory access. This is kind of a deal breaker. Threadripper is already memory bound, so the 2 new packages will just be waiting on RAM most the time. Cite: http://en.community.dell.com/cfs-file/__key/telligent-evolution-components-attachments/13-4491-00-00-20-44-47-63/Direct-from-Development-_2D00_-PowerEdge-NUMA-Configs-with-AMD-EPYC-Processors.pdf?forcedownload=true http://en.community.dell.com/cfs-file/__key/telligent-evolut... It would be nice if L3 wasn’t an eviction cache :/

loeg 8y ago
> Threadripper is already memory bound This depends heavily on your workflow. If you find TR memory bound, then you want a server part. TR has 4 memory controllers that will each do DDR4-3200 in TR2 per TFA. That should be able to push ~95 GiB/s. What do you compare that against? Intel's highest end HEDT part, i9 7980XE with 18 cores, only supports DDR4-2666 and has the same number of controllers, so it should hit ~80 GiB/s. It retails for $2000. If you want more memory bandwidth you're either buying an even more expensive Xeon, or an AMD Epyc part.
- valarauca1 8y ago
  >This depends heavily on your workflow. If you find TR memory bound, then you want a server part. TR has 4 memory controllers that will each do DDR4-3200 in TR2 per TFA. That should be able to push ~95 GiB/s. Wrong. You neither read the OP or my citation. Eypc can push 95GiB/s with its 8 channels. TR2 isn't increasing the number of channels, so 2 CCX's (8 cores, 16hyper threads) don't have direct memory access. They have to work with in the inter-chip CCX bus to get memory access. This is going to carry a lot of performance penalties.
  - loeg 8y ago
    > Wrong. What? Are you seriously claiming that a workflow that only uses 4 GiB/s of memory throughput is somehow memory bound on a controller that can push 95 GiB/s? Workflow matters and affects reasonable CPU choice. > You neither read the OP or my citation. Eypc can push 95GiB/s with its 8 channels. Um, also not true. Your link does not include any memory throughput figures supporting your contention that TR is "memory bound." I believe you are totally mistaken about the 95 GiB/s figure for Epyc; see below: TR, quad channel: https://en.wikichip.org/wiki/amd/ryzen_threadripper/1950x#Memory_controller https://en.wikichip.org/wiki/amd/ryzen_threadripper/1950x#Me... Epyc, octa channel: https://en.wikichip.org/wiki/amd/epyc/7551p#Memory_controller https://en.wikichip.org/wiki/amd/epyc/7551p#Memory_controlle... Epyc can push 159 GiB/s with DDR4-2666. Per socket: > In a dual-socket configuration, the maximum supported memory doubles to 4 TiB along with the maximum theoretical bandwidth of 317.9 GiB/s. You said: > TR2 isn't increasing the number of channels, so 2 CCX's (8 cores, 16hyper threads) don't have direct memory access. Understood. I have never said otherwise. > This is going to carry a lot of performance penalties. The extra cores will have higher NUMA latency than the memory-local cores, yes. But it does not somehow decrease the total memory bandwidth available across the CPU.
ashleyn 8y ago
Cache coherence in application development is more important now than ever

loeg 8y ago

> Threadripper is already memory bound This depends heavily on your workflow. If you find TR memory bound, then you want a server part. TR has 4 memory controllers that will each do DDR4-3200 in TR2 per TFA. That should be able to push ~95 GiB/s. What do you compare that against? Intel's highest end HEDT part, i9 7980XE with 18 cores, only supports DDR4-2666 and has the same number of controllers, so it should hit ~80 GiB/s. It retails for $2000. If you want more memory bandwidth you're either buying an even more expensive Xeon, or an AMD Epyc part.

valarauca1 8y ago
>This depends heavily on your workflow. If you find TR memory bound, then you want a server part. TR has 4 memory controllers that will each do DDR4-3200 in TR2 per TFA. That should be able to push ~95 GiB/s. Wrong. You neither read the OP or my citation. Eypc can push 95GiB/s with its 8 channels. TR2 isn't increasing the number of channels, so 2 CCX's (8 cores, 16hyper threads) don't have direct memory access. They have to work with in the inter-chip CCX bus to get memory access. This is going to carry a lot of performance penalties.
- loeg 8y ago
  > Wrong. What? Are you seriously claiming that a workflow that only uses 4 GiB/s of memory throughput is somehow memory bound on a controller that can push 95 GiB/s? Workflow matters and affects reasonable CPU choice. > You neither read the OP or my citation. Eypc can push 95GiB/s with its 8 channels. Um, also not true. Your link does not include any memory throughput figures supporting your contention that TR is "memory bound." I believe you are totally mistaken about the 95 GiB/s figure for Epyc; see below: TR, quad channel: https://en.wikichip.org/wiki/amd/ryzen_threadripper/1950x#Memory_controller https://en.wikichip.org/wiki/amd/ryzen_threadripper/1950x#Me... Epyc, octa channel: https://en.wikichip.org/wiki/amd/epyc/7551p#Memory_controller https://en.wikichip.org/wiki/amd/epyc/7551p#Memory_controlle... Epyc can push 159 GiB/s with DDR4-2666. Per socket: > In a dual-socket configuration, the maximum supported memory doubles to 4 TiB along with the maximum theoretical bandwidth of 317.9 GiB/s. You said: > TR2 isn't increasing the number of channels, so 2 CCX's (8 cores, 16hyper threads) don't have direct memory access. Understood. I have never said otherwise. > This is going to carry a lot of performance penalties. The extra cores will have higher NUMA latency than the memory-local cores, yes. But it does not somehow decrease the total memory bandwidth available across the CPU.

ashleyn 8y ago

Cache coherence in application development is more important now than ever