4 ms·

Yeah, they are mostly for cloud providers. They will be running all of our docker containers and Kubernetes clusters and function as a service workflows as wel

Yeah, they are mostly for cloud providers. They will be running all of our docker containers and Kubernetes clusters and function as a service workflows as well as our databases. It really should mean that cloud data centres should be able to greatly increase capacity without getting larger in terms of physical size. That is a huge net win for the cloud providers.

mikepurvis 2y ago
I assume you end up getting bottlenecks to RAM, disk, and network when there’s this much parallelism in CPU. So even for cloud providers it’s probably most suited to some fairly specific CPU-heavy workloads.
- adgjlsfhk1 2y ago
  you're under-estimating the power of 512 mb of cache, 12 channel ddr5, pcie 5 NVMEs and 880GB/s networking cards. The giant L3 cache means that memory pressure is significantly reduced (especially if many of the cores use similar data). Ram is the weakest link here (12 channels of ddr5 6000 is a ton from a consumer standpoint, but scaling up from dual channel on a 16 core cpu would only bring you up to 96 cores). The 128 lanes of PCIE though means that you don't have much bottleneck outside of ram (if you're doing things properly). If you use 64 of those for storage, that can be 100s of TB of ridiculously fast storage, and you still have another 64 left over for ridiculously fast networking.
- everforward 2y ago
  In the cloud vendor space (and really virtualization in general) local disk is practically non-existent. The vast majority of apps these days are stateless and generate very little IO, and even if they do generate IO, it's more than likely going to some kind of network share so that the VM can fail over. If the VM's data is on local disk, you can't migrate it to a new host if the host it's on dies (along with its local disks). Presuming the other responder is correct about there being 880GB/s of network cards (and that the units are correct at GB/s instead of Gb/s or Gbps), that should be more than plenty. At the largest CPUs, 192 cores would be 4.5 GB/s per core (2.3 GB/s if dual socket). That's more than enough; I see a lot of dual-socket servers with 24 or so total cores running on 10 Gbps or maybe bonded 10 Gbps. That's about 100 MB/s per core.
jiggawatts 2y ago
You would think so, but both Azure and Amazon have dragged their feet deploying the last couple of CPU generations from both Intel and AMD. I suspect it's a sign of the global downturn, cloud adoption in general seems to have stalled. Hence, no new CPU models being deployed at scale. When the downturn lifts, I full expect the big cloud providers to start deploying huge numbers of these but that seems to be at least a year away.
- bhouston 2y ago
  I think that Amazon has been deploying Graviton though.
woleium 2y ago
usually dcs are limited by the power and or cooling per rack, you end up with having to run more power and hvac, which can be prohibitively expensive. Alternatively you half fill racks when you upgrade.

mikepurvis 2y ago

I assume you end up getting bottlenecks to RAM, disk, and network when there’s this much parallelism in CPU. So even for cloud providers it’s probably most suited to some fairly specific CPU-heavy workloads.

adgjlsfhk1 2y ago
you're under-estimating the power of 512 mb of cache, 12 channel ddr5, pcie 5 NVMEs and 880GB/s networking cards. The giant L3 cache means that memory pressure is significantly reduced (especially if many of the cores use similar data). Ram is the weakest link here (12 channels of ddr5 6000 is a ton from a consumer standpoint, but scaling up from dual channel on a 16 core cpu would only bring you up to 96 cores). The 128 lanes of PCIE though means that you don't have much bottleneck outside of ram (if you're doing things properly). If you use 64 of those for storage, that can be 100s of TB of ridiculously fast storage, and you still have another 64 left over for ridiculously fast networking.
everforward 2y ago
In the cloud vendor space (and really virtualization in general) local disk is practically non-existent. The vast majority of apps these days are stateless and generate very little IO, and even if they do generate IO, it's more than likely going to some kind of network share so that the VM can fail over. If the VM's data is on local disk, you can't migrate it to a new host if the host it's on dies (along with its local disks). Presuming the other responder is correct about there being 880GB/s of network cards (and that the units are correct at GB/s instead of Gb/s or Gbps), that should be more than plenty. At the largest CPUs, 192 cores would be 4.5 GB/s per core (2.3 GB/s if dual socket). That's more than enough; I see a lot of dual-socket servers with 24 or so total cores running on 10 Gbps or maybe bonded 10 Gbps. That's about 100 MB/s per core.

jiggawatts 2y ago

You would think so, but both Azure and Amazon have dragged their feet deploying the last couple of CPU generations from both Intel and AMD. I suspect it's a sign of the global downturn, cloud adoption in general seems to have stalled. Hence, no new CPU models being deployed at scale. When the downturn lifts, I full expect the big cloud providers to start deploying huge numbers of these but that seems to be at least a year away.

bhouston 2y ago
I think that Amazon has been deploying Graviton though.

woleium 2y ago

usually dcs are limited by the power and or cooling per rack, you end up with having to run more power and hvac, which can be prohibitively expensive. Alternatively you half fill racks when you upgrade.