3 ms·

I assume you end up getting bottlenecks to RAM, disk, and network when there’s this much parallelism in CPU. So even for cloud providers it’s probably most suit

I assume you end up getting bottlenecks to RAM, disk, and network when there’s this much parallelism in CPU. So even for cloud providers it’s probably most suited to some fairly specific CPU-heavy workloads.

adgjlsfhk1 2y ago
you're under-estimating the power of 512 mb of cache, 12 channel ddr5, pcie 5 NVMEs and 880GB/s networking cards. The giant L3 cache means that memory pressure is significantly reduced (especially if many of the cores use similar data). Ram is the weakest link here (12 channels of ddr5 6000 is a ton from a consumer standpoint, but scaling up from dual channel on a 16 core cpu would only bring you up to 96 cores). The 128 lanes of PCIE though means that you don't have much bottleneck outside of ram (if you're doing things properly). If you use 64 of those for storage, that can be 100s of TB of ridiculously fast storage, and you still have another 64 left over for ridiculously fast networking.
everforward 2y ago
In the cloud vendor space (and really virtualization in general) local disk is practically non-existent. The vast majority of apps these days are stateless and generate very little IO, and even if they do generate IO, it's more than likely going to some kind of network share so that the VM can fail over. If the VM's data is on local disk, you can't migrate it to a new host if the host it's on dies (along with its local disks). Presuming the other responder is correct about there being 880GB/s of network cards (and that the units are correct at GB/s instead of Gb/s or Gbps), that should be more than plenty. At the largest CPUs, 192 cores would be 4.5 GB/s per core (2.3 GB/s if dual socket). That's more than enough; I see a lot of dual-socket servers with 24 or so total cores running on 10 Gbps or maybe bonded 10 Gbps. That's about 100 MB/s per core.

adgjlsfhk1 2y ago

you're under-estimating the power of 512 mb of cache, 12 channel ddr5, pcie 5 NVMEs and 880GB/s networking cards. The giant L3 cache means that memory pressure is significantly reduced (especially if many of the cores use similar data). Ram is the weakest link here (12 channels of ddr5 6000 is a ton from a consumer standpoint, but scaling up from dual channel on a 16 core cpu would only bring you up to 96 cores). The 128 lanes of PCIE though means that you don't have much bottleneck outside of ram (if you're doing things properly). If you use 64 of those for storage, that can be 100s of TB of ridiculously fast storage, and you still have another 64 left over for ridiculously fast networking.

everforward 2y ago

In the cloud vendor space (and really virtualization in general) local disk is practically non-existent. The vast majority of apps these days are stateless and generate very little IO, and even if they do generate IO, it's more than likely going to some kind of network share so that the VM can fail over. If the VM's data is on local disk, you can't migrate it to a new host if the host it's on dies (along with its local disks). Presuming the other responder is correct about there being 880GB/s of network cards (and that the units are correct at GB/s instead of Gb/s or Gbps), that should be more than plenty. At the largest CPUs, 192 cores would be 4.5 GB/s per core (2.3 GB/s if dual socket). That's more than enough; I see a lot of dual-socket servers with 24 or so total cores running on 10 Gbps or maybe bonded 10 Gbps. That's about 100 MB/s per core.