3 ms·

> Are there really no use-case where 128 Core+ with NUMA offer some advantage? Are there any use cases where 128+ core single socket wouldn't be preferred to a

> Are there really no use-case where 128 Core+ with NUMA offer some advantage? Are there any use cases where 128+ core single socket wouldn't be preferred to a 128+ core multiple socket design that is burdened by NUMA? AMD has been showing us that integrating the interconnects into the CPU package directly and letting it handle all the issues is a better design.

dragontamer 5y ago
When a hypothetical 128-core single socket comes out, will there be no workload that prefers to use a 2x128-core dual socket instead? AMD CPUs remain largely dual-socket compatible. Today's 64-core EPYCs can be dual-socketed into 2x64-core beasts. It just seems silly to me that if you're building say 200 computers in 10x racks (20-computers per 10x 40U racks) that you'd prefer single socket over dual-socket. If you're scaling up and out so much, what exactly is the problem with dual socket? Its not costs: dual socket remains cost-effective on a per-core basis over single-socket. Dual-sockets cuts the number of computers you need to work with in half. Etc. etc.
- brendangregg 5y ago
  I don't have a workload I'd prefer to see on 2x128-core: We're already microservices running across a pool of instances, and would prefer a bigger pool of faster instances than a smaller pool of slower ones at the same cost. Once we get a workload running on 100+ cores, I often see a lot of lock contention anyway. Going bigger usually makes that worse (worse ROI). As for datacenter size/cost, it's a good point, but what if two 1-Socket servers could take up the same space as one 2-Socket server? :-) That may never happen, but some level of space optimization will, so it's not a simple doubling of size. E.g., Facebook's work in the OCP with 1-socket sleds (or blades): https://www.opencompute.org/documents/ocp-yosemite-v3-platform-design-specification-1v16-pdf https://www.opencompute.org/documents/ocp-yosemite-v3-platfo...
  - dragontamer 5y ago
    > As for datacenter size/cost, it's a good point, but what if two 1-Socket servers could take up the same space as one 2-Socket server? :-) That may never happen, but some level of space optimization will Oh it certainly exists. Computers are space-optimized to the point of nonsense. IIRC, most people don't even bother to use widely available 1U servers because you run out of power before you fill up 40U racks. Hyperscalers, such as Google / Netflix / Amazon are a bit different of course (IIRC, you work at one right?), since they can specially build their data centers to have far denser power-delivery and actually support 40-computers or even 80-computers per rack. But more typical offices simply do not have the power-density to run 1U nodes or smaller (Ex: Supermicro 2xNodes in 1U nodes or Supermicro 4xNode in 2U). In effect: modern computer systems usually run out of power before they run out of rack space. Especially when you consider that every Watt-delivered turns into Heat (Watts) generated, which then requires a more powerful air-conditioner to keep the room within operating specs. So you're right that modern datacenters probably don't care about size. Space is relatively cheap, power-lines are expensive! 2x Sockets for 2x per 1U == 160 CPUs per 40U rack. 10 such racks would use over a Megawatt of power once we factor in air conditioning, so a typical building just won't handle that. ------------- > I don't have a workload I'd prefer to see on 2x128-core: We're already microservices running across a pool of instances, and would prefer a bigger pool of faster instances than a smaller pool of slower ones at the same cost. Once we get a workload running on 100+ cores, I often see a lot of lock contention anyway. Going bigger usually makes that worse (worse ROI). But that "lock contention" you're measuring is something like 250ns to 500ns over a channel that's 800Gbit/sec thick. EDIT: To be more specific: I'm talking about the MESI messages going over the NUMA fabric. In contrast, a packet over 10 Gbit Ethernet is basically two orders of magnitude less bandwidth and an order of magnitude more latency (maybe 2500 to 5000 nanoseconds of latency?). If the application were truly limited by the communication paradigm between the NUMA Fabric, switching to Ethernet or InfiniBand would only slow it down further. EDIT: Case in point: we don't do spinlocks over Ethernet. I mean, we could in theory (RDMA a region of memory over Ethernet and then hold it as a Spinlock), but we all know its a bad idea. We do spinlocks at L3 and/or the NUMA Fabric level (and maybe we'll do it over PCIe 5.0 / CCIX level, as cache-coherent I/O becomes possible).
    - deleted 5y ago
      [deleted]
- drewg123 5y ago
  There are a few problems with dual sockets: - The flip side of cuts the computers you need to work with in half is that it doubles the blast radius in case of PSU/fan/mobo/etc failure - If you're interested in I/O, dual sockets can be problematic because few motherboards are "balanced" with an equal number of PCIe slots local to each socket. - NUMA makes everything harder. Even after the work that I've done to make NUMA useful for Netflix's Open Connect (CDN) on FreeBSD, I'd very much rather just use flat machines wherever I can. NUMA gives lots of opportunities for comically bad performance if any little thing is placed incorrectly.
  - dragontamer 5y ago
    I do appreciate the difficulty of getting software configured correctly. But I'm of the opinion that software configuration is quicker and easier than redeveloping algorithms to become faster on FPGAs or GPUs. Its really odd to have a talk about how FPGAs are part of a hypothetical mainstream future (when so few people even know how to code in Verilog, let alone know how to synthesize a systolic array or other obscure parallel architecture). And then turn around and say that Dual-Socket computers are too hard to configure. Verilog / FPGAs aren't magic. They're just highly configurable logic gates + some preconfigured ALUs that allow for alternative parallel structures. These alternative parallel structures (most commonly a systolic array) are often highly specific to a task. But ultimately: the mode of compute still needs to be super-parallel to beat a CPU. Remember: CPUs have higher clock-speeds than FPGAs. That's why FPGAs have mini-ALUs inside of them (ex: multipliers), because ASIC beats configurable logic in every spec that matters (GHz, power-efficiency, mm^2 on die).

dragontamer 5y ago

When a hypothetical 128-core single socket comes out, will there be no workload that prefers to use a 2x128-core dual socket instead? AMD CPUs remain largely dual-socket compatible. Today's 64-core EPYCs can be dual-socketed into 2x64-core beasts. It just seems silly to me that if you're building say 200 computers in 10x racks (20-computers per 10x 40U racks) that you'd prefer single socket over dual-socket. If you're scaling up and out so much, what exactly is the problem with dual socket? Its not costs: dual socket remains cost-effective on a per-core basis over single-socket. Dual-sockets cuts the number of computers you need to work with in half. Etc. etc.

brendangregg 5y ago
I don't have a workload I'd prefer to see on 2x128-core: We're already microservices running across a pool of instances, and would prefer a bigger pool of faster instances than a smaller pool of slower ones at the same cost. Once we get a workload running on 100+ cores, I often see a lot of lock contention anyway. Going bigger usually makes that worse (worse ROI). As for datacenter size/cost, it's a good point, but what if two 1-Socket servers could take up the same space as one 2-Socket server? :-) That may never happen, but some level of space optimization will, so it's not a simple doubling of size. E.g., Facebook's work in the OCP with 1-socket sleds (or blades): https://www.opencompute.org/documents/ocp-yosemite-v3-platform-design-specification-1v16-pdf https://www.opencompute.org/documents/ocp-yosemite-v3-platfo...
- dragontamer 5y ago
  > As for datacenter size/cost, it's a good point, but what if two 1-Socket servers could take up the same space as one 2-Socket server? :-) That may never happen, but some level of space optimization will Oh it certainly exists. Computers are space-optimized to the point of nonsense. IIRC, most people don't even bother to use widely available 1U servers because you run out of power before you fill up 40U racks. Hyperscalers, such as Google / Netflix / Amazon are a bit different of course (IIRC, you work at one right?), since they can specially build their data centers to have far denser power-delivery and actually support 40-computers or even 80-computers per rack. But more typical offices simply do not have the power-density to run 1U nodes or smaller (Ex: Supermicro 2xNodes in 1U nodes or Supermicro 4xNode in 2U). In effect: modern computer systems usually run out of power before they run out of rack space. Especially when you consider that every Watt-delivered turns into Heat (Watts) generated, which then requires a more powerful air-conditioner to keep the room within operating specs. So you're right that modern datacenters probably don't care about size. Space is relatively cheap, power-lines are expensive! 2x Sockets for 2x per 1U == 160 CPUs per 40U rack. 10 such racks would use over a Megawatt of power once we factor in air conditioning, so a typical building just won't handle that. ------------- > I don't have a workload I'd prefer to see on 2x128-core: We're already microservices running across a pool of instances, and would prefer a bigger pool of faster instances than a smaller pool of slower ones at the same cost. Once we get a workload running on 100+ cores, I often see a lot of lock contention anyway. Going bigger usually makes that worse (worse ROI). But that "lock contention" you're measuring is something like 250ns to 500ns over a channel that's 800Gbit/sec thick. EDIT: To be more specific: I'm talking about the MESI messages going over the NUMA fabric. In contrast, a packet over 10 Gbit Ethernet is basically two orders of magnitude less bandwidth and an order of magnitude more latency (maybe 2500 to 5000 nanoseconds of latency?). If the application were truly limited by the communication paradigm between the NUMA Fabric, switching to Ethernet or InfiniBand would only slow it down further. EDIT: Case in point: we don't do spinlocks over Ethernet. I mean, we could in theory (RDMA a region of memory over Ethernet and then hold it as a Spinlock), but we all know its a bad idea. We do spinlocks at L3 and/or the NUMA Fabric level (and maybe we'll do it over PCIe 5.0 / CCIX level, as cache-coherent I/O becomes possible).
  - deleted 5y ago
    [deleted]
drewg123 5y ago
There are a few problems with dual sockets: - The flip side of cuts the computers you need to work with in half is that it doubles the blast radius in case of PSU/fan/mobo/etc failure - If you're interested in I/O, dual sockets can be problematic because few motherboards are "balanced" with an equal number of PCIe slots local to each socket. - NUMA makes everything harder. Even after the work that I've done to make NUMA useful for Netflix's Open Connect (CDN) on FreeBSD, I'd very much rather just use flat machines wherever I can. NUMA gives lots of opportunities for comically bad performance if any little thing is placed incorrectly.
- dragontamer 5y ago
  I do appreciate the difficulty of getting software configured correctly. But I'm of the opinion that software configuration is quicker and easier than redeveloping algorithms to become faster on FPGAs or GPUs. Its really odd to have a talk about how FPGAs are part of a hypothetical mainstream future (when so few people even know how to code in Verilog, let alone know how to synthesize a systolic array or other obscure parallel architecture). And then turn around and say that Dual-Socket computers are too hard to configure. Verilog / FPGAs aren't magic. They're just highly configurable logic gates + some preconfigured ALUs that allow for alternative parallel structures. These alternative parallel structures (most commonly a systolic array) are often highly specific to a task. But ultimately: the mode of compute still needs to be super-parallel to beat a CPU. Remember: CPUs have higher clock-speeds than FPGAs. That's why FPGAs have mini-ALUs inside of them (ex: multipliers), because ASIC beats configurable logic in every spec that matters (GHz, power-efficiency, mm^2 on die).