4 ms·

> Wait, so latency over NUMA is too much, but you're willing to incur a 100Gb network link? The point is not just the latency, but latency and bandwidth. If t

> Wait, so latency over NUMA is too much, but you're willing to incur a 100Gb network link? The point is not just the latency, but latency and bandwidth. If the application is relying heavily on the NUMA interconnect to transfer tons of data, it's not going to be making efficient use of the processor cores or the RAM bandwidth. It's a total all around bust. You're just wasting money at that point. ^1 If you aren't relying heavily on the NUMA interconnect, and each node is operating independently with only small bits of information exchanged across the interconnect, then you'd save a metric ton of money by switching to separate machines and using just high speed fiber network links -- such as 100Gb. I'm not saying that network would be better than the NUMA interconnect. I'm saying that you're not going to be having a happy day if you're relying on the interconnect for large amounts of data transfer. The only situation where the NUMA set up is better is if you have a need for frequent, low latency communication between NUMA nodes... where very little data is being transferred between nodes, so the entire problem is just latency. At that point, you're still suffering a lot by the NUMA interconnect, and it would be better to use larger processors... such as Epyc Rome processors. So you really have to be in a very obscure situation which can't fit onto a dual socket Rome server, but can fit within a machine less than 2x larger. (28 cores * 8 sockets is less than 2x larger than 64 cores * 2 sockets) You see how complicated this is and how absurdly niche those Platinum 8-socket machines are? They're almost never the right choice. ^1: The exception is if you have an even more niche use case that somehow is built for exactly this scenario in mind, and manages to balance everything perfectly. Such a software is almost certainly ridiculously overcomplicated at this point.

dragontamer 6y ago
> If the application is relying heavily on the NUMA interconnect to transfer tons of data, it's not going to be making efficient use of the processor cores or the memory bandwidth. It's a total all around bust. You're just wasting money at that point. If the interconnect is your bottleneck, you spend money on the interconnect to make it faster. Basic engineering: you attack the bottleneck. You don't start talking about slower systems and how they're cheaper. Because that just slows down the rest of your system. ------ If you just wanted cores, you buy a 28core Xeon Gold. The point of 28-core Xeon Platinum is for the 8-way interconnect and scaling up to 8-way NUMA systems. The only person who would ever buy a Xeon Platinum is someone who wants 40GBps UPI connections at relatively low latencies. (or maybe even the 300GBps connections that IBM offers, but that's a similar high-cost vertical-scaling system) > So you really have to be in a very obscure situation which can't fit onto a dual socket Rome server, but can fit within a machine less than 2x larger. (28 cores * 8 sockets is less than 2x larger than 64 cores * 2 sockets) That's not even that hard to figure out! A 48TB Memcached / Redis, which is far more useful than a 8TB Memcached / Redis box. A bit basic, but yeah. That's the point: spend more money on hardware and then don't spend much engineering time thinking about optimization. If 48TBs of RAM solves a problem that 8TB cannot, then just get the 48TB system.
- coder543 6y ago
  Preface: If you have actually encountered applications that must be run on 8-socket systems because those are literally the only fit for the application... I would love to hear about those experiences. With the advent of Epyc Rome, most use cases for these 8-socket systems vanished instantly. It would be fascinating to hear about use cases that still exist. Your experiences are obviously different than mine. If you need more than 8TB of RAM, with the right application design you can probably do better with fast Optane Persistent Memory or Optane SSDs, and an effective caching strategy. You can have many dozens of terabytes of Optane storage connected to a single system, and Optane is consistently low latency (though not as low latency as RAM, obviously). If you need more compute power, you can generally do better with multiple linked machines. You can only scale an 8-socket system up to 8 sockets. You can link way more machines than that together to get more CPU performance than any 8 socket system could dream of. ---------- I didn't expect you to read and respond so quickly, so I had edited my previous comment before you submitted your reply. This was a key quote added to my previous comment: >> So you really have to be in a very obscure situation which can't fit onto a dual socket Rome server, but can fit within a machine less than 2x larger. (28 cores * 8 sockets is less than 2x larger than 64 cores * 2 sockets) In response to your current comment, > If the interconnect is your bottleneck, you spend money on the interconnect to make it faster. Basic engineering: you attack the bottleneck. Exactly. Using a dual-socket Epyc Rome system would be more than half as powerful as the biggest 8-socket Intel systems, but it would reduce contention over the interconnect dramatically, which means that many applications that are simply wasting money on an 8-socket system would suddenly work better. This also goes back to my comment about using accelerators instead of an 8-socket system. The odds of encountering a situation that just happens to work well with Intel's ridiculously complicated 8-socket NUMA interconnect, but can't work well over a network, and can't work well on a system half the size and requires enormous amounts of RAM to keep the cores fed, the odds seem vanishingly small... and in that case, we still have to consider whether an accelerator (GPU, FPGA, or ASIC) could be used to make a solution that is a better fit for the application anyways, and if so, you'll save large amount of money that way as well. So, to make buying an 8-socket system make sense, the application must require performance that is... - less than twice a dual socket Epyc Rome system, but greater than one dual socket Epyc Rome system can handle - not dependent on transferring huge amounts of data around the interconnect - dependent on very low latency communication between NUMA nodes - needs enormous memory bandwidth for each NUMA node - needs huge amounts of RAM on each memory channel (so you can't just use HBM2 on a GPU to get massive amounts of bandwidth, for example) - etc. It's a niche within a niche within a niche. As I said in an earlier comment, I probably should be more impressed instead of being so cynical about the usefulness of such a machine. They are engineering marvels... but in almost every case, you can save money with a different approach and get equal or better results. That's why 8-socket server sales made up such a small percentage of the market, even before Epyc Rome came in and completely obliterated almost all of the very little value proposition that remained.

dragontamer 6y ago

> If the application is relying heavily on the NUMA interconnect to transfer tons of data, it's not going to be making efficient use of the processor cores or the memory bandwidth. It's a total all around bust. You're just wasting money at that point. If the interconnect is your bottleneck, you spend money on the interconnect to make it faster. Basic engineering: you attack the bottleneck. You don't start talking about slower systems and how they're cheaper. Because that just slows down the rest of your system. ------ If you just wanted cores, you buy a 28core Xeon Gold. The point of 28-core Xeon Platinum is for the 8-way interconnect and scaling up to 8-way NUMA systems. The only person who would ever buy a Xeon Platinum is someone who wants 40GBps UPI connections at relatively low latencies. (or maybe even the 300GBps connections that IBM offers, but that's a similar high-cost vertical-scaling system) > So you really have to be in a very obscure situation which can't fit onto a dual socket Rome server, but can fit within a machine less than 2x larger. (28 cores * 8 sockets is less than 2x larger than 64 cores * 2 sockets) That's not even that hard to figure out! A 48TB Memcached / Redis, which is far more useful than a 8TB Memcached / Redis box. A bit basic, but yeah. That's the point: spend more money on hardware and then don't spend much engineering time thinking about optimization. If 48TBs of RAM solves a problem that 8TB cannot, then just get the 48TB system.

coder543 6y ago
Preface: If you have actually encountered applications that must be run on 8-socket systems because those are literally the only fit for the application... I would love to hear about those experiences. With the advent of Epyc Rome, most use cases for these 8-socket systems vanished instantly. It would be fascinating to hear about use cases that still exist. Your experiences are obviously different than mine. If you need more than 8TB of RAM, with the right application design you can probably do better with fast Optane Persistent Memory or Optane SSDs, and an effective caching strategy. You can have many dozens of terabytes of Optane storage connected to a single system, and Optane is consistently low latency (though not as low latency as RAM, obviously). If you need more compute power, you can generally do better with multiple linked machines. You can only scale an 8-socket system up to 8 sockets. You can link way more machines than that together to get more CPU performance than any 8 socket system could dream of. ---------- I didn't expect you to read and respond so quickly, so I had edited my previous comment before you submitted your reply. This was a key quote added to my previous comment: >> So you really have to be in a very obscure situation which can't fit onto a dual socket Rome server, but can fit within a machine less than 2x larger. (28 cores * 8 sockets is less than 2x larger than 64 cores * 2 sockets) In response to your current comment, > If the interconnect is your bottleneck, you spend money on the interconnect to make it faster. Basic engineering: you attack the bottleneck. Exactly. Using a dual-socket Epyc Rome system would be more than half as powerful as the biggest 8-socket Intel systems, but it would reduce contention over the interconnect dramatically, which means that many applications that are simply wasting money on an 8-socket system would suddenly work better. This also goes back to my comment about using accelerators instead of an 8-socket system. The odds of encountering a situation that just happens to work well with Intel's ridiculously complicated 8-socket NUMA interconnect, but can't work well over a network, and can't work well on a system half the size and requires enormous amounts of RAM to keep the cores fed, the odds seem vanishingly small... and in that case, we still have to consider whether an accelerator (GPU, FPGA, or ASIC) could be used to make a solution that is a better fit for the application anyways, and if so, you'll save large amount of money that way as well. So, to make buying an 8-socket system make sense, the application must require performance that is... - less than twice a dual socket Epyc Rome system, but greater than one dual socket Epyc Rome system can handle - not dependent on transferring huge amounts of data around the interconnect - dependent on very low latency communication between NUMA nodes - needs enormous memory bandwidth for each NUMA node - needs huge amounts of RAM on each memory channel (so you can't just use HBM2 on a GPU to get massive amounts of bandwidth, for example) - etc. It's a niche within a niche within a niche. As I said in an earlier comment, I probably should be more impressed instead of being so cynical about the usefulness of such a machine. They are engineering marvels... but in almost every case, you can save money with a different approach and get equal or better results. That's why 8-socket server sales made up such a small percentage of the market, even before Epyc Rome came in and completely obliterated almost all of the very little value proposition that remained.