7 ms·

Truly mind boggling scale. Twenty years ago we had just 1-2 cores per CPU, so we were lucky to have 4 cores in a dual socket server. A single server can now h

Truly mind boggling scale. Twenty years ago we had just 1-2 cores per CPU, so we were lucky to have 4 cores in a dual socket server. A single server can now have almost 400 cores. Yes, we can have even more ARM cores but they don't perform as well as these do, at least for now.

zer00eyz 2y ago
700+ threads over 2 cores, can saturate 2 400gbe Nic's 500 wats per chip (less than 2 wats per thread)... All of that in a 2U package.... 20 years ago that would have been racks of gear.
- smolder 2y ago
  > 700+ threads over 2 cores I assume you mean 2 sockets.
- jmrm 2y ago
  I think those really 2 watts per thread are a lot more important than what us home users usually think. Having to deliver less power and having to dissipate less watts in form of heat in a data centre are really good news to its operative costs, which is usually a lot bigger than the cost of the purchase of the servers
- justmarc 2y ago
  With these CPUs one can definitely hit much higher rates. 800Gbps from a single server was achieved by Netflix on much lesser systems two years ago: https://nabstreamingsummit.com/wp-content/uploads/2022/05/2022-Streaming-Summit-Netflix.pdf https://nabstreamingsummit.com/wp-content/uploads/2022/05/20... If I were to guess, this hardware can do double that, also helping that we now have actual 800Gbps Ethernet hardware. Indeed 20 years ago this would have been racks of gear at a very high cost and a huge power bill.
  - immibis 2y ago
    That's one hell of a pixelflut server.
chx 2y ago
Indeed: the first dual core server chips only launched in 2005 afaik with 90nm Denmark/Italy/Egypt Opterons and Paxville Xeons but on the Intel side it wasn't until 2007 when they were in full swing.
- p_l 2y ago
  first dual core server chips show up generally available in 2001 with IBM POWER4, then HP PA-RISC ones in 2004, and then Opterons which was followed by "emergency" design of essentially two "sockets" on one die of of NetBurst dual core systems.
  - chx 2y ago
    Ehhhh the MAJC 5200 was generally available in 1999 and I am sure even older designs could be found if we were to dig deep enough. Their market share would also need some digging too. To quote the announcement: "two VLIW (very long instruction word) microprocessors on a single piece of silicon"
    - formerly_proven 2y ago
      Power and PA-RISC shipped servers, though. MAJC on the other hand > Sun built a single model of the MAJC, the two-core MAJC 5200, which was the heart of Sun's XVR-1000 and XVR-4000 workstation graphics boards.
      - kristianp 2y ago
        MAJC: https://en.m.wikipedia.org/wiki/MAJC https://en.m.wikipedia.org/wiki/MAJC Why have I not heard of those before?
        p_l 2y ago
        By the time of XVR GPUs Sun was pretty much exiting the workstation market, and unlike Elite3D and Creator3D the competition has widened for "good enough" PC alternatives using Nvidia and similar chips
RobinL 2y ago
I wonder what percentage of 'big data' jobs that run in clusters would now be far faster on a single big machine with e.g. duckdb rather than spark
- conjecTech 2y ago
  The difference in throughput for local versus distributed orchestration would mainly come from serdes, networking, switching. Serdes can be substantial. Networking and switching has been aggressively offloaded from CPU through better hardware support. Individual tasks would definitely have better latency, but I'd suspect the impact on throughput/CPU usage might be muted. Of course at the extremes (very small jobs, very large/complex objects being passed) you'd see big gains.
  - mtremsal 2y ago
    Would you mind expanding on how SerDes become a bottleneck? I’m not familiar and reading the Wikipedia article wasn’t enough to connect the dots.
    - conjecTech 2y ago
      When you talk between remote machines, you have to translate to a format that can transmitted and distributed between machines(serialization). You then have to undo at the other end(deserialization). If what you are sending along is just a few floats, that can be very cheap. If you're sending along a large nested dictionary or even a full program, not so much. Imagine an example where you have two arrays of 1 billion numbers, and you want to add them pairwise. You could use spark to do that by having each "task" be a single addition. But the time it would take to structure and transmit the 1 billion requests will be many multiples of the amount of time it would take to just do the additions.
  - RobinL 2y ago
    By way of a single example, we've been migrating recently from spark to duckdb. Our jobs are not huge, but too big for a single 'normal' machine. We've gone from a 2.5 hour runtime on a cluster of 10 machines (40,vCPU total) to a 15 minute runtime on a 32vCPU single machine. I don't know for sure, but I think this is largely because it eliminates expensive shuffles and serde. Obviously results vary hugely depending on workload, and some jobs are simply too big even for a 192 core machine. But I suspect a high proportion of workloads would be better run on single large machines nowadays
varispeed 2y ago
Nowadays very much most services can fit on single server and serve millions of users a day. I wonder how it will affect overly expensive cloud services where you can rent a beefy dedicated server for under a grand and make tens of thousands in savings (enough to hire full time administrator with plenty of money left for other things).
Dylan16807 2y ago
On the other hand, at the time we would have expected twenty years of progress to make the cores a thousand times faster. Instead that number is more like 5x.
- hypercube33 2y ago
  On a different hand the way things were scaling 20 years ago (1ghz took 35 watts) we'd have 5,000W processors - instead we have 196 for 300 watts. If these are anything like ThreadRipper I wonder if they can unlock to 1000W with liquid cooling. On the flip side we are rolling about 1 to 2 watts per core which is wild. Also, can't some of these do 512bit math instructions instead of just 32bit?
  - Dylan16807 2y ago
    Well, a 286 was about 3 watts right? So if our medium point is 35 watts then the final would be 400 watts. Or if our medium point is a 60 watt pentium 4 then our final point would be a 1200 watt single core with multiple millions of times the performance, and dropping down to 300 watts would still be crazy fast.

zer00eyz 2y ago

700+ threads over 2 cores, can saturate 2 400gbe Nic's 500 wats per chip (less than 2 wats per thread)... All of that in a 2U package.... 20 years ago that would have been racks of gear.

smolder 2y ago
> 700+ threads over 2 cores I assume you mean 2 sockets.
jmrm 2y ago
I think those really 2 watts per thread are a lot more important than what us home users usually think. Having to deliver less power and having to dissipate less watts in form of heat in a data centre are really good news to its operative costs, which is usually a lot bigger than the cost of the purchase of the servers
justmarc 2y ago
With these CPUs one can definitely hit much higher rates. 800Gbps from a single server was achieved by Netflix on much lesser systems two years ago: https://nabstreamingsummit.com/wp-content/uploads/2022/05/2022-Streaming-Summit-Netflix.pdf https://nabstreamingsummit.com/wp-content/uploads/2022/05/20... If I were to guess, this hardware can do double that, also helping that we now have actual 800Gbps Ethernet hardware. Indeed 20 years ago this would have been racks of gear at a very high cost and a huge power bill.
- immibis 2y ago
  That's one hell of a pixelflut server.

chx 2y ago

Indeed: the first dual core server chips only launched in 2005 afaik with 90nm Denmark/Italy/Egypt Opterons and Paxville Xeons but on the Intel side it wasn't until 2007 when they were in full swing.

p_l 2y ago
first dual core server chips show up generally available in 2001 with IBM POWER4, then HP PA-RISC ones in 2004, and then Opterons which was followed by "emergency" design of essentially two "sockets" on one die of of NetBurst dual core systems.
- chx 2y ago
  Ehhhh the MAJC 5200 was generally available in 1999 and I am sure even older designs could be found if we were to dig deep enough. Their market share would also need some digging too. To quote the announcement: "two VLIW (very long instruction word) microprocessors on a single piece of silicon"
  - formerly_proven 2y ago
    Power and PA-RISC shipped servers, though. MAJC on the other hand > Sun built a single model of the MAJC, the two-core MAJC 5200, which was the heart of Sun's XVR-1000 and XVR-4000 workstation graphics boards.
    - kristianp 2y ago
      MAJC: https://en.m.wikipedia.org/wiki/MAJC https://en.m.wikipedia.org/wiki/MAJC Why have I not heard of those before?
      - p_l 2y ago
        By the time of XVR GPUs Sun was pretty much exiting the workstation market, and unlike Elite3D and Creator3D the competition has widened for "good enough" PC alternatives using Nvidia and similar chips

RobinL 2y ago

I wonder what percentage of 'big data' jobs that run in clusters would now be far faster on a single big machine with e.g. duckdb rather than spark

conjecTech 2y ago
The difference in throughput for local versus distributed orchestration would mainly come from serdes, networking, switching. Serdes can be substantial. Networking and switching has been aggressively offloaded from CPU through better hardware support. Individual tasks would definitely have better latency, but I'd suspect the impact on throughput/CPU usage might be muted. Of course at the extremes (very small jobs, very large/complex objects being passed) you'd see big gains.
- mtremsal 2y ago
  Would you mind expanding on how SerDes become a bottleneck? I’m not familiar and reading the Wikipedia article wasn’t enough to connect the dots.
  - conjecTech 2y ago
    When you talk between remote machines, you have to translate to a format that can transmitted and distributed between machines(serialization). You then have to undo at the other end(deserialization). If what you are sending along is just a few floats, that can be very cheap. If you're sending along a large nested dictionary or even a full program, not so much. Imagine an example where you have two arrays of 1 billion numbers, and you want to add them pairwise. You could use spark to do that by having each "task" be a single addition. But the time it would take to structure and transmit the 1 billion requests will be many multiples of the amount of time it would take to just do the additions.
- RobinL 2y ago
  By way of a single example, we've been migrating recently from spark to duckdb. Our jobs are not huge, but too big for a single 'normal' machine. We've gone from a 2.5 hour runtime on a cluster of 10 machines (40,vCPU total) to a 15 minute runtime on a 32vCPU single machine. I don't know for sure, but I think this is largely because it eliminates expensive shuffles and serde. Obviously results vary hugely depending on workload, and some jobs are simply too big even for a 192 core machine. But I suspect a high proportion of workloads would be better run on single large machines nowadays

varispeed 2y ago

Nowadays very much most services can fit on single server and serve millions of users a day. I wonder how it will affect overly expensive cloud services where you can rent a beefy dedicated server for under a grand and make tens of thousands in savings (enough to hire full time administrator with plenty of money left for other things).

Dylan16807 2y ago

On the other hand, at the time we would have expected twenty years of progress to make the cores a thousand times faster. Instead that number is more like 5x.

hypercube33 2y ago
On a different hand the way things were scaling 20 years ago (1ghz took 35 watts) we'd have 5,000W processors - instead we have 196 for 300 watts. If these are anything like ThreadRipper I wonder if they can unlock to 1000W with liquid cooling. On the flip side we are rolling about 1 to 2 watts per core which is wild. Also, can't some of these do 512bit math instructions instead of just 32bit?
- Dylan16807 2y ago
  Well, a 286 was about 3 watts right? So if our medium point is 35 watts then the final would be 400 watts. Or if our medium point is a 60 watt pentium 4 then our final point would be a 1200 watt single core with multiple millions of times the performance, and dropping down to 300 watts would still be crazy fast.