4 ms·

Shouldn’t they instead figure out how to run cooler and with less power in general? That’s where it seems everyone else is going…

booi 4y ago
I need… more powahhh… No but seriously why don’t they build a 128-core atom server. That’s really all anybody wants. I don’t need the fastest most immersed cpu ever, just a bunch of decent ones at 30W or less.
- tadfisher 4y ago
  That was Larrabee/Xeon Phi, was it not? Discontinued for lack of sales.
  - sseagull 4y ago
    That’s basically what I thought. However, IIRC they were marketed more towards high-performance computing (with avx-512). They were an uncomfortable middle ground though, between normal CPUs and GPUs. My benchmarks showed that there wasn’t much of an advantage over 20-ish normal xeon cores (for my HPC workloads). (Memory is a little fuzzy - that was 4-6 years ago).
    - glowingly 4y ago
      While not exactly what you are looking for, Intel Snow Ridge is a continuation of their Atom-based (next to their line of Core-based) networking processors. 8-24 cores. https://www.intel.com/content/www/us/en/products/details/processors/atom/p.html https://www.intel.com/content/www/us/en/products/details/pro... Though, unless if you 100% need X86, there is the Ampere Altra 128 core Cortex-N1 chip.
- astrange 4y ago
  That’d be a pretty unbalanced architecture - may use less rack space than 128 servers, but with only one servers’ worth of IO, network, PSUs, it’d be less reliable and maybe not even faster.
  - booi 4y ago
    A single AWS graviton server is roughly 128 cores with, I believe, 2x or 4x 40gbit networking. It’s not crazy really.
  - Dylan16807 4y ago
    One core is probably not pushing more than 100mbit on average. Scaling that much networking up a hundredfold is child's play.
- matja 4y ago
  AMD's plan with EPYC Zen 4 "Bergamo", with up to 128 Zen 4c cores.
- tyrfing 4y ago
  128 cores at 30 watts isn't something I've seen anyone planning. What's more likely is 128 cores at 300-400+ watts, and scaling from there is most likely to increase power usage and core counts. Bergamo (AMD), Graviton (AWS), Sierra Forest (Intel), Grace (NVIDIA) are all going for that. 30 watts is low power mobile and "edge" compute.
  - booi 4y ago
    You’re definitely right. I meant 30W per cpu or less which usually means maybe 4-8 cpus per host or ~300W. I did mean to compare it to Graviton and will definitely look at the other contenders in this space.
    - tyrfing 4y ago
      Oh OK. Recent server CPUs (Ice Lake) are roughly 8-12W (maybe up to 15W, there are a lot of SKUs) as a baseline. AMD's latest Milan generation brings it down to 3.5W/core, with some closer to 5W; Intel's Sapphire Rapids only goes down to ~ 6W. After M1 and Graviton, there's a big rush for making denser server CPUs, pretty much bifurcating the market. Much more powerful than old Atom cores, but more power-efficient than regular ones, and targeted mostly at hyperscalers. AMD should have a dense variant of Zen 4 out next year, going up to 128 cores and reducing per-core power significantly. Intel will likely have a competitor out in 2024, and AMD might hit 256 cores then, although rumors of things that far out aren't very specific. On the ARM side of things, NVIDIA is launching CPUs like this next year. Qualcomm bought Nuvia, which was working on similar server CPUs, but probably switched focus to PCs. There are some other oddballs like Tenstorrent as well. One interesting part with pushing per-core power so low is that communication between cores starts to be a serious part of total power consumption.
- Dylan16807 4y ago
  Atoms generally use 3-5 watts per core. So the biggest CPUs they're making are on par with atoms. They keep designing chips that use less and less per core. Why not cram as many cores as possible into a single server? Lower density means more wasted material, and once you go below 2GHz you're not saving very much power any more.
  - c_o_n_v_e_x 4y ago
    > Why not cram as many cores as possible into a single server? Manufacturing yields
    - Dylan16807 4y ago
      That's why you limit die size to a certain extent. But that's not a limiting factor on cores per server. You can cram many more 150mm² dies into your server's sockets than you can actually feed with power and cooling and RAM.
      - xxs 4y ago
        A major issue is the cache coherency. If you dont have coherency, effectively you have an architecture that just multiple distinct servers.
        Dylan16807 4y ago
        You can run coherency protocols across dies. How many cores you decide to make coherent is largely unrelated to how your die yields work out, as far as I know. Not that it matters either way for this discussion? Whether it's "one server" in that 2U box, or "twenty servers" in that box, you're still shoving in lots of cores and lots of watts to get that density up.
- benbenolson 4y ago
  That's almost exactly what the Xeon Phi was, both in coprocessor form and in machine form. It included MCDRAM, too, which was a high-bandwidth memory. Very efficient, but lacked sales; people didn't want it. Part of that, I think, was lack of parallelism in applications: in order to fully take advantage of those cores, you need to have a nearly-embarrassingly-parallel problem. Otherwise, you're not going to get the performance that you'd expect (but you'll get power efficiency!). The fact of the matter is, that's not "all anybody wants."
  - adgjlsfhk1 4y ago
    also, of you're problem is embarrassingly parallel, you can probably run it on a GPU and it's will be faster.
icegreentea2 4y ago
Intel (and everyone else) do work on improving compute efficiency. But as the article points out, if 40% of your DC's power consumption is in cooling, then you'd be foolish not to target that slice. Liquid and immersion cooling allows higher power density, which all things being equal (I know there's a lot of heavy lifting being done by this...) will be preferred. Why distribute your components over a rack if you could fit it into a single 4U board? Why distribute your components over an aisle if you could fit it into a rack?
- lumost 4y ago
  The other advantage of power density is that it creates stronger convection currents. Whereas data centers have traditionally been actively cooled, it’s not unreasonable to imagine open air dcs with air channels to support convection.
  - walrus01 4y ago
    it's sort of been done, though there are still a lot of active fans to move the hot air. https://www.google.com/search?channel=fs&client=ubuntu&q=chicken+coop+datacenter+ https://www.google.com/search?channel=fs&client=ubuntu&q=chi...
voldacar 4y ago
There is only so much computation you can do per watt on current process nodes. To increase our computation per chip, which is the goal, we need to increase the amount of watts we consume per chip. The goal should be to make more powerful processors, not ones that do the same with less power.
- temac 4y ago
  > we need to increase the amount of watts we consume per chip. Not sure we need that, except in niches. At scale you often want at least some efficiency, which is certainly not max TDP per core (because the best efficiency point is with lower frequencies and higher width, not the max freq you can achieve). So remains the question of large number of cores, but at some point the area of silicon also goes stupid high. And you can put multiple packages, without sacrificing overall system density too much, and without departing from simpler, and probably lower TCO pollution. For small systems it depends, but you actually often have even more limited thermal budget, except again in niches if you are ready to tolerate the drawbacks (stupid power req it even becomes hard to have just a few machines on a basic electrical network in standard homes or offices, high noise under load, obviously high TDP so heating up a lot). But you have less space constraints so if you really want absurd systems you already can. So do we really need to e.g. double or triple the (electrical/thermal) power density at scale? Do we need 2 kW chips? Do we need to sacrifice the efficiency now, and increase the nominal consumption now, instead of waiting just a few years for node improvements? (And I could even ask: do we really need that much increase of processing power, shouldn't we start to optimise for the total ecological cost instead? and I've not tried to do some prospective in that area but maybe this would mean slowing down the processing power growth...)
  - voldacar 4y ago
    >So do we really need to e.g. double or triple the (electrical/thermal) power density at scale? Will we ever be able to double or triple our (general purpose) compute per cpu any other way? Moore's law is essentially over. Node improvements aren't really happening outside of TSMC, which doesn't have enough manufacturing ability to supply everyone, and even then those node improvements are getting more and more incremental. And regarding power consumption, I think we really need to be consuming more energy across most sectors of human activity. The most likely explanation for the "great stagnation" is that our energy consumption has basically flatlined since the 70s. It appears that on a civilizational scale, reaching greater levels of development and expression simply requires more Joules. If you disagree, I highly recommend the book Where is my Flying Car?
dev_tty01 4y ago
Absolutely. This is the wrong direction. Lots of power to run the chips, then lots more power to cool them. The ARM server market is eventually going to eat them alive if they can't rearchitect and seriously up their performance/watt game. I hope this is just an intermediate term solution.
- imtringued 4y ago
  What ARM server market? There is at best one ARM server chip per hyperscaler and most have zero ARM server chips. Conventional ARM server efforts failed because the ARM server vendors don't want to talk to mortals, only a very limited set of customers of which most intend to design their own chips and obsolete them. The amount of companies that failed to make their ARM servers available to the public is astonishing. Linus Torvalds said that ARM needs to be widespread on the desktop, because you need a critical mass of developers targeting ARM. ARM server vendors don't want that critical mass, they want special deals with a handful of big companies which is obviously doomed to fail.

booi 4y ago

I need… more powahhh… No but seriously why don’t they build a 128-core atom server. That’s really all anybody wants. I don’t need the fastest most immersed cpu ever, just a bunch of decent ones at 30W or less.

tadfisher 4y ago
That was Larrabee/Xeon Phi, was it not? Discontinued for lack of sales.
- sseagull 4y ago
  That’s basically what I thought. However, IIRC they were marketed more towards high-performance computing (with avx-512). They were an uncomfortable middle ground though, between normal CPUs and GPUs. My benchmarks showed that there wasn’t much of an advantage over 20-ish normal xeon cores (for my HPC workloads). (Memory is a little fuzzy - that was 4-6 years ago).
  - glowingly 4y ago
    While not exactly what you are looking for, Intel Snow Ridge is a continuation of their Atom-based (next to their line of Core-based) networking processors. 8-24 cores. https://www.intel.com/content/www/us/en/products/details/processors/atom/p.html https://www.intel.com/content/www/us/en/products/details/pro... Though, unless if you 100% need X86, there is the Ampere Altra 128 core Cortex-N1 chip.
astrange 4y ago
That’d be a pretty unbalanced architecture - may use less rack space than 128 servers, but with only one servers’ worth of IO, network, PSUs, it’d be less reliable and maybe not even faster.
- booi 4y ago
  A single AWS graviton server is roughly 128 cores with, I believe, 2x or 4x 40gbit networking. It’s not crazy really.
- Dylan16807 4y ago
  One core is probably not pushing more than 100mbit on average. Scaling that much networking up a hundredfold is child's play.
matja 4y ago
AMD's plan with EPYC Zen 4 "Bergamo", with up to 128 Zen 4c cores.
tyrfing 4y ago
128 cores at 30 watts isn't something I've seen anyone planning. What's more likely is 128 cores at 300-400+ watts, and scaling from there is most likely to increase power usage and core counts. Bergamo (AMD), Graviton (AWS), Sierra Forest (Intel), Grace (NVIDIA) are all going for that. 30 watts is low power mobile and "edge" compute.
- booi 4y ago
  You’re definitely right. I meant 30W per cpu or less which usually means maybe 4-8 cpus per host or ~300W. I did mean to compare it to Graviton and will definitely look at the other contenders in this space.
  - tyrfing 4y ago
    Oh OK. Recent server CPUs (Ice Lake) are roughly 8-12W (maybe up to 15W, there are a lot of SKUs) as a baseline. AMD's latest Milan generation brings it down to 3.5W/core, with some closer to 5W; Intel's Sapphire Rapids only goes down to ~ 6W. After M1 and Graviton, there's a big rush for making denser server CPUs, pretty much bifurcating the market. Much more powerful than old Atom cores, but more power-efficient than regular ones, and targeted mostly at hyperscalers. AMD should have a dense variant of Zen 4 out next year, going up to 128 cores and reducing per-core power significantly. Intel will likely have a competitor out in 2024, and AMD might hit 256 cores then, although rumors of things that far out aren't very specific. On the ARM side of things, NVIDIA is launching CPUs like this next year. Qualcomm bought Nuvia, which was working on similar server CPUs, but probably switched focus to PCs. There are some other oddballs like Tenstorrent as well. One interesting part with pushing per-core power so low is that communication between cores starts to be a serious part of total power consumption.
Dylan16807 4y ago
Atoms generally use 3-5 watts per core. So the biggest CPUs they're making are on par with atoms. They keep designing chips that use less and less per core. Why not cram as many cores as possible into a single server? Lower density means more wasted material, and once you go below 2GHz you're not saving very much power any more.
- c_o_n_v_e_x 4y ago
  > Why not cram as many cores as possible into a single server? Manufacturing yields
  - Dylan16807 4y ago
    That's why you limit die size to a certain extent. But that's not a limiting factor on cores per server. You can cram many more 150mm² dies into your server's sockets than you can actually feed with power and cooling and RAM.
    - xxs 4y ago
      A major issue is the cache coherency. If you dont have coherency, effectively you have an architecture that just multiple distinct servers.
      - Dylan16807 4y ago
        You can run coherency protocols across dies. How many cores you decide to make coherent is largely unrelated to how your die yields work out, as far as I know. Not that it matters either way for this discussion? Whether it's "one server" in that 2U box, or "twenty servers" in that box, you're still shoving in lots of cores and lots of watts to get that density up.
benbenolson 4y ago
That's almost exactly what the Xeon Phi was, both in coprocessor form and in machine form. It included MCDRAM, too, which was a high-bandwidth memory. Very efficient, but lacked sales; people didn't want it. Part of that, I think, was lack of parallelism in applications: in order to fully take advantage of those cores, you need to have a nearly-embarrassingly-parallel problem. Otherwise, you're not going to get the performance that you'd expect (but you'll get power efficiency!). The fact of the matter is, that's not "all anybody wants."
- adgjlsfhk1 4y ago
  also, of you're problem is embarrassingly parallel, you can probably run it on a GPU and it's will be faster.

icegreentea2 4y ago

Intel (and everyone else) do work on improving compute efficiency. But as the article points out, if 40% of your DC's power consumption is in cooling, then you'd be foolish not to target that slice. Liquid and immersion cooling allows higher power density, which all things being equal (I know there's a lot of heavy lifting being done by this...) will be preferred. Why distribute your components over a rack if you could fit it into a single 4U board? Why distribute your components over an aisle if you could fit it into a rack?

lumost 4y ago
The other advantage of power density is that it creates stronger convection currents. Whereas data centers have traditionally been actively cooled, it’s not unreasonable to imagine open air dcs with air channels to support convection.
- walrus01 4y ago
  it's sort of been done, though there are still a lot of active fans to move the hot air. https://www.google.com/search?channel=fs&client=ubuntu&q=chicken+coop+datacenter+ https://www.google.com/search?channel=fs&client=ubuntu&q=chi...

voldacar 4y ago

There is only so much computation you can do per watt on current process nodes. To increase our computation per chip, which is the goal, we need to increase the amount of watts we consume per chip. The goal should be to make more powerful processors, not ones that do the same with less power.

temac 4y ago
> we need to increase the amount of watts we consume per chip. Not sure we need that, except in niches. At scale you often want at least some efficiency, which is certainly not max TDP per core (because the best efficiency point is with lower frequencies and higher width, not the max freq you can achieve). So remains the question of large number of cores, but at some point the area of silicon also goes stupid high. And you can put multiple packages, without sacrificing overall system density too much, and without departing from simpler, and probably lower TCO pollution. For small systems it depends, but you actually often have even more limited thermal budget, except again in niches if you are ready to tolerate the drawbacks (stupid power req it even becomes hard to have just a few machines on a basic electrical network in standard homes or offices, high noise under load, obviously high TDP so heating up a lot). But you have less space constraints so if you really want absurd systems you already can. So do we really need to e.g. double or triple the (electrical/thermal) power density at scale? Do we need 2 kW chips? Do we need to sacrifice the efficiency now, and increase the nominal consumption now, instead of waiting just a few years for node improvements? (And I could even ask: do we really need that much increase of processing power, shouldn't we start to optimise for the total ecological cost instead? and I've not tried to do some prospective in that area but maybe this would mean slowing down the processing power growth...)
- voldacar 4y ago
  >So do we really need to e.g. double or triple the (electrical/thermal) power density at scale? Will we ever be able to double or triple our (general purpose) compute per cpu any other way? Moore's law is essentially over. Node improvements aren't really happening outside of TSMC, which doesn't have enough manufacturing ability to supply everyone, and even then those node improvements are getting more and more incremental. And regarding power consumption, I think we really need to be consuming more energy across most sectors of human activity. The most likely explanation for the "great stagnation" is that our energy consumption has basically flatlined since the 70s. It appears that on a civilizational scale, reaching greater levels of development and expression simply requires more Joules. If you disagree, I highly recommend the book Where is my Flying Car?

dev_tty01 4y ago

Absolutely. This is the wrong direction. Lots of power to run the chips, then lots more power to cool them. The ARM server market is eventually going to eat them alive if they can't rearchitect and seriously up their performance/watt game. I hope this is just an intermediate term solution.

imtringued 4y ago
What ARM server market? There is at best one ARM server chip per hyperscaler and most have zero ARM server chips. Conventional ARM server efforts failed because the ARM server vendors don't want to talk to mortals, only a very limited set of customers of which most intend to design their own chips and obsolete them. The amount of companies that failed to make their ARM servers available to the public is astonishing. Linus Torvalds said that ARM needs to be widespread on the desktop, because you need a critical mass of developers targeting ARM. ARM server vendors don't want that critical mass, they want special deals with a handful of big companies which is obviously doomed to fail.