4 ms·

I thought TDP was a selling point of ARM for servers. Cavium has a higher TDP (180W) than either of the Intel Xeon's listed in the article (150W, 165W). Is th

I thought TDP was a selling point of ARM for servers. Cavium has a higher TDP (180W) than either of the Intel Xeon's listed in the article (150W, 165W). Is this more about "performance per watt" then? EDIT: Also makes me wonder just how much more energy efficient and/or performant Cavium would be if they moved away from TSMC 16nm fab over to Samsung 10nm fab (like Qualcomm). There's a sizeable difference between 16nm vs 10nm (assuming the fabs are using the same metrics). Qualcomm using Samsung 10nm fab is only 120W.

Symmetry 8y ago
Small microservers with cellphone class cores in them were something a few companies pursued but they never really took off. And there was one company with Atom based servers in the same space, it wasn't just all ARM A9s. This one is about trading off minimum latency for more throughput with lots of more medium sized cores in a way that's probably better suited to most server workloads.
notacoward 8y ago
Yes, it's about performance per watt, and also performance per dollar. The Intel chips lose badly by that measure. Also, TDP isn't necessarily the right figure of merit here. It's defined and measured differently by different vendors, so it's not always directly comparable. The more important and more directly comparable measure is actual system-wide consumption when running a defined workload.
- ajross 8y ago
  > The Intel chips lose badly by that measure. That's true, but then the Intel chips shown in that chart are Xeon Gold/Platinum parts that heretofore have had essentially no competition in the market and thus command huge premiums. Intel is making crazy bank on its server parts, basically. So in the presence of real competition (which this looks likely to be, though to be fair it's not actually in the market yet and is likely priced at a point intended to grab eyeballs), it's likely that Xeon parts will drop in price. You won't be looking at that delta for long if Intel starts losing market share. So... if they were merely cost-equivalent, is this part worth it? Because the pure silicon metrics here (performance per watt and per die area), as mentioned upthread, are sort of a wash. And maturity/stability issues (c.f. the bit in the linked article where they point out that the Cavium board draws 300W at idle!) kill products in the datacenter space all the time. Consumers are OK with occasional glitches, but no one is willing to put anything on their rack that has any kind of failure risk at all.
  - notacoward 8y ago
    > no one is willing to put anything on their rack that has any kind of failure risk at all. Are you sure about that? The people who have hundreds of thousands to millions of machines also have systems designed to handle a certain failure rate, and have historically been quite happy to run on non-premium hardware. The costs do add up. Even if it's just a handful of companies, in terms of market percentage that's far from no one.
    - ajross 8y ago
      It's a standard thing in the enterprise space, yes. No one buys anything that seems flaky. It's true that big datacenter operators (Amazon, Google, Netflix et. al.) have a lot more freedom to handle risk. But regardless, things like the 300W idle draw are going to kill a sale even there. That's almost a dollar a day, so there goes the price delta with Intel over the life of the system already.
      - notacoward 8y ago
        The idle draw is exactly the sort of thing one would expect to be fixed well before volume production. I've been at companies that made hardware (in fact at least one of my co-workers from one of those jobs is at Cavium right now) so I honestly think that's a bit of a red herring. Comparing pre-production hardware with production is always a bit unfair; let's not make it more so.
        ajross 8y ago
        Wait wait wait. If it's preproduction hardware then the prices in the article are meaningless because you can't buy it. If it's unfair to compare performance metrics on preproduction hardware than surely it's equally unfair to compare prices (I mean, they may not even know the yields yet). You don't get to claim "Intel loses badly" and then hide behind "well, they can't have expected to have fixed everything yet!". For comparison, those Xeon parts are Skylake-SP dies that first shipped 11 months ago. You can buy them on Amazon and rent them on AWS. They're a mature product in a well-established channel and their prices are stable and matched to the market. The bottom line is that Cavium pulled out all the stops, shipped a board that works and performs... very adequately, with a few glitches. And frankly I don't know that glitchy acceptability is going to sell a whole lot of servers. So they slapped a price tag on it that draws eyeballs that the performance alone won't. If they can do that it's probably good news for them, but it's still a long road from here to revenue.
jandrewrogers 8y ago
I know many people that have evaluated ARM for a variety of server applications and ARM's advantages, in terms of tradeoffs, are not what many people assume. Where ARM shows well is applications that spend a significant amount of time idle in power-restricted environments (battery-powered analytics clusters being a common example) and running software that is weakly optimized for efficiency (some web app stacks look like this). In practice, this is a niche set of requirements for servers. The challenge that ARM has long had in servers is that highly optimized and efficient Intel codes have competitive performance per watt and performance per dollar. Cavium is quite open that their cores are optimized for low ILP/IPC software, which is essentially arbitraging software that is poorly designed for x86 silicon. As data and analysis scales have increased, more standard server software is specifically designed to be hyper-efficient on Intel, not just HPC codes like the old days. This makes it increasingly difficult for ARM to compete with the operation throughput per watt/dollar that is possible with Intel and well optimized software. There is the additional issue with platforms like ThunderX2 that even though it could be competitive in theory, it would require software to be specifically engineered around the peculiarities of the microarchitecture. Software that assumes Intel microarchitectures and optimizes toward that, either explicitly or implicitly, will inherently be suboptimal on ThunderX2. This isn't something that can be fixed in a compiler, it is intrinsic to the architecture of the software at a higher level e.g. C++ codes embed many assumptions about CPU cache topology and properties.
- namibj 8y ago
  For cache-tuning, there is polyhedral optimization, available in LLVM via Polly and in GCC via Graphite. Of course they don't fix everything, but they are great at automating loop nesting decisions with the goal of matching data locality to cache size(s), and, potentially, even SSD/HDD storage (think compiling the software so that it makes good use of the 1GB ram/core you can give it, and not be awfully inefficient with it's swapping.
  - gnufx 8y ago
    Unfortunately, the polyhedral optimization was still broken in gcc last I tried. Perhaps it's fixed in gcc 8.
floatboth 8y ago
> assuming the fabs are using the same metrics They're not. Nanometers are almost meaningless now.

Symmetry 8y ago

Small microservers with cellphone class cores in them were something a few companies pursued but they never really took off. And there was one company with Atom based servers in the same space, it wasn't just all ARM A9s. This one is about trading off minimum latency for more throughput with lots of more medium sized cores in a way that's probably better suited to most server workloads.

notacoward 8y ago

Yes, it's about performance per watt, and also performance per dollar. The Intel chips lose badly by that measure. Also, TDP isn't necessarily the right figure of merit here. It's defined and measured differently by different vendors, so it's not always directly comparable. The more important and more directly comparable measure is actual system-wide consumption when running a defined workload.

ajross 8y ago
> The Intel chips lose badly by that measure. That's true, but then the Intel chips shown in that chart are Xeon Gold/Platinum parts that heretofore have had essentially no competition in the market and thus command huge premiums. Intel is making crazy bank on its server parts, basically. So in the presence of real competition (which this looks likely to be, though to be fair it's not actually in the market yet and is likely priced at a point intended to grab eyeballs), it's likely that Xeon parts will drop in price. You won't be looking at that delta for long if Intel starts losing market share. So... if they were merely cost-equivalent, is this part worth it? Because the pure silicon metrics here (performance per watt and per die area), as mentioned upthread, are sort of a wash. And maturity/stability issues (c.f. the bit in the linked article where they point out that the Cavium board draws 300W at idle!) kill products in the datacenter space all the time. Consumers are OK with occasional glitches, but no one is willing to put anything on their rack that has any kind of failure risk at all.
- notacoward 8y ago
  > no one is willing to put anything on their rack that has any kind of failure risk at all. Are you sure about that? The people who have hundreds of thousands to millions of machines also have systems designed to handle a certain failure rate, and have historically been quite happy to run on non-premium hardware. The costs do add up. Even if it's just a handful of companies, in terms of market percentage that's far from no one.
  - ajross 8y ago
    It's a standard thing in the enterprise space, yes. No one buys anything that seems flaky. It's true that big datacenter operators (Amazon, Google, Netflix et. al.) have a lot more freedom to handle risk. But regardless, things like the 300W idle draw are going to kill a sale even there. That's almost a dollar a day, so there goes the price delta with Intel over the life of the system already.
    - notacoward 8y ago
      The idle draw is exactly the sort of thing one would expect to be fixed well before volume production. I've been at companies that made hardware (in fact at least one of my co-workers from one of those jobs is at Cavium right now) so I honestly think that's a bit of a red herring. Comparing pre-production hardware with production is always a bit unfair; let's not make it more so.
      - ajross 8y ago
        Wait wait wait. If it's preproduction hardware then the prices in the article are meaningless because you can't buy it. If it's unfair to compare performance metrics on preproduction hardware than surely it's equally unfair to compare prices (I mean, they may not even know the yields yet). You don't get to claim "Intel loses badly" and then hide behind "well, they can't have expected to have fixed everything yet!". For comparison, those Xeon parts are Skylake-SP dies that first shipped 11 months ago. You can buy them on Amazon and rent them on AWS. They're a mature product in a well-established channel and their prices are stable and matched to the market. The bottom line is that Cavium pulled out all the stops, shipped a board that works and performs... very adequately, with a few glitches. And frankly I don't know that glitchy acceptability is going to sell a whole lot of servers. So they slapped a price tag on it that draws eyeballs that the performance alone won't. If they can do that it's probably good news for them, but it's still a long road from here to revenue.

jandrewrogers 8y ago

I know many people that have evaluated ARM for a variety of server applications and ARM's advantages, in terms of tradeoffs, are not what many people assume. Where ARM shows well is applications that spend a significant amount of time idle in power-restricted environments (battery-powered analytics clusters being a common example) and running software that is weakly optimized for efficiency (some web app stacks look like this). In practice, this is a niche set of requirements for servers. The challenge that ARM has long had in servers is that highly optimized and efficient Intel codes have competitive performance per watt and performance per dollar. Cavium is quite open that their cores are optimized for low ILP/IPC software, which is essentially arbitraging software that is poorly designed for x86 silicon. As data and analysis scales have increased, more standard server software is specifically designed to be hyper-efficient on Intel, not just HPC codes like the old days. This makes it increasingly difficult for ARM to compete with the operation throughput per watt/dollar that is possible with Intel and well optimized software. There is the additional issue with platforms like ThunderX2 that even though it could be competitive in theory, it would require software to be specifically engineered around the peculiarities of the microarchitecture. Software that assumes Intel microarchitectures and optimizes toward that, either explicitly or implicitly, will inherently be suboptimal on ThunderX2. This isn't something that can be fixed in a compiler, it is intrinsic to the architecture of the software at a higher level e.g. C++ codes embed many assumptions about CPU cache topology and properties.

namibj 8y ago
For cache-tuning, there is polyhedral optimization, available in LLVM via Polly and in GCC via Graphite. Of course they don't fix everything, but they are great at automating loop nesting decisions with the goal of matching data locality to cache size(s), and, potentially, even SSD/HDD storage (think compiling the software so that it makes good use of the 1GB ram/core you can give it, and not be awfully inefficient with it's swapping.
- gnufx 8y ago
  Unfortunately, the polyhedral optimization was still broken in gcc last I tried. Perhaps it's fixed in gcc 8.

floatboth 8y ago

> assuming the fabs are using the same metrics They're not. Nanometers are almost meaningless now.