6 ms·

Intel Lines Up ThunderX ARM Against Xeons

65 points by baazaar 10y ago

ryao 10y ago
Intel might find that these benchmarks achieve the opposite of what they intended. I had no idea that viable ARMv8 alternatives to Intel's architecture were on the market. There are likely others who were similarly ignorant. Intel has just informed all of us that these options are out there.
- nickpsecurity 10y ago
  I've been doing it a while with Cavium via Octeon's: https://www.cavium.com/OCTEON-III_CN7XXX.html https://www.cavium.com/OCTEON-III_CN7XXX.html Looked awesome. The ThunderX is the Octeon ported to ARM ISA far as I can tell. The Octeon II's with 16 cores plus Gbps I/O and crypto accelerators in PCI card form can be had on eBay for $200-300. Now, I have performance sheets from Intel confirming it performed as well as I hoped. Great news for Cavium. ;)
  - imtringued 10y ago
    Just for the processor? I'm asking because that's enough to buy an AMD CPU + 8GB RAM + a mainboard + powersupply to get 70% of the multicore performance of a intel 6700k (again the cpu costs more than this setup).
    - sliken 10y ago
      I looked at that, but: * AMD no ECC, nor a similar chip with it. Intel has a Xeon E3 which is basically core as the 6700 with ECC enabled. Doesn't hurt that it's cheaper as well. * AMD had 70% of the throughput, but much less on anything single threaded. So many normal UI apps like say web browsing are slower. * the AMD solution runs hotter, so you need to either tolerate more noise or pay more for case/fans/power supplies * My use case includes running 24/7 so burning extra watts (and running the AC to cool them) adds up. After considering it all I went with the E3-1230v5. Near 6700k multicore performance, near 6700k single thread performance, max 80 watt TDP, and idles much lower. I also get a M.2 slot which allows pretty amazing performance (well above SATA) and a tiny footprint (gum stick or so). I don't buy systems often though, the machine it's replacing is a quad core 3.0 GHz machines (Q6600) with 8GB ram from 2007. So I don't mind spending a bit more if it's going to last. Hopefully with 32GB ram this new machine will last quite a while.
      - kornholi 10y ago
        AFAIK, all modern AMD CPUs support ECC. You just need motherboard support.
        ryao 10y ago
        That was true on the AM3+ Processors with the 890FX and 990FX chipsets. I do not believe it is the case on AMD's modern CPU sockets, but I would be happy to be shown to be wrong.
      - nickpsecurity 10y ago
        My machine is about as old given it's a Core Duo 2 with 4GB of RAM. One of the Dell laptops designed for Ubuntu. What do you use yours for where a quad-core 3GHz and 8GB RAM isn't enough now but was a while back? Just curious.
    - nickpsecurity 10y ago
      Intel and AMD are market leaders in raw price/performance. They drop mad money on custom designs at cutting-edge nodes to do it, too. The Cavium chips might be custom but probably standard cell. Cost way less. Further, you get 16 RISC cores with no x86 baggage, they run Linux, PCI form factor, and low power. Onboard are usually accelerators for TCP/IP, compression, encryption, and so on that run in Gbps that your processor cores don't have to handle. The CPU cores themselves run pretty fast. The hybrid seemed like one of the best designs. Probably why AMD started copying it with their semi-custom business followed by Intel. But those are still full-on x86 processors with all kinds of circuits you probably don't need plus maybe some backdoors for AMT, etc. Cavium's design is much more interesting and maybe safer. I've mainly encouraged people to clone it with RISC-V cores rather than straight-up buy them. ;) The newer ones are pretty impressive in performance and accelerators, though: https://www.cavium.com/OCTEON-III_CN7XXX.html https://www.cavium.com/OCTEON-III_CN7XXX.html
- emn13 10y ago
  Although given the relative performance, I can't imagine who'd pick the arm option currently, unless the pricing difference is truly extreme. There's just too many downsides to arm servers at this time: it's not just total overall performance, but particuarly single threaded performance (which affects e.g. latency that your readers experience), power usage (affects costs), and of course availability of software. It's getting close, that's for sure - and there may well be some niche where it already makes sense, but I wouldn't know which one. Probably the best value proposition is that the gap is likely to shrink, and investing now may help you prepare for a future switch when it makes more sense.
  - ryao 10y ago
    > It's getting close, that's for sure - and there may well be some niche where it already makes sense, but I wouldn't know which one. Proprietary blobs such as Intel's ME that cannot be removed from Intel Xeon systems pose a security risk. If the ARM systems are found to lack them and they achieve the status of being good enough, security minded individuals and organizations will buy them because of that. As long as that happens, Cavium will be able to use the revenues to develop the next generation of chips to better compete on traditional metrics like Intel was able to do against the RISC vendors of the past. As a security minded individual myself, I am well aware of that and I look forward to supporting such developments by purchasing such hardware for applications where it can be used. There are others thinking this too. Some of them are watching the Talos workstation motherboard come to market with that in mind. The people behind it are even talking about IBM's executives being willing to make the POWER9 processor's microcode open source if the POWER8 model succeeds. https://www.raptorengineering.com/TALOS/prerelease.php https://www.raptorengineering.com/TALOS/prerelease.php
    - gpderetta 10y ago
      "Proprietary blobs such as Intel's ME that cannot be removed from Intel Xeon systems pose a security risk. If the ARM systems are found to lack them and they achieve the status of being good enough, security minded individuals and organizations will buy them because of that." Do you expect the above to be anything more than a rounding error of the total server market? I doubt Cavium is going to finance its development from that.
- imtringued 10y ago
  >I had no idea that viable ARMv8 alternatives to Intel's architecture were on the market. For some definitions of "on the market". The ARM server chips can be hard to get still.
  - subway 10y ago
    Chips are easy to get. It's all that pesky supporting hardware everyone in the ARM world disagrees on that causes availability issues to end users.
- mtgx 10y ago
  Here's one more, which I think should be the most competitive one out there: https://www.apm.com/news/appliedmicro-announces-x-gene-3-the-industrys-first-armv8-a-finfet-server-s/ https://www.apm.com/news/appliedmicro-announces-x-gene-3-the... http://www.nextplatform.com/2016/04/28/arm-server-chips-xeon-class-matter/ http://www.nextplatform.com/2016/04/28/arm-server-chips-xeon... https://semiaccurate.com/2016/04/25/appliedmicros-x-gene-3-aims-for-intels-e5-xeons/ https://semiaccurate.com/2016/04/25/appliedmicros-x-gene-3-a...
ThenAsNow 10y ago
There seems to be at least one incorrect statement in the article. "The point Intel is making in the chart above is that it has mastered NUMA scaling, which is no surprise since the company has been at it for decades." SGI was one the pioneers of ccNUMA (Origin 2000?) in the late 90s. That would constitute decades. Around the same time, Intel delivered the ASCI Red supercomputer using P6 processors with a snooping coherency scheme. I don't think Intel had significant NUMA experience (at least in any commercial products) prior to AMD K8 (2003), and their first NUMA product would have been IA64 (E8870 chipset?) or the Nehalem microarchitecture chips. That is not quite a decade, let alone decades.
- creshal 10y ago
  I was under the impression that Itanium had it a while longer, but you're right. Itanium got it even later (2010) thanks to tape-out delays.
  - ryao 10y ago
    The SGI Altix 3000 had ccNUMA in 2003, but that was using SGI's ccNUMA technology to connect Intel's microprocessors rather than anything Intel developed. I believe that other UNIX vendors that adopted Intel's chips used their own ccNUMA technology to link them together too. Anyway, the article's author likely made the mistake of thinking that multisocket implies NUMA. It is an easy mistake to make for those who do not know the details of how systems are implemented.
    - ThenAsNow 10y ago
      SGI did it with MIPS processors in the 90s before they used Intel chips. For example: http://dl.acm.org/citation.cfm?doid=264107.264206 http://dl.acm.org/citation.cfm?doid=264107.264206 This was the system in which the directory logic was formally verified: http://www.sgidepot.co.uk/origin/compcon97_dv.pdf http://www.sgidepot.co.uk/origin/compcon97_dv.pdf I agree the author probably made a simple mistake, but it is misleading to people at the "enthusiast" level who then parrot things like this as flamewar fodder. A lot of respect is owed to computer engineers at corporations that are defunct or a fraction of their former greatness (e.g., SGI, DEC, HP). As a further example, Bob Colwell's book, "The Pentium Chronicles", is a great read and teaches some powerful lessons about how to successfully do engineering in the kind of large organizations that can scale production of a design. But if you take it at face value, the book makes no meaningful attempt to inform the reader that other companies had done Out-of-Order well before P6. It leaves a naive reader with the idea Intel pioneered OOO as well, which is no way true. I guess I'm just (maybe overly) sensitive to sloppiness and omission leaving an inaccurate impression of the history, especially when it's so easy these days to fact-check.
jakozaur 10y ago
With Moore's law slowing down, semiconductors can get commoditized. It happened with DRAM business in 1980s: https://en.wikipedia.org/wiki/Intel#From_DRAM_to_microprocessors https://en.wikipedia.org/wiki/Intel#From_DRAM_to_microproces... Maybe CPU margins are going to drop too? ARM is likely the successor. ARM already won mobile market, maybe it start eating some of the cloud market? Do we really care on which CPU architecture runs S3 servers and etc.? That gives cloud providers huge leverage in negotiations. E.g. AWS can demand more discounts for Intel CPUs and if they won't comply slowing, but surely move some portion of it's infrastructure to ARM.
- pjmlp 10y ago
  > Do we really care on which CPU architecture runs S3 servers and etc.? Not at all, specially when the language runtimes are rich enough to abstract OS and hardware architectures. Even GPUs can be treated as an "acceleration device" by rich GPGPU runtimes.
- gpderetta 10y ago
  The problem with comoditization is that margins become razor thin, which in turn means less money to invest in R&D, which means that it is hard for a company that isn't already there to come up with high performance (or even entry level, really) server solutions. Intel ate the server market in the late '90/eary '00 thanks to the fat margins it had on the desktop. That's not currently the case in the ARM world were only Apple and maybe Qualcomm have the needed margins, and Apple currently doesn't seem interested in the server market.
  - bhouston 10y ago
    If ARM is providing the expensive R&D-based IP, then any licensee can be a competitor.
    - gpderetta 10y ago
      But AFAIK ARM hasn't provided any server specific design. Most ARM based server attempts so far have been with custom designs (e.g. AppliedMicro X-Gene).

65 points by baazaar 10y ago

ryao 10y ago

Intel might find that these benchmarks achieve the opposite of what they intended. I had no idea that viable ARMv8 alternatives to Intel's architecture were on the market. There are likely others who were similarly ignorant. Intel has just informed all of us that these options are out there.

nickpsecurity 10y ago
I've been doing it a while with Cavium via Octeon's: https://www.cavium.com/OCTEON-III_CN7XXX.html https://www.cavium.com/OCTEON-III_CN7XXX.html Looked awesome. The ThunderX is the Octeon ported to ARM ISA far as I can tell. The Octeon II's with 16 cores plus Gbps I/O and crypto accelerators in PCI card form can be had on eBay for $200-300. Now, I have performance sheets from Intel confirming it performed as well as I hoped. Great news for Cavium. ;)
- imtringued 10y ago
  Just for the processor? I'm asking because that's enough to buy an AMD CPU + 8GB RAM + a mainboard + powersupply to get 70% of the multicore performance of a intel 6700k (again the cpu costs more than this setup).
  - sliken 10y ago
    I looked at that, but: * AMD no ECC, nor a similar chip with it. Intel has a Xeon E3 which is basically core as the 6700 with ECC enabled. Doesn't hurt that it's cheaper as well. * AMD had 70% of the throughput, but much less on anything single threaded. So many normal UI apps like say web browsing are slower. * the AMD solution runs hotter, so you need to either tolerate more noise or pay more for case/fans/power supplies * My use case includes running 24/7 so burning extra watts (and running the AC to cool them) adds up. After considering it all I went with the E3-1230v5. Near 6700k multicore performance, near 6700k single thread performance, max 80 watt TDP, and idles much lower. I also get a M.2 slot which allows pretty amazing performance (well above SATA) and a tiny footprint (gum stick or so). I don't buy systems often though, the machine it's replacing is a quad core 3.0 GHz machines (Q6600) with 8GB ram from 2007. So I don't mind spending a bit more if it's going to last. Hopefully with 32GB ram this new machine will last quite a while.
    - kornholi 10y ago
      AFAIK, all modern AMD CPUs support ECC. You just need motherboard support.
      - ryao 10y ago
        That was true on the AM3+ Processors with the 890FX and 990FX chipsets. I do not believe it is the case on AMD's modern CPU sockets, but I would be happy to be shown to be wrong.
    - nickpsecurity 10y ago
      My machine is about as old given it's a Core Duo 2 with 4GB of RAM. One of the Dell laptops designed for Ubuntu. What do you use yours for where a quad-core 3GHz and 8GB RAM isn't enough now but was a while back? Just curious.
  - nickpsecurity 10y ago
    Intel and AMD are market leaders in raw price/performance. They drop mad money on custom designs at cutting-edge nodes to do it, too. The Cavium chips might be custom but probably standard cell. Cost way less. Further, you get 16 RISC cores with no x86 baggage, they run Linux, PCI form factor, and low power. Onboard are usually accelerators for TCP/IP, compression, encryption, and so on that run in Gbps that your processor cores don't have to handle. The CPU cores themselves run pretty fast. The hybrid seemed like one of the best designs. Probably why AMD started copying it with their semi-custom business followed by Intel. But those are still full-on x86 processors with all kinds of circuits you probably don't need plus maybe some backdoors for AMT, etc. Cavium's design is much more interesting and maybe safer. I've mainly encouraged people to clone it with RISC-V cores rather than straight-up buy them. ;) The newer ones are pretty impressive in performance and accelerators, though: https://www.cavium.com/OCTEON-III_CN7XXX.html https://www.cavium.com/OCTEON-III_CN7XXX.html
emn13 10y ago
Although given the relative performance, I can't imagine who'd pick the arm option currently, unless the pricing difference is truly extreme. There's just too many downsides to arm servers at this time: it's not just total overall performance, but particuarly single threaded performance (which affects e.g. latency that your readers experience), power usage (affects costs), and of course availability of software. It's getting close, that's for sure - and there may well be some niche where it already makes sense, but I wouldn't know which one. Probably the best value proposition is that the gap is likely to shrink, and investing now may help you prepare for a future switch when it makes more sense.
- ryao 10y ago
  > It's getting close, that's for sure - and there may well be some niche where it already makes sense, but I wouldn't know which one. Proprietary blobs such as Intel's ME that cannot be removed from Intel Xeon systems pose a security risk. If the ARM systems are found to lack them and they achieve the status of being good enough, security minded individuals and organizations will buy them because of that. As long as that happens, Cavium will be able to use the revenues to develop the next generation of chips to better compete on traditional metrics like Intel was able to do against the RISC vendors of the past. As a security minded individual myself, I am well aware of that and I look forward to supporting such developments by purchasing such hardware for applications where it can be used. There are others thinking this too. Some of them are watching the Talos workstation motherboard come to market with that in mind. The people behind it are even talking about IBM's executives being willing to make the POWER9 processor's microcode open source if the POWER8 model succeeds. https://www.raptorengineering.com/TALOS/prerelease.php https://www.raptorengineering.com/TALOS/prerelease.php
  - gpderetta 10y ago
    "Proprietary blobs such as Intel's ME that cannot be removed from Intel Xeon systems pose a security risk. If the ARM systems are found to lack them and they achieve the status of being good enough, security minded individuals and organizations will buy them because of that." Do you expect the above to be anything more than a rounding error of the total server market? I doubt Cavium is going to finance its development from that.
imtringued 10y ago
>I had no idea that viable ARMv8 alternatives to Intel's architecture were on the market. For some definitions of "on the market". The ARM server chips can be hard to get still.
- subway 10y ago
  Chips are easy to get. It's all that pesky supporting hardware everyone in the ARM world disagrees on that causes availability issues to end users.
mtgx 10y ago
Here's one more, which I think should be the most competitive one out there: https://www.apm.com/news/appliedmicro-announces-x-gene-3-the-industrys-first-armv8-a-finfet-server-s/ https://www.apm.com/news/appliedmicro-announces-x-gene-3-the... http://www.nextplatform.com/2016/04/28/arm-server-chips-xeon-class-matter/ http://www.nextplatform.com/2016/04/28/arm-server-chips-xeon... https://semiaccurate.com/2016/04/25/appliedmicros-x-gene-3-aims-for-intels-e5-xeons/ https://semiaccurate.com/2016/04/25/appliedmicros-x-gene-3-a...

ThenAsNow 10y ago

There seems to be at least one incorrect statement in the article. "The point Intel is making in the chart above is that it has mastered NUMA scaling, which is no surprise since the company has been at it for decades." SGI was one the pioneers of ccNUMA (Origin 2000?) in the late 90s. That would constitute decades. Around the same time, Intel delivered the ASCI Red supercomputer using P6 processors with a snooping coherency scheme. I don't think Intel had significant NUMA experience (at least in any commercial products) prior to AMD K8 (2003), and their first NUMA product would have been IA64 (E8870 chipset?) or the Nehalem microarchitecture chips. That is not quite a decade, let alone decades.

creshal 10y ago
I was under the impression that Itanium had it a while longer, but you're right. Itanium got it even later (2010) thanks to tape-out delays.
- ryao 10y ago
  The SGI Altix 3000 had ccNUMA in 2003, but that was using SGI's ccNUMA technology to connect Intel's microprocessors rather than anything Intel developed. I believe that other UNIX vendors that adopted Intel's chips used their own ccNUMA technology to link them together too. Anyway, the article's author likely made the mistake of thinking that multisocket implies NUMA. It is an easy mistake to make for those who do not know the details of how systems are implemented.
  - ThenAsNow 10y ago
    SGI did it with MIPS processors in the 90s before they used Intel chips. For example: http://dl.acm.org/citation.cfm?doid=264107.264206 http://dl.acm.org/citation.cfm?doid=264107.264206 This was the system in which the directory logic was formally verified: http://www.sgidepot.co.uk/origin/compcon97_dv.pdf http://www.sgidepot.co.uk/origin/compcon97_dv.pdf I agree the author probably made a simple mistake, but it is misleading to people at the "enthusiast" level who then parrot things like this as flamewar fodder. A lot of respect is owed to computer engineers at corporations that are defunct or a fraction of their former greatness (e.g., SGI, DEC, HP). As a further example, Bob Colwell's book, "The Pentium Chronicles", is a great read and teaches some powerful lessons about how to successfully do engineering in the kind of large organizations that can scale production of a design. But if you take it at face value, the book makes no meaningful attempt to inform the reader that other companies had done Out-of-Order well before P6. It leaves a naive reader with the idea Intel pioneered OOO as well, which is no way true. I guess I'm just (maybe overly) sensitive to sloppiness and omission leaving an inaccurate impression of the history, especially when it's so easy these days to fact-check.

jakozaur 10y ago

With Moore's law slowing down, semiconductors can get commoditized. It happened with DRAM business in 1980s: https://en.wikipedia.org/wiki/Intel#From_DRAM_to_microprocessors https://en.wikipedia.org/wiki/Intel#From_DRAM_to_microproces... Maybe CPU margins are going to drop too? ARM is likely the successor. ARM already won mobile market, maybe it start eating some of the cloud market? Do we really care on which CPU architecture runs S3 servers and etc.? That gives cloud providers huge leverage in negotiations. E.g. AWS can demand more discounts for Intel CPUs and if they won't comply slowing, but surely move some portion of it's infrastructure to ARM.

pjmlp 10y ago
> Do we really care on which CPU architecture runs S3 servers and etc.? Not at all, specially when the language runtimes are rich enough to abstract OS and hardware architectures. Even GPUs can be treated as an "acceleration device" by rich GPGPU runtimes.
gpderetta 10y ago
The problem with comoditization is that margins become razor thin, which in turn means less money to invest in R&D, which means that it is hard for a company that isn't already there to come up with high performance (or even entry level, really) server solutions. Intel ate the server market in the late '90/eary '00 thanks to the fat margins it had on the desktop. That's not currently the case in the ARM world were only Apple and maybe Qualcomm have the needed margins, and Apple currently doesn't seem interested in the server market.
- bhouston 10y ago
  If ARM is providing the expensive R&D-based IP, then any licensee can be a competitor.
  - gpderetta 10y ago
    But AFAIK ARM hasn't provided any server specific design. Most ARM based server attempts so far have been with custom designs (e.g. AppliedMicro X-Gene).