4 ms·

There are lies, damned lies, and benchmarks. The Xeon machine is running at under 15% utilization due to the gigabit ethernet bottleneck, yet they're using the

There are lies, damned lies, and benchmarks. The Xeon machine is running at under 15% utilization due to the gigabit ethernet bottleneck, yet they're using the system TDP (that's the worst case thermal specs which you design your cooling around). If you put a 10 gigabit NIC in there, the advantage would be closer to 2x performance/watt, since the Xeon would be processing 6.66x as many requests (1/0.15). This is also a last generation Xeon - Ivy will be out by the time this CPU is. That they're not just taking the power at the wall of both systems probably means doing so would tip the balance closer to the Xeon - and you pay for at the wall power, so that's what matters. In addition, their choice of 4gb for the platform is worrying; it's almost certainly a 32 bit machine. Which rules out an almost perfect use case for the machine; a memcache server. ARM in servers is coming, to be sure. 64 bit ARM will bring unprecedented gigabytes of ram per watt. But toe to toe on CPU heavy tasks, they will still lose out badly on performance/watt at full load. At least from the numbers in the post, I'd stick with virtualisation.

mykhal 14y ago
ad RAM: the platform (if it is real) is little bit more interesting - they have cards with 16GB RAM per four 4-core CPUs (SoCs) - these cards are to be used together in amount of tens, connected with high speed switched fabric in the host board.. so it's something like server cluster on board design.. http://www.calxeda.com/technology/products/energycards/quadnode/ http://www.calxeda.com/technology/products/energycards/quadn...
- reitzensteinm 14y ago
  Now that's interesting. Why on earth aren't they leading with it? I can understand these kinds of faux benchmarks and whitepapers to convince pointy haired bosses to switch between mature, mainstream technologies. But the early adopters of fringe technology like this are going to be companies with specific needs, and those making the decisions are going to be highly technical. Not to generalize, but they (me included) love details and possibilities, and abhor marketing puffery. Calxeda should be sketching out novel ways to use the interconnect bandwidth to solve hard problems, with power efficiency x86 can't touch. Not running ab against an ARM core and misinterpreting the results. Sheesh.
  - mykhal 14y ago
    well, there is a promising HP project Moonshot, which was going to build on this platform, but it seems that they recently decided to switch to Intel solution.. http://www.engadget.com/2012/06/20/project-moonshot-take-two-hps-low-power-gemini-servers-let-go/ http://www.engadget.com/2012/06/20/project-moonshot-take-two...
  - brooksbp 14y ago
    Calxeda five XAUI (50 Gbps) to a 4 core 1.1GHz 5W. 200ns per hop. A ToR switch will slow you down to 2000-3500ns. Tilera TilePro64, 64 core 866MHz 22W, 1.7Tbps core-to-core, 46ns L1/L2 miss found in adjacent core, four XAUI (40 Gbps). When compared to Tilera, Calxeda has a high bandwith per compute ratio. But 200ns core-to-core is not as fast as 46ns. Tilera also has higher core/watt ratio.
- zokier 14y ago
  I think you misread that page. I think they mean that each SoC has one memory slot, which supports max 4GB RAM. So that 16 gigs is total amount for all the four CPUs. This still strongly points towards a 32-bit design.
  - mykhal 14y ago
    16 GB per 4 CPUs does not mean 4GB per CPU? sorry for my english.. :)
    - hatcravat 14y ago
      16 GB shared among 4 CPUs is different from 4 CPUs which each access 4 GB of private memory.
      - unwind 14y ago
        The page says: Support for 16GB DDR3/3L memory via four (4) mini-DIMM sockets – one socket dedicated per SoC supports up to 4GB of ECC memory (supplied separately) Which really doesn't sound as if the memory is shared, to me.
mtgx 14y ago
They don't need to beat Intel "at the high-end" for "CPU intensive tasks". They just need to find their own market that requires low energy usage over high performance, and of course better performance/Watt, which I do think ARM beats Intel in. But Cortex A9 was never really meant to be a server CPU, so I would wait at least until Cortex A15, or even better for the 64 bit chips that are coming in 2014 (Cortex A15's successor, Nvidia's Project Denver, etc).
- reitzensteinm 14y ago
  I have to say a big [citation needed] to the claim of ARM beating Intel's high end chips on performance per watt, at least on general workloads. I think it's common to extrapolate Atom vs ARM to Xeon vs ARM in HPC, without thinking through the implications. We may well get higher performance/watt for single threads under ARM - I'm not disputing that, especially for integer work. However, Amahdl's law is going to raise its head. In the same machine, a higher number of lower performance threads is going to cause lock contention. You'll also have to split computations over more boxes, since the absolute performance of an Intel server will remain far higher (by 2014, we're talking 64 core/128 thread Haswell). Both of which are likely to be a massive tax on performance. To fight this, performance per core is likely to see a substantial rise, both in clock frequencies, and as a result of single core complexity. However, this will directly work against the two things that makes ARM performance/watt so impressive currently. Also, Intel's entire company is built around making those 100 watt scale processors fast and well. They really stumbled entering the Atom market; both because of a weak design (the chipset drew more power than the CPU itself!), as well as a lack of commitment (using 2-4 year old process nodes). I think we're likely to see a similar teething pains with companies trying to enter the server market for ARM. The instituational knowledge just won't be there. Make a cache architecture that effectively feeds 64 cores? Way different to improving power drain on a mobile CPU, for the seventh generation. I expect it will be at least a few generations before design teams are fully up to speed. Remember that AMD is reasonably well funded, also focused around server CPUs, and often stumbles. AMD is competent; Intel just makes them look incompetent by comparison. I'm not saying we won't see certain workloads that are better off under ARM; memcached and static http serving are both likely to do well, since they're effectively just shuffling bits around, aren't particularly CPU intensive, and are embarassingly parallel. But I believe they'll turn out to be the exception, not the rule. Which is to say, there's nothing magic about ARM that will let them beat x86 at the high end. They'll have to fight for it, and against Intel on their own turf no less.
mercuryrising 14y ago
What is the performance difference between 32 bit and 64 bit for memcached? I know redis has a lot of overhead with the 64 bit version [1], does memcached suffer the same fate? [1] http://serverfault.com/questions/221695/deploying-memcached-as-32bit-or-64bit http://serverfault.com/questions/221695/deploying-memcached-...

mykhal 14y ago

ad RAM: the platform (if it is real) is little bit more interesting - they have cards with 16GB RAM per four 4-core CPUs (SoCs) - these cards are to be used together in amount of tens, connected with high speed switched fabric in the host board.. so it's something like server cluster on board design.. http://www.calxeda.com/technology/products/energycards/quadnode/ http://www.calxeda.com/technology/products/energycards/quadn...

reitzensteinm 14y ago
Now that's interesting. Why on earth aren't they leading with it? I can understand these kinds of faux benchmarks and whitepapers to convince pointy haired bosses to switch between mature, mainstream technologies. But the early adopters of fringe technology like this are going to be companies with specific needs, and those making the decisions are going to be highly technical. Not to generalize, but they (me included) love details and possibilities, and abhor marketing puffery. Calxeda should be sketching out novel ways to use the interconnect bandwidth to solve hard problems, with power efficiency x86 can't touch. Not running ab against an ARM core and misinterpreting the results. Sheesh.
- mykhal 14y ago
  well, there is a promising HP project Moonshot, which was going to build on this platform, but it seems that they recently decided to switch to Intel solution.. http://www.engadget.com/2012/06/20/project-moonshot-take-two-hps-low-power-gemini-servers-let-go/ http://www.engadget.com/2012/06/20/project-moonshot-take-two...
- brooksbp 14y ago
  Calxeda five XAUI (50 Gbps) to a 4 core 1.1GHz 5W. 200ns per hop. A ToR switch will slow you down to 2000-3500ns. Tilera TilePro64, 64 core 866MHz 22W, 1.7Tbps core-to-core, 46ns L1/L2 miss found in adjacent core, four XAUI (40 Gbps). When compared to Tilera, Calxeda has a high bandwith per compute ratio. But 200ns core-to-core is not as fast as 46ns. Tilera also has higher core/watt ratio.
zokier 14y ago
I think you misread that page. I think they mean that each SoC has one memory slot, which supports max 4GB RAM. So that 16 gigs is total amount for all the four CPUs. This still strongly points towards a 32-bit design.
- mykhal 14y ago
  16 GB per 4 CPUs does not mean 4GB per CPU? sorry for my english.. :)
  - hatcravat 14y ago
    16 GB shared among 4 CPUs is different from 4 CPUs which each access 4 GB of private memory.
    - unwind 14y ago
      The page says: Support for 16GB DDR3/3L memory via four (4) mini-DIMM sockets – one socket dedicated per SoC supports up to 4GB of ECC memory (supplied separately) Which really doesn't sound as if the memory is shared, to me.

mtgx 14y ago

They don't need to beat Intel "at the high-end" for "CPU intensive tasks". They just need to find their own market that requires low energy usage over high performance, and of course better performance/Watt, which I do think ARM beats Intel in. But Cortex A9 was never really meant to be a server CPU, so I would wait at least until Cortex A15, or even better for the 64 bit chips that are coming in 2014 (Cortex A15's successor, Nvidia's Project Denver, etc).

reitzensteinm 14y ago
I have to say a big [citation needed] to the claim of ARM beating Intel's high end chips on performance per watt, at least on general workloads. I think it's common to extrapolate Atom vs ARM to Xeon vs ARM in HPC, without thinking through the implications. We may well get higher performance/watt for single threads under ARM - I'm not disputing that, especially for integer work. However, Amahdl's law is going to raise its head. In the same machine, a higher number of lower performance threads is going to cause lock contention. You'll also have to split computations over more boxes, since the absolute performance of an Intel server will remain far higher (by 2014, we're talking 64 core/128 thread Haswell). Both of which are likely to be a massive tax on performance. To fight this, performance per core is likely to see a substantial rise, both in clock frequencies, and as a result of single core complexity. However, this will directly work against the two things that makes ARM performance/watt so impressive currently. Also, Intel's entire company is built around making those 100 watt scale processors fast and well. They really stumbled entering the Atom market; both because of a weak design (the chipset drew more power than the CPU itself!), as well as a lack of commitment (using 2-4 year old process nodes). I think we're likely to see a similar teething pains with companies trying to enter the server market for ARM. The instituational knowledge just won't be there. Make a cache architecture that effectively feeds 64 cores? Way different to improving power drain on a mobile CPU, for the seventh generation. I expect it will be at least a few generations before design teams are fully up to speed. Remember that AMD is reasonably well funded, also focused around server CPUs, and often stumbles. AMD is competent; Intel just makes them look incompetent by comparison. I'm not saying we won't see certain workloads that are better off under ARM; memcached and static http serving are both likely to do well, since they're effectively just shuffling bits around, aren't particularly CPU intensive, and are embarassingly parallel. But I believe they'll turn out to be the exception, not the rule. Which is to say, there's nothing magic about ARM that will let them beat x86 at the high end. They'll have to fight for it, and against Intel on their own turf no less.

mercuryrising 14y ago

What is the performance difference between 32 bit and 64 bit for memcached? I know redis has a lot of overhead with the 64 bit version [1], does memcached suffer the same fate? [1] http://serverfault.com/questions/221695/deploying-memcached-as-32bit-or-64bit http://serverfault.com/questions/221695/deploying-memcached-...