5 ms·

AMD is not just competitive, it is better than Intel. Thus Google should adopt it and role it out, and faster than any other cloud provider. This will win the

AMD is not just competitive, it is better than Intel. Thus Google should adopt it and role it out, and faster than any other cloud provider. This will win them customers. I want 256 threads per machine at competitive prices.

stcredzero 7y ago
"roll"
cjhanks 7y ago
Very few developers are prepared to write code that can efficiently use 256 threads / machine. At that level, cache coherency becomes a real and non-trivial problem. In most cases, I suspect developers will see improved wall-clock times with substantively worse FLOPS/watt. Good for developers, bad for data-centers.
- Nursie 7y ago
  Most devs I know are creating async workloads which don't require cache coherency, as they use parallelism to parallel process separate requests and workloads. I can see things being pretty linear in that sort of space.
  - gmueckl 7y ago
    Except that they are typically not. The Zen architectures are NUMA and controlling where memory is allocated is key to decent threaded performance. You may even have to do seemingly counterintuitive things like duplicating central data structures across nodes and other tricks from the distributed systems playbook.
    - coder543 7y ago
      Epyc 2's memory layout is not like Epyc 1. Epyc 2 is very simple.
      - jdsully 7y ago
        Yup everything is equally slow now. Kinda sad, but the original NUMA design was treated as a glass half empty situation instead of AMD letting people maximize performance. This change lets them avoid the bad press and everyone is happier despite the final design being slower than it could have been.
        gmueckl 7y ago
        Epyc 2 has different memory latencies within and across NUMA nodes according to the infirmation I have. So it is not equally slow for all memory. Can you point me to a source that says otherwise? Edit: my source is this German article: https://www.heise.de/newsticker/meldung/AMD-Server-CPUs-Epyc-7002-Mit-Zen-2-an-Intel-vorbei-4489732.html https://www.heise.de/newsticker/meldung/AMD-Server-CPUs-Epyc...
        jdsully 7y ago
        See the architecture diagram here: https://www.anandtech.com/show/14694/amd-rome-epyc-2nd-gen/2 https://www.anandtech.com/show/14694/amd-rome-epyc-2nd-gen/2 Everything goes through the central crossbar on the I/O die, where Zen1 had memory attached directly to each CPU chiplet which would relay as necessary. On Zen1 if you accessed direct attached memory you wouldn't pay the latency penalty from relaying the data. In Zen2 all data is relayed via the I/O die with the associated delay that entails.
        gmueckl 7y ago
        I did some more digging. It seems like the Linux NUMA topology shown in the anandtech article is a deliberate lie. There are different latencies between cores and memory comtrollers on the same socket, but these are deemed to be insignificant enough to not expose them in the reported NUMA topology.
        jdsully 7y ago
        That is true with Intel chips as well. In the HFT space people actively work with Intel to determine which cores they should pin tasks to. The speed of light is constant, and some cores will always be a little closer to various resources.
  - cjhanks 7y ago
    They are not linear unless all requests take an identical amount of time OR the system is not oversubscribed (common in many workloads) - and even then, the current linux CFS scheduler has a complexity of `O(log N)`. When you have variable length requests, you will find cores will not always be balanced, it is simply a statistical reality. And in those cases, the kernel will have to migrate your process to a different core, and if you have 256 cores, that core might be really far away.
- mrb 7y ago
  «Very few developers are prepared to write code that can efficiently use 256 threads / machine» This junk justification has no longer been relevant for years. Most developers don't care because (1) they rely on core applications that are already multi-threaded (web servers, SQL engines, transcoding, etc), or (2) in today's age of containers, VMs, etc, it doesn't matter to them. Now we scale by adding more containers and VMs per physical machine. Bottom line, data centers always need more cores/threads per machine.
  - cjhanks 7y ago
    Correct, if you partition a 256 core machine into 32 virtual 8-core machines partitioned by their NUMA architecture - you are relatively unaffected by core count (minus the consequence of some scheduling algorithms not tuned for N > 8). Unsure what the percentage of VM's that use no time sharing or oversubscription is though.
- fulafel 7y ago
  I suspect cache coherency doesn't mean what you think it means. It's a hardware feature. But yes, writing correct and performant highly parallel code is difficult & error prone, often prohibitively so.
sseveran 7y ago
AWS has already rolled out EPYC instances. https://aws.amazon.com/ec2/amd/ https://aws.amazon.com/ec2/amd/
- danarmak 7y ago
  Those are the previous generation, Zen 1 EPYC CPUs, whichn were rolled out on AWS back in November.
bluedino 7y ago
Don't forget power consumption. Electricity costs are probably just as big as a factor as Cperformance when it comes to the number of computer Google has. IBM quoted them as willing to switch to POWER if they could save 10% in energy costs
jrockway 7y ago
AWS has AMD machines. Amazon claims they perform worse than the Intel machines they use, and they are priced lower as a result.
- corint 7y ago
  I thought they were cheaper due to the better energy efficiency. Less electricity means less cooling required, double whammy!
- philliphaydon 7y ago
  T3 are intel, newer than the T2, and perform worse than the T2. T3a are AMD and perform on par/slightly better than T3 for less cost. (From my own testing. Not a claim I can backup this is just my observation)
- bradstewart 7y ago
  Where did you hear that? We're running a handful of m5a instances with fantastic performance. I figured they were priced lower because they're cheaper to purchase and operate.
jimmaswell 7y ago
Depends on the use case. Intel is still top for many games and some apps like Excel/Photoshop.

stcredzero 7y ago

"roll"

cjhanks 7y ago

Very few developers are prepared to write code that can efficiently use 256 threads / machine. At that level, cache coherency becomes a real and non-trivial problem. In most cases, I suspect developers will see improved wall-clock times with substantively worse FLOPS/watt. Good for developers, bad for data-centers.

Nursie 7y ago
Most devs I know are creating async workloads which don't require cache coherency, as they use parallelism to parallel process separate requests and workloads. I can see things being pretty linear in that sort of space.
- gmueckl 7y ago
  Except that they are typically not. The Zen architectures are NUMA and controlling where memory is allocated is key to decent threaded performance. You may even have to do seemingly counterintuitive things like duplicating central data structures across nodes and other tricks from the distributed systems playbook.
  - coder543 7y ago
    Epyc 2's memory layout is not like Epyc 1. Epyc 2 is very simple.
    - jdsully 7y ago
      Yup everything is equally slow now. Kinda sad, but the original NUMA design was treated as a glass half empty situation instead of AMD letting people maximize performance. This change lets them avoid the bad press and everyone is happier despite the final design being slower than it could have been.
      - gmueckl 7y ago
        Epyc 2 has different memory latencies within and across NUMA nodes according to the infirmation I have. So it is not equally slow for all memory. Can you point me to a source that says otherwise? Edit: my source is this German article: https://www.heise.de/newsticker/meldung/AMD-Server-CPUs-Epyc-7002-Mit-Zen-2-an-Intel-vorbei-4489732.html https://www.heise.de/newsticker/meldung/AMD-Server-CPUs-Epyc...
        jdsully 7y ago
        See the architecture diagram here: https://www.anandtech.com/show/14694/amd-rome-epyc-2nd-gen/2 https://www.anandtech.com/show/14694/amd-rome-epyc-2nd-gen/2 Everything goes through the central crossbar on the I/O die, where Zen1 had memory attached directly to each CPU chiplet which would relay as necessary. On Zen1 if you accessed direct attached memory you wouldn't pay the latency penalty from relaying the data. In Zen2 all data is relayed via the I/O die with the associated delay that entails.
        gmueckl 7y ago
        I did some more digging. It seems like the Linux NUMA topology shown in the anandtech article is a deliberate lie. There are different latencies between cores and memory comtrollers on the same socket, but these are deemed to be insignificant enough to not expose them in the reported NUMA topology.
        jdsully 7y ago
        That is true with Intel chips as well. In the HFT space people actively work with Intel to determine which cores they should pin tasks to. The speed of light is constant, and some cores will always be a little closer to various resources.
- cjhanks 7y ago
  They are not linear unless all requests take an identical amount of time OR the system is not oversubscribed (common in many workloads) - and even then, the current linux CFS scheduler has a complexity of `O(log N)`. When you have variable length requests, you will find cores will not always be balanced, it is simply a statistical reality. And in those cases, the kernel will have to migrate your process to a different core, and if you have 256 cores, that core might be really far away.
mrb 7y ago
«Very few developers are prepared to write code that can efficiently use 256 threads / machine» This junk justification has no longer been relevant for years. Most developers don't care because (1) they rely on core applications that are already multi-threaded (web servers, SQL engines, transcoding, etc), or (2) in today's age of containers, VMs, etc, it doesn't matter to them. Now we scale by adding more containers and VMs per physical machine. Bottom line, data centers always need more cores/threads per machine.
- cjhanks 7y ago
  Correct, if you partition a 256 core machine into 32 virtual 8-core machines partitioned by their NUMA architecture - you are relatively unaffected by core count (minus the consequence of some scheduling algorithms not tuned for N > 8). Unsure what the percentage of VM's that use no time sharing or oversubscription is though.
fulafel 7y ago
I suspect cache coherency doesn't mean what you think it means. It's a hardware feature. But yes, writing correct and performant highly parallel code is difficult & error prone, often prohibitively so.

sseveran 7y ago

AWS has already rolled out EPYC instances. https://aws.amazon.com/ec2/amd/ https://aws.amazon.com/ec2/amd/

danarmak 7y ago
Those are the previous generation, Zen 1 EPYC CPUs, whichn were rolled out on AWS back in November.

bluedino 7y ago

Don't forget power consumption. Electricity costs are probably just as big as a factor as Cperformance when it comes to the number of computer Google has. IBM quoted them as willing to switch to POWER if they could save 10% in energy costs

jrockway 7y ago

AWS has AMD machines. Amazon claims they perform worse than the Intel machines they use, and they are priced lower as a result.

corint 7y ago
I thought they were cheaper due to the better energy efficiency. Less electricity means less cooling required, double whammy!
philliphaydon 7y ago
T3 are intel, newer than the T2, and perform worse than the T2. T3a are AMD and perform on par/slightly better than T3 for less cost. (From my own testing. Not a claim I can backup this is just my observation)
bradstewart 7y ago
Where did you hear that? We're running a handful of m5a instances with fantastic performance. I figured they were priced lower because they're cheaper to purchase and operate.

jimmaswell 7y ago

Depends on the use case. Intel is still top for many games and some apps like Excel/Photoshop.