4 ms·

> What's stopping more people from moving cloud workloads to AMD? It's not just about CPU bound workloads but also things that are heavily I/O dependent (typic

> What's stopping more people from moving cloud workloads to AMD? It's not just about CPU bound workloads but also things that are heavily I/O dependent (typically on bare metal hardware that you own, rather than instances you rent somewhere). There's lots of networking and storage things that would be performance bottlenecked on an intel cpu with less PCI-E lanes. Having a 16-core CPU around $950 that has 128 PCI-E 4.0 lanes is very useful for many things. And not just for EPYC but also the single socket threadripper parts, which are used in both higher end workstations and some types of server.

ixfo 6y ago
100% this. With NVMe and higher bandwidth NICs, fast and plentiful PCIe lanes are increasingly important and Intel have just been so far behind in this (or gated equivalance at hilarious price differentials from AMD - like having to go to $4k Intel parts (and beyond) to match AMD prosumer desktop parts).
- bayindirh 6y ago
  Just try to attach a 8 GPU A100 unit to a 2P Intel system, then try to add fast Infiniband with NVMe for scratch, and whole platform just chokes.
0x000000E2 6y ago
^^ this is huge. I was looking at CPU's for a ML recently and Intel is out of the question. Chips with enough lanes to hit full speed on 4 GPUs + SSD cost twice as much as AMD. This may even apply to high end gaming machines. Soon as you have 2 SSD's or video cards you will exceed link budget on most Intel CPU's and everything slows down. It also happens to routers. 10 gig NICs plus attached SSD storage, over link budget again. Intel's stupid market segmentation is biting them in the rear
- walrus01 6y ago
  Not just 10Gbps (such as the Intel card which is four 10Gbps SFP ports in one slot), but single and dual 100GbE per slot like this: https://www.intel.com/content/www/us/en/products/docs/network-io/ethernet/network-adapters/ethernet-800-series-network-adapters/e810-cqda1-cqda2-100gbe-brief.html https://www.intel.com/content/www/us/en/products/docs/networ... In calculating the bandwidth and pci-e bus throughput needed, a single 100GbE port is full duplex, so one has to budget about 210Gbps per port. The funny thing is that some of the best 100GbE NICs for x86-64 servers on the market right now are Intel, but are best used on an AMD platform...
  - drewg123 6y ago
    What? Why do you think the Intel 100GbE NIC is good? We've been quite happy with Mellanox and Chelsio 100GbE NICs. The latest from each can do in-line HW TLS offload, which is a killer feature for us. No Intel NIC can do that. IMHO the last good Intel NIC was the 10GbE "ixgbe" NIC. The design of the NIC was so tight as to be almost beautiful. Recent 40GbE (and 10GbE based on the 40GbE chipset), and the new 100GbE NIC have the feel of being designed by a committee with endless features of questionable value stuffed in and consuming power and chip area.
    - walrus01 6y ago
      Primarily the state of its Linux driver, and rock solid support in VyOS (derived from Debian stable). For a router, TLS offload isn't so much of a consideration when the system isn't doing anything at layers 4-7 in the OSI model. If I had to make a perhaps overly broad generalization, I see more Chelsio and Mellanox NICs used in end point servers, and more Intel used in DIY whitebox network equipment.
      - drewg123 6y ago
        That's fair; I come at this from a CDN perspective where end system performance is most important. Do you see any benefit from the fancy features? Can it source/sink min sized frames at 100GbE? (144Mpps) ?
  - 6y ago

ixfo 6y ago

100% this. With NVMe and higher bandwidth NICs, fast and plentiful PCIe lanes are increasingly important and Intel have just been so far behind in this (or gated equivalance at hilarious price differentials from AMD - like having to go to $4k Intel parts (and beyond) to match AMD prosumer desktop parts).

bayindirh 6y ago
Just try to attach a 8 GPU A100 unit to a 2P Intel system, then try to add fast Infiniband with NVMe for scratch, and whole platform just chokes.

0x000000E2 6y ago

^^ this is huge. I was looking at CPU's for a ML recently and Intel is out of the question. Chips with enough lanes to hit full speed on 4 GPUs + SSD cost twice as much as AMD. This may even apply to high end gaming machines. Soon as you have 2 SSD's or video cards you will exceed link budget on most Intel CPU's and everything slows down. It also happens to routers. 10 gig NICs plus attached SSD storage, over link budget again. Intel's stupid market segmentation is biting them in the rear

walrus01 6y ago
Not just 10Gbps (such as the Intel card which is four 10Gbps SFP ports in one slot), but single and dual 100GbE per slot like this: https://www.intel.com/content/www/us/en/products/docs/network-io/ethernet/network-adapters/ethernet-800-series-network-adapters/e810-cqda1-cqda2-100gbe-brief.html https://www.intel.com/content/www/us/en/products/docs/networ... In calculating the bandwidth and pci-e bus throughput needed, a single 100GbE port is full duplex, so one has to budget about 210Gbps per port. The funny thing is that some of the best 100GbE NICs for x86-64 servers on the market right now are Intel, but are best used on an AMD platform...
- drewg123 6y ago
  What? Why do you think the Intel 100GbE NIC is good? We've been quite happy with Mellanox and Chelsio 100GbE NICs. The latest from each can do in-line HW TLS offload, which is a killer feature for us. No Intel NIC can do that. IMHO the last good Intel NIC was the 10GbE "ixgbe" NIC. The design of the NIC was so tight as to be almost beautiful. Recent 40GbE (and 10GbE based on the 40GbE chipset), and the new 100GbE NIC have the feel of being designed by a committee with endless features of questionable value stuffed in and consuming power and chip area.
  - walrus01 6y ago
    Primarily the state of its Linux driver, and rock solid support in VyOS (derived from Debian stable). For a router, TLS offload isn't so much of a consideration when the system isn't doing anything at layers 4-7 in the OSI model. If I had to make a perhaps overly broad generalization, I see more Chelsio and Mellanox NICs used in end point servers, and more Intel used in DIY whitebox network equipment.
    - drewg123 6y ago
      That's fair; I come at this from a CDN perspective where end system performance is most important. Do you see any benefit from the fancy features? Can it source/sink min sized frames at 100GbE? (144Mpps) ?
- 6y ago