4 ms·

Does anyone know what type of hardware they are using to accomplish this? Is it just your standard Intel X520/540 10GbE adapter, or is there some secret hardwa

Does anyone know what type of hardware they are using to accomplish this? Is it just your standard Intel X520/540 10GbE adapter, or is there some secret hardware they have access to? FWIW, saturating line rate 10GbE has been problematic at times in the past due to challenges like single-threaded drivers, which could only run on 1 core at a time, even if there are 128 logical cores in a system (like the X1 instances). There are also challenges offloading VXLAN encapsulation to hardware at 10GbE line rate, but it seems like AWS has figured that out.

wmf 10y ago
It's from http://www.annapurnalabs.com/ http://www.annapurnalabs.com/ so it's probably a custom(ized) ARM SoC. See http://thread.gmane.org/gmane.linux.network/416446 http://thread.gmane.org/gmane.linux.network/416446 for more details.
netguy6 10y ago
Since it offers 20Gbps to the VM, i'd guess it's a 25GbE NIC, since those and accompanying switches have just come out. VXLAN in the NIC (and the switch) at full line-rate offload has been a thing for at least 18-24 months. Most Modern NICs also support all of the features listed here, like checksum offload, Receive steering, multi-queue, etc. Check out Mellanox CX4/5, Intel X710, QLogic, and Netronome if you want fancy stuff. They aren't even that expensive anymore. Their Linux driver has support for every current and future standard: 1/2.5/5/10/25/40/50/100/200/400G, nice future proofing. They also have a DPDK driver: http://dpdk.org/browse/dpdk/tree/drivers/net/ena/base/ena_com.c http://dpdk.org/browse/dpdk/tree/drivers/net/ena/base/ena_co...
- illumin8 10y ago
  Thanks, but my guess is that it's actually a bonded 2x 10Gbps link, which would also give you 20gbps, and cost a lot less per port.
  - wmf 10y ago
    I'm seeing 25G at the same price as 10G, but I'm not using Amazonomics.
  - netguy6 10y ago
    With current per port costs and economics, that doesn't make much sense. The $/Gbps/Watt improves with every spin of the chip, and every increase in serdes bandwidth. 10GbE (64x10G) - ~$4,500, $70/port 10GbE (32x40G) - ~$6,000, $50/port 25GbE (72x25G) - ~$7,000, $100/port (Seen as low as $5k) 25GbE (32x100G) - ~$9,000, $75/port (Seen as low as $7k) 2x 10GbE = $100 1x 25GbE = $75 Not to mention the extra cabling costs, and complexity.

wmf 10y ago

It's from http://www.annapurnalabs.com/ http://www.annapurnalabs.com/ so it's probably a custom(ized) ARM SoC. See http://thread.gmane.org/gmane.linux.network/416446 http://thread.gmane.org/gmane.linux.network/416446 for more details.

netguy6 10y ago

Since it offers 20Gbps to the VM, i'd guess it's a 25GbE NIC, since those and accompanying switches have just come out. VXLAN in the NIC (and the switch) at full line-rate offload has been a thing for at least 18-24 months. Most Modern NICs also support all of the features listed here, like checksum offload, Receive steering, multi-queue, etc. Check out Mellanox CX4/5, Intel X710, QLogic, and Netronome if you want fancy stuff. They aren't even that expensive anymore. Their Linux driver has support for every current and future standard: 1/2.5/5/10/25/40/50/100/200/400G, nice future proofing. They also have a DPDK driver: http://dpdk.org/browse/dpdk/tree/drivers/net/ena/base/ena_com.c http://dpdk.org/browse/dpdk/tree/drivers/net/ena/base/ena_co...

illumin8 10y ago
Thanks, but my guess is that it's actually a bonded 2x 10Gbps link, which would also give you 20gbps, and cost a lot less per port.
- wmf 10y ago
  I'm seeing 25G at the same price as 10G, but I'm not using Amazonomics.
- netguy6 10y ago
  With current per port costs and economics, that doesn't make much sense. The $/Gbps/Watt improves with every spin of the chip, and every increase in serdes bandwidth. 10GbE (64x10G) - ~$4,500, $70/port 10GbE (32x40G) - ~$6,000, $50/port 25GbE (72x25G) - ~$7,000, $100/port (Seen as low as $5k) 25GbE (32x100G) - ~$9,000, $75/port (Seen as low as $7k) 2x 10GbE = $100 1x 25GbE = $75 Not to mention the extra cabling costs, and complexity.