3 ms·
Indeed. The AWS p4 instances are wired together by the next generation NVLink/NLink internconnect (600 GB/s bidirectional bandwidth) and also the Elastic Fabric
by carlsborg 4y ago
Indeed. The AWS p4 instances are wired together by the next generation NVLink/NLink internconnect (600 GB/s bidirectional bandwidth) and also the Elastic Fabric Adapters (400 GB/s) which is likely Mellanox under the covers.
- Moissanite 4y agoEFA is not InfiniBand. It has similar goals (low latency via kernel bypass), but is implemented using AWS Nitro NICs and regular Ethernet networking. Other providers (Azure and Oracle) do offer InfiniBand, but that creates a problem with "islands" of compute which can't be easily expanded. There are also more security considerations when using IB in a multi-tenant environment which makes it not such a slam-dunk advantage as you might first assume.
- carlsborg 3y agoYou're right, EFA is custom nitro NICs. Although Mellanox (now Nvidia ConnectX) is not solely Inifiband either - Nvidia offers them with Ethernet upto 400GbE as well, same speed as EFA says on the label.