I'm guessing part of the problem infiniband switches get expensive AF when you're going for 32x100 or rack-level anything. And you're still bound to up to 200Gb

I'm guessing part of the problem infiniband switches get expensive AF when you're going for 32x100 or rack-level anything. And you're still bound to up to 200Gbit per link (and the cpu load that entails) or lower bandwidth bonded links, why is not as interesting as QPI. Aside from purely guessing I'm interested in scalability pains war stories with infiniband too :-) as I'm looking into a 400GbE-everywhere datacenter build...