It's not for general use but if you build large infrastructure stuff IB can be a very interesting transport choice. Naturally you will only do this if you want

It's not for general use but if you build large infrastructure stuff IB can be a very interesting transport choice. Naturally you will only do this if you want to either use RDMA directly (via IB Verbs) or use protocols like IB SCSI RDMA Protocol which do so under the hood. Things like "Datacentre Ethernet" and RDMA over Converged Ethernet (RoCE) are basically Infiniband anyway (you will find that such features are only available on Connect-X HBAs from Mellanox which can generally run in 100GE or IB mode). The main reason these things matter is they are circuit switched and generally combined with network topologies that ensure full bisectional bandwidth (or something that is essentially good enough for the communication pattern the machine is designed for). This means the network becomes "lossless" once it's properly verified as online and circuits are setup. For storage this is essentially unmatched, it's equivalent to Fiber Channel but way faster and converged so you can also run your workload over the same links rather than having separate storage and network links.