> Actually, IPoIB was a customer requirement. In the Myrinet era, people frequently built clusters with a very poor ethernet, and used IPoverMyrinet to access s

> Actually, IPoIB was a customer requirement. In the Myrinet era, people frequently built clusters with a very poor ethernet, and used IPoverMyrinet to access storage etc. Interesting. I know we had a general-purpose cluster wired together with IPoIB just because it was the fastest option at the time. The production systems we were building would have IB for main communications using RDMA primitives in our middleware and a gigabit ethernet maintenance network for coordination and monitoring. > I don't agree with you about the way the ecosystem is headed. Tightly-coupled problems are tightly-coupled, period. It's true that there is a lot of interest in more loosely-connected clusters, but that doesn't get rid of the existing market. My point was more that MPI forces a greater degree of coupling to cluster architecture than some problems require, just because it was the way MPI was designed. Maybe I am overestimating the portion, but some number of problems solved in MPI prior to the development of more modern MapReduce frameworks was done simply because the problem needed the throughput of a cluster and MPI was the only way to use it without writing your own network middleware. That portion of the market is going away.