That's a good point, I have seen cases where we've separated a portion of nodes for VSAN or ScaleIO out to a separate cluster to avoid this behaviour on latency

That's a good point, I have seen cases where we've separated a portion of nodes for VSAN or ScaleIO out to a separate cluster to avoid this behaviour on latency sensitive workloads. These nodes don't need a ton of RAM and you'd better have good E-W bandwidth for them, but it's always a tradeoff...