2 ms·
http://hyperdex.org/extras/ http://hyperdex.org/extras/ makes assumptions that just aren't true these days. Who knows what the latency between 2 EC2 nodes are
by aidenn0 14y ago
http://hyperdex.org/extras/ http://hyperdex.org/extras/ makes assumptions that just aren't true these days. Who knows what the latency between 2 EC2 nodes are going to be, even when in the same AZ? Is it not possible to run a cluster that isn't on a single LAN?
- rescrv 14y agoEC2 is just one platform. Although many people point to EC2 as the gold standard, it's worth noting that Google and Facebook both avoid virtualized servers. On the other hand, EC2 is about leveraging spare capacity and intentionally oversubscribing machines.
- aidenn0 14y agoYes, but it's a popular platform that doesn't (at first blush) appear to meat the assumptions on the page I posted. My question still stands at "Do I need to have all my machines in the same datacenter"
- rescrv 14y agoYou don't need all machines in the same data center. Latency between machines does have effects on performance, so it is faster in a single data center.
- jws 14y agoGood questions. I have no answers but only this guideline… Reality is generally more constrained than the models people make for theorems. That can help or hurt. Back in the days when ethernet meant unswitched 10mbps and it was proven that you couldn't get more than about 1mbps utilization and should probably go with token ring or one of the better thought out networks… my name was on a paper showing 9mbps utilization in our real application without doing anything special. Except we were using real world hardware that was not even close to capable of sending back to back packets, so the our data streams were naturally interleaving and collisions went to near zero.
- aidenn0 14y agoThat's a case of reality being less constrained than the theory, assuming the theory actually proved a maximum utilization of ~1mbps; perhaps they were assuming all nodes were saturating the line?
- jws 14y agoIts been a while. The best I recall, the model the theory used was for a random distribution of legal interpacket timings that didn't account for the rather significant minimum interpacket gap imposed by the hardware. It was also significant here because people forgot all the preconditions of the original paper and just applied the sound bite synopsis.