4 ms·

> While latency over a local network connection is low, it's nowhere near as low as over a PCI or SATA connection that spans less than a meter. This means that

> While latency over a local network connection is low, it's nowhere near as low as over a PCI or SATA connection that spans less than a meter. This means that the average latency of disk operations (from the perspective of the operating system) can be on the order of a couple milliseconds, compared to half a millisecond for directly-attached disks. uh, you can get a packet from your software on one machine, through a switch and into your software on the second machine in 1-2us if you know what you're doing (of course, that's without any cloud bullshit like passing the packet through half a dozen levels of virtualisation, then through software defined networks)

Hello71 4y ago
while i am not a "cloud expert", i'm pretty sure that google datacenters have more than one switch. furthermore, while the precise datacenter layout is probably secret, i infer that the the compute servers and the data servers cannot be housed in the same or nearby racks, since both are independently provisionable and can be arbitrarily attached. for example, if i start a vm and connect a drive, even if google could put them right next to each other in the datacenter, what happens if i start another vm, which for fragmentation reasons gets placed way on the other side of the building, detach the drive and connect it to this new one? therefore, there must at least be several switches between the devices, and realistically, probably one or more routers.
- blibble 4y ago
  it was an example with modern switches it barely matters how many they are in between the machines
PragmaticPulp 4y ago
> uh, you can get a packet from your software on one machine, through a switch and into your software on the second machine in 1-2us if you know what you're doing This isn’t a pair of servers sitting in a room with a switch in between them on a simple /24 subnet. It’s a gigantic cloud data center with a massive network spanning a huge number of devices. The simple things you can get to work in a small setup don’t apply at this scale.
- blibble 4y ago
  the example I gave was a simplified (though true) situation of a system local bus it's common to have a dedicated trading network completely parallel and isolated from the main DC network for each upstream connection, of which there are dozens and these are (at least) double redundant through the entire network, typically using redundant packet arbitration to near guarantee zero drops over a session Google's network is super-dynamic and flexible, but slow and likely over-contended ours is the exact opposite
Cyph0n 4y ago
> that's without any cloud bullshit like passing the packet through half a dozen levels of virtualisation, then through software defined networks I mean, by definition, a public cloud wouldn’t work without layers of virtualization and SDN.
jmillikin 4y ago
The other replies are assuming networking in a big network is inherently slower than in a small network. I used to work at Google in Tech Infra, so I'll offer an alternate perspective while trying not to spill secrets. First, Google has enough money that they can build their entire network out of custom hardware, custom firmware, and patch the kernel + userspace. A datacenter at Google scale is architecturally similar to a supercomputer cluster running on InfiniBand. You will never be able to replicate the performance of Google's network by buying some rackmounted servers from Dell and plumbing them together with Cisco switches. Second, assuming a reasonably competent design, adding more machines to a network doesn't significantly increase the latency of that network. You'll see better latencies between machines in the same rack than between two racks, but this is a matter of single microseconds rather than milliseconds. Additional latency from intermediate switches is measured in nanoseconds. Third, Google publishes an SLA on round-trip network latency between customer VMs at <https://cloud.google.com/vpc/docs/vpc https://cloud.google.com/vpc/docs/vpc>. Their "tail latencies less than 80μs at the 99th percentile" translates to ~40μs for one way, and honestly for customer VMs a lot of that happens in the customer kernel + virtualization layer. A process running on bare-metal, such as a kernel reading a remote network block device, can (IIRC) expect single-microsecond latencies to get one packet onto a nearby machine.
- blibble 4y ago
  > The other replies are assuming networking in a big network is inherently slower than in a small network. well yes, not true with modern switches that support cut-through forwarding it's super-common in our space to bypass the kernel entirely, writing into the NIC buffers directly with prepared packet headers, and the card has pushed part of the packet out onto the wire, through switches and into the target machine's NIC buffers before it's even finished being written typical "SLA"s are 0 packets dropped during a session, where a single drop raises an alert that is then investigated > You will never be able to replicate the performance of Google's network by buying some rackmounted servers from Dell and plumbing them together with Cisco switches. and yet, somehow we do quite a bit better (admittedly they are very, very expensive switches) I get that people that work at Google like to think they're working on problems more advanced than those of mere mortals, but with the latencies you've described we'd be out of business several times over (not to mention none of the clouds support multicast)

Hello71 4y ago

while i am not a "cloud expert", i'm pretty sure that google datacenters have more than one switch. furthermore, while the precise datacenter layout is probably secret, i infer that the the compute servers and the data servers cannot be housed in the same or nearby racks, since both are independently provisionable and can be arbitrarily attached. for example, if i start a vm and connect a drive, even if google could put them right next to each other in the datacenter, what happens if i start another vm, which for fragmentation reasons gets placed way on the other side of the building, detach the drive and connect it to this new one? therefore, there must at least be several switches between the devices, and realistically, probably one or more routers.

blibble 4y ago
it was an example with modern switches it barely matters how many they are in between the machines

PragmaticPulp 4y ago

> uh, you can get a packet from your software on one machine, through a switch and into your software on the second machine in 1-2us if you know what you're doing This isn’t a pair of servers sitting in a room with a switch in between them on a simple /24 subnet. It’s a gigantic cloud data center with a massive network spanning a huge number of devices. The simple things you can get to work in a small setup don’t apply at this scale.

blibble 4y ago
the example I gave was a simplified (though true) situation of a system local bus it's common to have a dedicated trading network completely parallel and isolated from the main DC network for each upstream connection, of which there are dozens and these are (at least) double redundant through the entire network, typically using redundant packet arbitration to near guarantee zero drops over a session Google's network is super-dynamic and flexible, but slow and likely over-contended ours is the exact opposite

Cyph0n 4y ago

> that's without any cloud bullshit like passing the packet through half a dozen levels of virtualisation, then through software defined networks I mean, by definition, a public cloud wouldn’t work without layers of virtualization and SDN.

jmillikin 4y ago

The other replies are assuming networking in a big network is inherently slower than in a small network. I used to work at Google in Tech Infra, so I'll offer an alternate perspective while trying not to spill secrets. First, Google has enough money that they can build their entire network out of custom hardware, custom firmware, and patch the kernel + userspace. A datacenter at Google scale is architecturally similar to a supercomputer cluster running on InfiniBand. You will never be able to replicate the performance of Google's network by buying some rackmounted servers from Dell and plumbing them together with Cisco switches. Second, assuming a reasonably competent design, adding more machines to a network doesn't significantly increase the latency of that network. You'll see better latencies between machines in the same rack than between two racks, but this is a matter of single microseconds rather than milliseconds. Additional latency from intermediate switches is measured in nanoseconds. Third, Google publishes an SLA on round-trip network latency between customer VMs at <https://cloud.google.com/vpc/docs/vpc https://cloud.google.com/vpc/docs/vpc>. Their "tail latencies less than 80μs at the 99th percentile" translates to ~40μs for one way, and honestly for customer VMs a lot of that happens in the customer kernel + virtualization layer. A process running on bare-metal, such as a kernel reading a remote network block device, can (IIRC) expect single-microsecond latencies to get one packet onto a nearby machine.

blibble 4y ago
> The other replies are assuming networking in a big network is inherently slower than in a small network. well yes, not true with modern switches that support cut-through forwarding it's super-common in our space to bypass the kernel entirely, writing into the NIC buffers directly with prepared packet headers, and the card has pushed part of the packet out onto the wire, through switches and into the target machine's NIC buffers before it's even finished being written typical "SLA"s are 0 packets dropped during a session, where a single drop raises an alert that is then investigated > You will never be able to replicate the performance of Google's network by buying some rackmounted servers from Dell and plumbing them together with Cisco switches. and yet, somehow we do quite a bit better (admittedly they are very, very expensive switches) I get that people that work at Google like to think they're working on problems more advanced than those of mere mortals, but with the latencies you've described we'd be out of business several times over (not to mention none of the clouds support multicast)