3 ms·

it's not the hash table lookup that is expensive, it's the network request. Computers are not good at network requests because physics says they can't be - it

it's not the hash table lookup that is expensive, it's the network request. Computers are not good at network requests because physics says they can't be - it takes orders of magnitude more time for light/electricity to traverse even one side of a datacenter to another than it does to go from the CPU side of the motherboard to the RAM side. For plenty of use cases, the additional network roundtrip is very affordable. But it's simple math that explains why this might easily become half or more than half of the total load on your systems if you want to "just do an additional (network-mediated) hash table lookup" on every request.

sneak 5y ago
The 100-2000ns it takes signals to move across a datacenter and back is not the primary cause of a network request being slow in comparison with a cpu.
- spookthesunset 5y ago
  LOL to 2000ns. Seriously? 10ms minimum to go to the next rack. For 99% of the requests. For the unlucky 1% you'll get outlier performance due to.... well... you spend the next sprint figuring it out and the next quarter trying to solve that scaling bottleneck.... It could be network, it could be some weird design thing, it could be redis, who knows? But yeah... enjoy figuring it out instead of focusing on your product. 1% of all requests is actually a lot of requests. And if you think 1% of your auth requests aren't 10x the median... you aren't measuring it hard enough.
  - brobinson 5y ago
    What on earth kind of clownish network setup do you have where O(1) lookups over a network take 10ms? I worked on a tiny (3 people) engineering team serving 250,000 http requests/second 27/7/365 doing 5-6 over-the-network requests to other services (memcached, redis) per request we handled and our 99th percentile time to get a response on the wire was < 15 milliseconds.
    - brobinson 5y ago
      "24"/7/365... we weren't warping spacetime to get responses on the wire that fast.
  - deleted 5y ago
    [deleted]
  - sneak 5y ago
    That's exactly my point, you are agreeing with me. I was refuting the comment claiming that networks were slow compared to on-die comms because of physics and lightspeed. The signals going through the wire and back are 2000ns in a big datacenter. There are orders of magnitude more delays in the processing of those bits than their transmission through space via twisted pair or fiber. It's not 10ms to go to the next rack though unless your network is broken. You're much closer to 100us (100,000ns).
  - imtringued 5y ago
    Yes, it takes 2000ns for the signal to travel at the speed of light. ledauphin never talked about the time spent processing the signals, which is incredibly odd because that's where most of the time is being wasted. This is a very common pattern in networking, people underestimate the speed of light and overestimate the speed of network equipment.

sneak 5y ago

The 100-2000ns it takes signals to move across a datacenter and back is not the primary cause of a network request being slow in comparison with a cpu.

spookthesunset 5y ago
LOL to 2000ns. Seriously? 10ms minimum to go to the next rack. For 99% of the requests. For the unlucky 1% you'll get outlier performance due to.... well... you spend the next sprint figuring it out and the next quarter trying to solve that scaling bottleneck.... It could be network, it could be some weird design thing, it could be redis, who knows? But yeah... enjoy figuring it out instead of focusing on your product. 1% of all requests is actually a lot of requests. And if you think 1% of your auth requests aren't 10x the median... you aren't measuring it hard enough.
- brobinson 5y ago
  What on earth kind of clownish network setup do you have where O(1) lookups over a network take 10ms? I worked on a tiny (3 people) engineering team serving 250,000 http requests/second 27/7/365 doing 5-6 over-the-network requests to other services (memcached, redis) per request we handled and our 99th percentile time to get a response on the wire was < 15 milliseconds.
  - brobinson 5y ago
    "24"/7/365... we weren't warping spacetime to get responses on the wire that fast.
- deleted 5y ago
  [deleted]
- sneak 5y ago
  That's exactly my point, you are agreeing with me. I was refuting the comment claiming that networks were slow compared to on-die comms because of physics and lightspeed. The signals going through the wire and back are 2000ns in a big datacenter. There are orders of magnitude more delays in the processing of those bits than their transmission through space via twisted pair or fiber. It's not 10ms to go to the next rack though unless your network is broken. You're much closer to 100us (100,000ns).
- imtringued 5y ago
  Yes, it takes 2000ns for the signal to travel at the speed of light. ledauphin never talked about the time spent processing the signals, which is incredibly odd because that's where most of the time is being wasted. This is a very common pattern in networking, people underestimate the speed of light and overestimate the speed of network equipment.