5 ms·

So many people in the industry don't understand when I say that a 100ms outage is a problem, it's depressing.

danpalmer 6y ago
This depends what "industry" you're talking about. If you're talking about "tech", I'd say 100ms doesn't matter for the vast majority of use-cases. Bear in mind that Australia/New Zealand have a ~250ms round trip latency to most web services, 4G can easily have a 300ms latency to Google, and developing countries are often in a much worse position. Depending on <100ms for anything is a non-starter for most of the web. If you're a service provider several layers removed from a human, then there may be enough between you and the user that it's critical that you respond in <100ms, but this is not most of the industry. More generally, I find this sort of comment to be similar to the sort of comment that asks why anyone would write a web service in a language as slow as Ruby or Python when we have languages like Go and Rust. They ignore the fact that there are huge, successful companies running on Ruby, Python, and other "slow" languages, all for very good reasons.
- smolder 6y ago
  100ms becomes significant if you're doing teleconferencing or other real time interaction. If we somehow improved latency and jitter on the internet generally, I think that'd do more to improve its usefulness than expanding bandwidth. And don't forget about the studies out there that show a relationship between load time and revenue in e-commerce: "A 100ms decrease in checkout page load speed amounted to a 1.55% increase in session-based conversion."
  - lostcolony 6y ago
    If doing realtime interaction it's unlikely they'd negotiate more than one connection (reusing connections or not), so the test they've done hardly seems useful. I also have huge issues with a lot of those studies; 100ms from 200ms -> 100ms is immensely different than 1500ms -> 1400ms.
  - joosters 6y ago
    This is 100ms for establishing a TLS connection - that's not the same as 100ms latency for all of a video stream.
    - smolder 6y ago
      Oh yeah, you're absolutely right. I got a bit tangential. Slow establishment wouldn't matter for a stream. I'm sure it's something more like the e-commerce page load time vs conversions scenario for OP.
      - axismundi 6y ago
        I could live with slow handshake, but it's ~110ms for every request.
        Spooky23 6y ago
        I had a similar problem working with an managed file transfer service. Lots of small files would get bogged down by these sorts of issues. The other side of the connection wouldn’t change their config, so I implemented a proxy that maintained session state on our side.
  - danpalmer 6y ago
    Teleconferencing is a good example of where this does matter, although I'd suggest that despite the current situation we're all in, it's not a significant part of the industry. > And don't forget about the studies out there that show a relationship between load time and revenue in e-commerce: "A 100ms decrease in checkout page load speed amounted to a 1.55% increase in session-based conversion." This is true, although I believe this relationship doesn't hold when you're moving from 200ms to 100ms, it's more for when you're moving from 1000ms to 900ms.
  - user5994461 6y ago
    Latency is capped by two third the speed of light. The network latency is actually pretty good overall considering this (the issue is really on the user end with terrible Wifi and mobile connections). The speed of light is quite impactful when you're looking at different countries or continents.
    - iso1210 6y ago
      > Latency is capped by two third the speed of light Starlink: hold my beer. I'm not a HFT, I know I can't increase the speed of light, I know I can't tunnel through the earth to shave a few microseconds off a transmission. I can appreciate the difference between 300ms rtt and 900ms rtt - the later being a practical minimum when you're coping with outages in the 100-200ms range and networks reroute. Give me jitter over drops any day.
  - axismundi 6y ago
    Yes, this is a real time platform. I spend some of my dev time optimising performance.
- bufferoverflow 6y ago
  Except it does matter. Every 100ms of latency caused Amazon 1% in sales. Every 500ms of latency cost Google 20% of their traffic. And these are 10 year old metrics, back when people tolerated slowness much more.
- iso1210 6y ago
  Computer industry in general, but network providers specifically. It's not about <100ms latency, it's about a <100ms outage I know I can't rely on providers to not have outages on 100ms -- a link fails in Sudan and BGP flaps on a peering it cane take seconds, let alone milliseconds, to respond. I have to be very careful, because if I want a two-way communication at that 250ms round trip time to NZ, that means I have to ensure my packets get there and back in 250ms, and not drop in the middle and require retransmissions, which rapidly escalates latency from 'barely noticable' up to 'painful' levels (1 second or above is meaningless, may as well send a fax[1]) 0ms : send packet containing informaiton 125ms : receive packet 135ms: process informaiton, send response 260ms: receive response 270ms: process response Job done. Trouble is, a 100ms outage from 0ms to 100ms means 0ms : send packet containing informaiton 125ms : fail to receive packet 135ms : notice packet missing, issue request for retransmit 260ms: receive retransmit request 260ms: retransmit 385ms : receive packet 395ms: process informaiton, send response 620ms: receive response 630ms: process response Given I therefore need (630-270 = 360ms) of buffer, that's more than doubled the response. But it gets worse 125ms : fail to receive packet 135ms : notice packet missing, issue request for retransmit (request gets lost) 395ms: notice retransmit request not received, issue request for retransmit 405ms: issue retransmit request 530ms : receive retransmit request, retransmit 655ms: receive packet 665ms: process information, send response 790ms: process response So to cope with a 100ms outage my application needs to insert a 520ms buffer on a 270ms conversation - practically tripling latency, and converting a near- real time convertsation into near-catastrophic talking over each other. Now this is fine, I can have two circuits, send the packets down both, and the chance of both being broken is minimal -- assuming the links are really separate (which is another challange. Hint, both patchs down the same undersea cable on different frequencies is not separation) However getting network providers to acknowlege, let alone measure, outages of 100ms, is practically impossible on a country basis, let alone an intercontinental basis. [1] hyperbole. Just.
- deleted 6y ago
  [deleted]
harlanji 6y ago
It’s mind boggling how much can happen in 100ms. That’s 6,000 protobuf kafka messages consumed in clojure, deserialized, serialized, sent and acknowledged in 2015 aws. Upgrading a task caused fraction of million message backlogs at times, quickly caught up on.
bob1029 6y ago
The developer/technical community at large seems to be inhuman in their ability to disregard how shitty a user experience can begin to feel once latency exceeds double digits.

This depends what "industry" you're talking about. If you're talking about "tech", I'd say 100ms doesn't matter for the vast majority of use-cases. Bear in mind that Australia/New Zealand have a ~250ms round trip latency to most web services, 4G can easily have a 300ms latency to Google, and developing countries are often in a much worse position. Depending on <100ms for anything is a non-starter for most of the web. If you're a service provider several layers removed from a human, then there may be enough between you and the user that it's critical that you respond in <100ms, but this is not most of the industry. More generally, I find this sort of comment to be similar to the sort of comment that asks why anyone would write a web service in a language as slow as Ruby or Python when we have languages like Go and Rust. They ignore the fact that there are huge, successful companies running on Ruby, Python, and other "slow" languages, all for very good reasons.

smolder 6y ago
100ms becomes significant if you're doing teleconferencing or other real time interaction. If we somehow improved latency and jitter on the internet generally, I think that'd do more to improve its usefulness than expanding bandwidth. And don't forget about the studies out there that show a relationship between load time and revenue in e-commerce: "A 100ms decrease in checkout page load speed amounted to a 1.55% increase in session-based conversion."
- lostcolony 6y ago
  If doing realtime interaction it's unlikely they'd negotiate more than one connection (reusing connections or not), so the test they've done hardly seems useful. I also have huge issues with a lot of those studies; 100ms from 200ms -> 100ms is immensely different than 1500ms -> 1400ms.
- joosters 6y ago
  This is 100ms for establishing a TLS connection - that's not the same as 100ms latency for all of a video stream.
  - smolder 6y ago
    Oh yeah, you're absolutely right. I got a bit tangential. Slow establishment wouldn't matter for a stream. I'm sure it's something more like the e-commerce page load time vs conversions scenario for OP.
    - axismundi 6y ago
      I could live with slow handshake, but it's ~110ms for every request.
      - Spooky23 6y ago
        I had a similar problem working with an managed file transfer service. Lots of small files would get bogged down by these sorts of issues. The other side of the connection wouldn’t change their config, so I implemented a proxy that maintained session state on our side.
- danpalmer 6y ago
  Teleconferencing is a good example of where this does matter, although I'd suggest that despite the current situation we're all in, it's not a significant part of the industry. > And don't forget about the studies out there that show a relationship between load time and revenue in e-commerce: "A 100ms decrease in checkout page load speed amounted to a 1.55% increase in session-based conversion." This is true, although I believe this relationship doesn't hold when you're moving from 200ms to 100ms, it's more for when you're moving from 1000ms to 900ms.
- user5994461 6y ago
  Latency is capped by two third the speed of light. The network latency is actually pretty good overall considering this (the issue is really on the user end with terrible Wifi and mobile connections). The speed of light is quite impactful when you're looking at different countries or continents.
  - iso1210 6y ago
    > Latency is capped by two third the speed of light Starlink: hold my beer. I'm not a HFT, I know I can't increase the speed of light, I know I can't tunnel through the earth to shave a few microseconds off a transmission. I can appreciate the difference between 300ms rtt and 900ms rtt - the later being a practical minimum when you're coping with outages in the 100-200ms range and networks reroute. Give me jitter over drops any day.
- axismundi 6y ago
  Yes, this is a real time platform. I spend some of my dev time optimising performance.
bufferoverflow 6y ago
Except it does matter. Every 100ms of latency caused Amazon 1% in sales. Every 500ms of latency cost Google 20% of their traffic. And these are 10 year old metrics, back when people tolerated slowness much more.
iso1210 6y ago
Computer industry in general, but network providers specifically. It's not about <100ms latency, it's about a <100ms outage I know I can't rely on providers to not have outages on 100ms -- a link fails in Sudan and BGP flaps on a peering it cane take seconds, let alone milliseconds, to respond. I have to be very careful, because if I want a two-way communication at that 250ms round trip time to NZ, that means I have to ensure my packets get there and back in 250ms, and not drop in the middle and require retransmissions, which rapidly escalates latency from 'barely noticable' up to 'painful' levels (1 second or above is meaningless, may as well send a fax[1]) 0ms : send packet containing informaiton 125ms : receive packet 135ms: process informaiton, send response 260ms: receive response 270ms: process response Job done. Trouble is, a 100ms outage from 0ms to 100ms means 0ms : send packet containing informaiton 125ms : fail to receive packet 135ms : notice packet missing, issue request for retransmit 260ms: receive retransmit request 260ms: retransmit 385ms : receive packet 395ms: process informaiton, send response 620ms: receive response 630ms: process response Given I therefore need (630-270 = 360ms) of buffer, that's more than doubled the response. But it gets worse 125ms : fail to receive packet 135ms : notice packet missing, issue request for retransmit (request gets lost) 395ms: notice retransmit request not received, issue request for retransmit 405ms: issue retransmit request 530ms : receive retransmit request, retransmit 655ms: receive packet 665ms: process information, send response 790ms: process response So to cope with a 100ms outage my application needs to insert a 520ms buffer on a 270ms conversation - practically tripling latency, and converting a near- real time convertsation into near-catastrophic talking over each other. Now this is fine, I can have two circuits, send the packets down both, and the chance of both being broken is minimal -- assuming the links are really separate (which is another challange. Hint, both patchs down the same undersea cable on different frequencies is not separation) However getting network providers to acknowlege, let alone measure, outages of 100ms, is practically impossible on a country basis, let alone an intercontinental basis. [1] hyperbole. Just.
deleted 6y ago
[deleted]

harlanji 6y ago

It’s mind boggling how much can happen in 100ms. That’s 6,000 protobuf kafka messages consumed in clojure, deserialized, serialized, sent and acknowledged in 2015 aws. Upgrading a task caused fraction of million message backlogs at times, quickly caught up on.

bob1029 6y ago

The developer/technical community at large seems to be inhuman in their ability to disregard how shitty a user experience can begin to feel once latency exceeds double digits.