10 ms·

Impressive that he managed to say so much without mentioning head-of-line blocking, which is the main TCP problem with realtime apps. This is not due to conges

by zurn 13y ago

Impressive that he managed to say so much without mentioning head-of-line blocking, which is the main TCP problem with realtime apps. This is not due to congestion control, but because of the ordered byte stream semantics of TCP. You can't deliver a packet until the retransmission dance is done for any previous lost packets. As an aside, these days it's worth noting that a 3G, WLAN or 4G link layer will try very hard not to drop any packets. So when you're doing UDP, watch out for those 40000 millisecond old packets that will burst to your lap once the rain cloud moves away from the cell tower or your co-worker's cocoa blings in the microwave.

lucb1e 13y ago
He didn't mention the term head-of-line blocking, but the issue was apparent to me as I read it. Perhaps because I have a bit of previous experience with tcp vs udp, but still.
yvdriess 13y ago
I was under the impression that packets behind the blocked TCP packet still sent through the network, relying on the client-side or last-hop to maintain the sequential stream illusion and thus reducing the delay of the subsequent frames to the delay blocked/resent frame. Could with a deeper knowledge confirm or deny this?
- mikeash 13y ago
  That's true, but the delay from waiting for the lost packet to be retransmitted is the problem, not a delay for subsequent data after retransmitting the lost packet. Retransmission can take a substantial amount of time.
- eggnet 13y ago
  Yes that is true but maintaining the illusion means that the application can't read the data after the lost packet until the packet is retransmitted and received. So yes, selective acks preserve network bandwidth but don't solve the real time problems. With UDP the client application can receive all packets when they arrive even if there are some missing.
  - ForHackernews 13y ago
    > With UDP the client application can receive all packets when they arrive even if there are some missing. Isn't that a good thing?
    - lotharbot 13y ago
      Depends on the game. I still play a game that's almost 20 years old (the original Descent), which has a small but active multiplayer community [0]. I regularly play against players in my own house as well as players from as far away as Brazil, Qatar, and Australia -- so I have to deal with various connection consistency issues. Given the game physics, I would much rather have out-of-order packets (resulting in an opposing ship briefly flashing in a wrong location and firing a shot from there) than having a delay on in-order packets (resulting in an opposing ship completely freezing, and then completing several seconds of movement and several seconds of shots in a fraction of a second). There are other games that behave differently, where TCP would be preferable. [0] descentrangers.com for anarchy, cooperative, and team games; descentchampions.org for competitive 1v1s.
    - Nacraile 13y ago
      Yes. It gives you the flexibility to make your own decision in the trade off between: increased latency vs. bandwidth overhead vs. loss of data. With TCP, you get increased latency. If your protocol is stateless, you might choose to lose the data, because you know that the loss doesn't matter as soon as you get a new update. This might be applicable in a games, if the state you need to synchronize (say, position) is quite small. Even if you need your protocol to be stateful: you might choose to design your protocol such that loss results in a temporary degradation in the client view of state, but which will converge back on the correct state as it receives new data. This is a very common pattern; the quake net protocol referenced in OP is an example, as are most streaming A/V codecs. If you absolutely cannot tolerate the loss of any data, and are also latency sensitive, you can also choose to add redundant data so you can recover the lost data without retransmitting. And you can make intermediate trade-offs: A little bit of extra overhead for error-correction, and in the now-rare cases where error rate overwhelms your correction ability, you accept a degradation / retransmit round-trip. Or you accept a certain amount of degradation before you need retransmission to recover. Etc.
rdtsc 13y ago
SCTP is sometimes promoted as an alternative to solve that problem. You can have multiple stream per connection so if one blocks the other ones can keep going.
- lallysingh 13y ago
  My thoughts exactly. Also, you can send some data reliably (e.g., a new character model) and other data unreliably on the the same SCTP association.
- scott_karana 13y ago
  SCTP has always looked intriguing to me, but would most backbones and ISPs pass it natively? It sounds like it would be a waste to tunnel it in UDP, which is the alternative...
  - _delirium 13y ago
    The poor prospects for getting widespread SCTP support was one (though not the only) justification the SPDY people cited for doing some similar things at the application-protocol level: http://www.chromium.org/spdy/spdy-whitepaper http://www.chromium.org/spdy/spdy-whitepaper
  - derefr 13y ago
    WebRTC Data Channels are, in fact, SCTP tunnelled over UDP (or rather, over DTLS over UDP.) They work pretty well. (I'm not sure how much of the redundancy is shaved off--does anyone know if WebRTC's tries to encapsulate "normative" SCTP, or whether it uses SCTP without checksum/port information?)
    - rdtsc 13y ago
      Yeah not sure either, good question. There is also QUIC protocol also by Google, which is yet another crack at solving the problem. Wonder if those 2 groups sat down and discussed merging or re-using code.
  - MichaelGG 13y ago

by zurn 13y ago

lucb1e 13y ago

He didn't mention the term head-of-line blocking, but the issue was apparent to me as I read it. Perhaps because I have a bit of previous experience with tcp vs udp, but still.

yvdriess 13y ago

I was under the impression that packets behind the blocked TCP packet still sent through the network, relying on the client-side or last-hop to maintain the sequential stream illusion and thus reducing the delay of the subsequent frames to the delay blocked/resent frame. Could with a deeper knowledge confirm or deny this?

mikeash 13y ago
That's true, but the delay from waiting for the lost packet to be retransmitted is the problem, not a delay for subsequent data after retransmitting the lost packet. Retransmission can take a substantial amount of time.
eggnet 13y ago
Yes that is true but maintaining the illusion means that the application can't read the data after the lost packet until the packet is retransmitted and received. So yes, selective acks preserve network bandwidth but don't solve the real time problems. With UDP the client application can receive all packets when they arrive even if there are some missing.
- ForHackernews 13y ago
  > With UDP the client application can receive all packets when they arrive even if there are some missing. Isn't that a good thing?
  - lotharbot 13y ago
    Depends on the game. I still play a game that's almost 20 years old (the original Descent), which has a small but active multiplayer community [0]. I regularly play against players in my own house as well as players from as far away as Brazil, Qatar, and Australia -- so I have to deal with various connection consistency issues. Given the game physics, I would much rather have out-of-order packets (resulting in an opposing ship briefly flashing in a wrong location and firing a shot from there) than having a delay on in-order packets (resulting in an opposing ship completely freezing, and then completing several seconds of movement and several seconds of shots in a fraction of a second). There are other games that behave differently, where TCP would be preferable. [0] descentrangers.com for anarchy, cooperative, and team games; descentchampions.org for competitive 1v1s.
  - Nacraile 13y ago
    Yes. It gives you the flexibility to make your own decision in the trade off between: increased latency vs. bandwidth overhead vs. loss of data. With TCP, you get increased latency. If your protocol is stateless, you might choose to lose the data, because you know that the loss doesn't matter as soon as you get a new update. This might be applicable in a games, if the state you need to synchronize (say, position) is quite small. Even if you need your protocol to be stateful: you might choose to design your protocol such that loss results in a temporary degradation in the client view of state, but which will converge back on the correct state as it receives new data. This is a very common pattern; the quake net protocol referenced in OP is an example, as are most streaming A/V codecs. If you absolutely cannot tolerate the loss of any data, and are also latency sensitive, you can also choose to add redundant data so you can recover the lost data without retransmitting. And you can make intermediate trade-offs: A little bit of extra overhead for error-correction, and in the now-rare cases where error rate overwhelms your correction ability, you accept a degradation / retransmit round-trip. Or you accept a certain amount of degradation before you need retransmission to recover. Etc.

rdtsc 13y ago

SCTP is sometimes promoted as an alternative to solve that problem. You can have multiple stream per connection so if one blocks the other ones can keep going.

lallysingh 13y ago
My thoughts exactly. Also, you can send some data reliably (e.g., a new character model) and other data unreliably on the the same SCTP association.
scott_karana 13y ago
SCTP has always looked intriguing to me, but would most backbones and ISPs pass it natively? It sounds like it would be a waste to tunnel it in UDP, which is the alternative...
- _delirium 13y ago
  The poor prospects for getting widespread SCTP support was one (though not the only) justification the SPDY people cited for doing some similar things at the application-protocol level: http://www.chromium.org/spdy/spdy-whitepaper http://www.chromium.org/spdy/spdy-whitepaper
- derefr 13y ago
  WebRTC Data Channels are, in fact, SCTP tunnelled over UDP (or rather, over DTLS over UDP.) They work pretty well. (I'm not sure how much of the redundancy is shaved off--does anyone know if WebRTC's tries to encapsulate "normative" SCTP, or whether it uses SCTP without checksum/port information?)
  - rdtsc 13y ago
    Yeah not sure either, good question. There is also QUIC protocol also by Google, which is yet another crack at solving the problem. Wonder if those 2 groups sat down and discussed merging or re-using code.
- MichaelGG 13y ago