4 ms·

There is nothing inherently wrong with modern TCP for long distance data transfer. You do need to use the right congestion algorithm and some extensions enabled

There is nothing inherently wrong with modern TCP for long distance data transfer. You do need to use the right congestion algorithm and some extensions enabled to handle large window sizes to get high speeds at long distances. But UDP does not magically solve anything; if you want reliable data transfer over UDP you have to basically implement almost everything TCP does on top of it. The main benefit is that you are no longer relying on the operating system's implementation of TCP, the drawback is that you won't get any help from the OS or even hardware.

_8j50 4y ago
I thought you can't keep increasing window sizes, it is a physics limit? The mere fact that you need to wait for an ACK is the problem? https://en.m.wikipedia.org/wiki/Bandwidth-delay_product https://en.m.wikipedia.org/wiki/Bandwidth-delay_product
- zinekeller 4y ago
  As GP said: > if you want reliable data transfer over UDP you have to basically implement almost everything TCP does on top of it. Even QUIC has an ACK-like signal so you will never avoid it as otherwise you will deal with the two generals' problem (https://en.wikipedia.org/wiki/Two_Generals%27_Problem https://en.wikipedia.org/wiki/Two_Generals%27_Problem). The reason QUIC is more efficient than TCP is simply due to the fact that TCP can't evolve further due to middleboxes (called ossification, https://en.wikipedia.org/wiki/Protocol_ossification https://en.wikipedia.org/wiki/Protocol_ossification), not that TCP is inherently bad.
  - _8j50 4y ago
    Thanks for the explanation, I didn't know that about QUIC. But QUIC is trying to implement reliable delivery for applications, which makes sense as a general purpose protocol. but if I were doing a file transfer app/protocol these days for example I would send hash+Mac+metadata reliably and then check chunks of the file (unreliable+encrypted UDP)as they are decrypted (have to anyways for integrity checking as part if decryption), and peform reliable retransmission request when decryptiom fails. That's what I meant by application layer, application and use-case specific protocols. but perhaps general purpose protocols are still better (I am not sure, I was asking in my post, sorry for not being clear)
    - zinekeller 4y ago
      Then you would generally try to look at probabilistic reliability. One way systems like digital broadcasting requires probabilistic reliability so a lot of research was done here. Digital broadcasting has produced a lot of answers in this area, as you need to deal with both data corruption and missing data. On the data corruption side, you would want to implement a forward-error correction (FEC, https://en.wikipedia.org/wiki/Error_correction_code#Forward_error_correction https://en.wikipedia.org/wiki/Error_correction_code#Forward_...) to allow recovery of corrupted packets, while on missing data it can be mitigated by fountain codes (https://en.wikipedia.org/wiki/Fountain_code https://en.wikipedia.org/wiki/Fountain_code) that is basically FEC but deals with missing pieces instead of incorrect data. This of course does not guarantee 100% delivery but in practical terms this is humanly reliable.
      - _8j50 4y ago
        Thank you, that certainly gives me a lot to think about in this area.
    - simmervigor 4y ago
      You could build this as an application mapping over QUIC. Within a single QUIC connection use reliable streams for the metadata delivery, and unreliable datagrams [1] for the payload. The benefit of a single connection is shared security context and shared fates of different datatype transfers. [1] - https://www.rfc-editor.org/rfc/rfc9221.html https://www.rfc-editor.org/rfc/rfc9221.html
- dilyevsky 4y ago
  With scaling tcp can use window up to 1GB. Which is enough for 1s rtt which is ridiculously high @ 8gbps which is also ridiculous for long haul on a single flow. In practice you probably almost always just parallelize into multiple flows
  - _8j50 4y ago
    I had no idea TCP can reach 8gbps over 1s rtt. I suppose folks that tune it and use the right algorithm can get that or multi-threaded transfers working. The problem is most services and applications are not specifically tuned for this.
    - gsliepen 4y ago
      They don't need to; it is the kernel that implements TCP, and on Linux at least you can easily change the congestion control algorithm that is used and some other parameters. Again, UDP doesn't magically solve anything, and QUIC implements congestion control in userspace, so then you have to wonder: is my QUIC implementation tuned for 8gbps over 1s rtt? Note that your example of file transfer is a special case that an application can be specifically tuned for; it can have a receive window that is as large as the file it is going to receive, because it can use the file on disk as the backing store for that window, and it can implement an unlimited form of selective ACK. This is really not something that you can do with a generic transport protocol that implements the POSIX sockets API.
Matthias247 4y ago
The congestion control algorithms that are applied in TCP and QUIC are actually even the same ones (e.g. Cubic, BBR, etc), so the technologies behave not too different over higher RTTs. Both will ramp up a congestion control window, which can get pretty big. There are differences in behavior regarding what happens if loss is encountered and what exactly is retransmitted, but those still don't make QUIC radically different in terms of behavior and performance.

_8j50 4y ago

I thought you can't keep increasing window sizes, it is a physics limit? The mere fact that you need to wait for an ACK is the problem? https://en.m.wikipedia.org/wiki/Bandwidth-delay_product https://en.m.wikipedia.org/wiki/Bandwidth-delay_product

zinekeller 4y ago
As GP said: > if you want reliable data transfer over UDP you have to basically implement almost everything TCP does on top of it. Even QUIC has an ACK-like signal so you will never avoid it as otherwise you will deal with the two generals' problem (https://en.wikipedia.org/wiki/Two_Generals%27_Problem https://en.wikipedia.org/wiki/Two_Generals%27_Problem). The reason QUIC is more efficient than TCP is simply due to the fact that TCP can't evolve further due to middleboxes (called ossification, https://en.wikipedia.org/wiki/Protocol_ossification https://en.wikipedia.org/wiki/Protocol_ossification), not that TCP is inherently bad.
- _8j50 4y ago
  Thanks for the explanation, I didn't know that about QUIC. But QUIC is trying to implement reliable delivery for applications, which makes sense as a general purpose protocol. but if I were doing a file transfer app/protocol these days for example I would send hash+Mac+metadata reliably and then check chunks of the file (unreliable+encrypted UDP)as they are decrypted (have to anyways for integrity checking as part if decryption), and peform reliable retransmission request when decryptiom fails. That's what I meant by application layer, application and use-case specific protocols. but perhaps general purpose protocols are still better (I am not sure, I was asking in my post, sorry for not being clear)
  - zinekeller 4y ago
    Then you would generally try to look at probabilistic reliability. One way systems like digital broadcasting requires probabilistic reliability so a lot of research was done here. Digital broadcasting has produced a lot of answers in this area, as you need to deal with both data corruption and missing data. On the data corruption side, you would want to implement a forward-error correction (FEC, https://en.wikipedia.org/wiki/Error_correction_code#Forward_error_correction https://en.wikipedia.org/wiki/Error_correction_code#Forward_...) to allow recovery of corrupted packets, while on missing data it can be mitigated by fountain codes (https://en.wikipedia.org/wiki/Fountain_code https://en.wikipedia.org/wiki/Fountain_code) that is basically FEC but deals with missing pieces instead of incorrect data. This of course does not guarantee 100% delivery but in practical terms this is humanly reliable.
    - _8j50 4y ago
      Thank you, that certainly gives me a lot to think about in this area.
  - simmervigor 4y ago
    You could build this as an application mapping over QUIC. Within a single QUIC connection use reliable streams for the metadata delivery, and unreliable datagrams [1] for the payload. The benefit of a single connection is shared security context and shared fates of different datatype transfers. [1] - https://www.rfc-editor.org/rfc/rfc9221.html https://www.rfc-editor.org/rfc/rfc9221.html
dilyevsky 4y ago
With scaling tcp can use window up to 1GB. Which is enough for 1s rtt which is ridiculously high @ 8gbps which is also ridiculous for long haul on a single flow. In practice you probably almost always just parallelize into multiple flows
- _8j50 4y ago
  I had no idea TCP can reach 8gbps over 1s rtt. I suppose folks that tune it and use the right algorithm can get that or multi-threaded transfers working. The problem is most services and applications are not specifically tuned for this.
  - gsliepen 4y ago
    They don't need to; it is the kernel that implements TCP, and on Linux at least you can easily change the congestion control algorithm that is used and some other parameters. Again, UDP doesn't magically solve anything, and QUIC implements congestion control in userspace, so then you have to wonder: is my QUIC implementation tuned for 8gbps over 1s rtt? Note that your example of file transfer is a special case that an application can be specifically tuned for; it can have a receive window that is as large as the file it is going to receive, because it can use the file on disk as the backing store for that window, and it can implement an unlimited form of selective ACK. This is really not something that you can do with a generic transport protocol that implements the POSIX sockets API.

Matthias247 4y ago

The congestion control algorithms that are applied in TCP and QUIC are actually even the same ones (e.g. Cubic, BBR, etc), so the technologies behave not too different over higher RTTs. Both will ramp up a congestion control window, which can get pretty big. There are differences in behavior regarding what happens if loss is encountered and what exactly is retransmitted, but those still don't make QUIC radically different in terms of behavior and performance.