5 ms·

I'll tell you a dirty little secret of the protocol design. Say, you want to design a protocol with reliable delivery and/or loss detection. You will then have

I'll tell you a dirty little secret of the protocol design. Say, you want to design a protocol with reliable delivery and/or loss detection. You will then have ACKs, send window and retransmissions. Guess what? If you don't follow windowing semantics of TCP, then one of two things will happen on saturated links - either TCP will end up with all the bandwidth or you will. So - surprise! - you have no choice but to design a TCP clone. [ EDIT ] That said, there is a fundamental problem with TCP, when it's used for carrying secure connections. Since TCP acts as a pure transport protocol, it has no per-packet authentication and so any connection can be trivially DoS'd with a single fake FIN or RST packet. There are ways to solve this, e.g. by reversing the TCP and security layers and running TCP over ESP-over-UDP or TLS-over-UDP (OpenVPN protocol). This requires either writing a user-space TCP library or doing some nasty tunneling at the kernel level, but even as cumbersome as this is, it's still not a reason enough to re-invent the wheel. Also, if you want compression, it's readily available in TLS or as a part of IPsec stack (IPcomp). If you want FEC, same thing - just add a custom transform to TLS and let the client and server negotiate if to use it or not. I mean, every network programmer invents a protocol or two in his lifetime. It's like a right of passage and it's really not a big deal. Unless it ends up attached to a brand name, in which case it starts attracting snarky remarks like this one :)

xyzzy123 14y ago
I agree with your original point, but none of the stuff after [EDIT] makes much sense to me. "Since TCP acts as a pure transport protocol, it has no per-packet authentication info and so any connection can be trivially DoS'd with a single fake FIN or RST packet." I guess you mean if you're sharing medium or your adversary has MITM? You're generally screwed for DoS in that scenario anyway, no matter what encryption thingy you use. Otherwise, any modern TCP stack should be able to reject blind FINs or RSTs as out-of-sequence. Generally, the reason people avoid TCP-over-TCP is not DoS, but basically "timer fights", as per: http://sites.inka.de/~W1011/devel/tcp-tcp.html http://sites.inka.de/~W1011/devel/tcp-tcp.html. That said, everything-over SSL is still popular for practical reasons, such as clientless VPNs which need to work around proxies, firewalls etc.
- latitude 14y ago
  No, not the MITM. A simple denial of service - I can terminate your TLS-over-TCP connection at will by sending a FIN or RST packet. More generally speaking, I can alter the state of your secure over-TCP connection with an unauthenticated packet. Compare this to ESP where every packet that can potentially alter the state is first authenticated.
  - xyzzy123 14y ago
    Blind, or with some way to observe the sequence numbers? If it's blind (e.g. window guessing) I'm interested in what your techniques are. If it's not blind and you don't have MITM and you don't share a medium, I'm interested in how you're doing that. Challenge: I have the following TCP connection open (oh man I'm really asking for it, but what the heck :) tcp 0 0 192.168.0.10:52352 216.240.155.220:80 ESTABLISHED 16284/nc If you can terminate it with a FIN or an RST in the next hour, I'll post proclaiming your victory. If either the box or my local link drops, we'll call it a draw.
    - tcoppi 14y ago
      There aren't that many combinations you need to brute force in order for it to work. http://www.blackhatlibrary.net/TCP-RST_Injection http://www.blackhatlibrary.net/TCP-RST_Injection has sample code and an explanation of the attack. EDIT: Just downloaded the sample code and it looks like it won't quite work out of the box, but I'm sure someone with more time could through together something with scapy or similar pretty quickly.
      - xyzzy123 14y ago
        Well I did post the source port, which reduces the effort by a factor of tens of thousands... also, the up-thread did say "with a single fake RST or FIN packet"... which is why I mentioned window guessing... Also, crap, I'm an idiot. Let me do that non-natted ;) tcp 0 0 216.240.155.220:4007 60.225.131.226:43972 ESTABLISHED 18694/nc
easytiger 14y ago
Yea you end up with a udp retrans storm on saturated links. It will start losing the rerequests and the rerequest for the lost packets. Some solutions to this use tcp to do the resetting of the connection.
exDM69 14y ago
> Say, you want to design a protocol with reliable delivery and/or loss detection. You will then have ACKs... > So - surprise! - you have no choice but to design a TCP clone. Some use cases don't really want TCP or a TCP clone, for example multi-player games might only use a semi-reliable loss detection system which has ACK's but does not attempt retransmission on failure to deliver. This works a lot better than TCP for that use case, and is not really a "TCP clone".
- xyzzy123 14y ago
  The really sad thing about this though is that "the internet" no longer works for stuff which isn't TCP. I guess it does, but not for your whole target market. The lowest common denominator is users behind an NTLM proxy. Mass market products don't generally work unless they have a fallback which is "I can make it tunnel through proxies". Which is HTTP or CONNECT :( Which means no teams will do something which isn't TCP anymore, since it's not a thing they can tunnel :( Interested in counterexamples though. This is the same reason why a UDP replacement is a great idea but a waste of time for most people developing a product. This makes me sad.
  - ajross 14y ago
    As mentioned upthread, most multiplayer games are still UDP. And indeed, they generally don't work through proxies.
  - potkor 14y ago
    You have it the wrong way around. People who cordon themselves off and only leave HTTP working aren't on the Internet. They don't want non-web apps to work. Those networks deliberately rule out networked games, uTP, VoIP, IPsec, WebRTC, etc. If you want to worry about people teetering on the edge of the Internet, you should worry about people behind NAT and its prevalence to the point of people confusing NAT boxes and routers... (Re your last sentence, it's not a UDP replacement, see other comments.)
deleted 14y ago
[deleted]
deleted 14y ago
[deleted]
gritzko 14y ago
Actually, there are also "scavenger" congestion control algorithms that use leftover bandwidth. The most high-profile example is LEDBAT used by BitTorrent's uTP. uTP is indeed a TCP clone, but I had my chance to develop a protocol which was using LEDBAT and it was NOT a TCP clone at all (it later became IETF PPSP). So, TCP eating all the bandwidth is not an issue sometimes.
- pjscott 14y ago
  uTP is designed to let TCP eat all the bandwidth on saturated links, and generally defer to TCP whenever possible. For BitTorrent, that's exactly what you want. For web browsing, you probably want to be more aggressive. I skimmed the source code, and it looks like they're using the same CUBIC congestion control algorithm that Linux's TCP stack has been using by default since 2007. http://en.wikipedia.org/wiki/CUBIC_TCP http://en.wikipedia.org/wiki/CUBIC_TCP
js2 14y ago
From quic_protocol.h: // Defines for all types of congestion feedback that will be negotiated in QUIC, // kTCP MUST be supported by all QUIC implementations to guarantee 100% // compatibility. enum CongestionFeedbackType { kTCP, // Used to mimic TCP. kInterArrival, // Use additional inter arrival information. kFixRate, // Provided for testing. }; :-)
cbsmith 14y ago
TCP has more than a few other issues as well, not the least of which is that signalling for those windowing semantics doesn't effectively distinguish between congestion and packet loss.
davesims 14y ago
"Say, you want to design a protocol with reliable delivery and/or loss detection. You will then have ACKs, send window and retransmissions." Maybe I'm missing something (likely) but it seems to me any potential replacement for UDP specifically doesn't care about packet loss, acks, etc. The whole point of UDP is speed over reliability, yes?

xyzzy123 14y ago

I agree with your original point, but none of the stuff after [EDIT] makes much sense to me. "Since TCP acts as a pure transport protocol, it has no per-packet authentication info and so any connection can be trivially DoS'd with a single fake FIN or RST packet." I guess you mean if you're sharing medium or your adversary has MITM? You're generally screwed for DoS in that scenario anyway, no matter what encryption thingy you use. Otherwise, any modern TCP stack should be able to reject blind FINs or RSTs as out-of-sequence. Generally, the reason people avoid TCP-over-TCP is not DoS, but basically "timer fights", as per: http://sites.inka.de/~W1011/devel/tcp-tcp.html http://sites.inka.de/~W1011/devel/tcp-tcp.html. That said, everything-over SSL is still popular for practical reasons, such as clientless VPNs which need to work around proxies, firewalls etc.

latitude 14y ago
No, not the MITM. A simple denial of service - I can terminate your TLS-over-TCP connection at will by sending a FIN or RST packet. More generally speaking, I can alter the state of your secure over-TCP connection with an unauthenticated packet. Compare this to ESP where every packet that can potentially alter the state is first authenticated.
- xyzzy123 14y ago
  Blind, or with some way to observe the sequence numbers? If it's blind (e.g. window guessing) I'm interested in what your techniques are. If it's not blind and you don't have MITM and you don't share a medium, I'm interested in how you're doing that. Challenge: I have the following TCP connection open (oh man I'm really asking for it, but what the heck :) tcp 0 0 192.168.0.10:52352 216.240.155.220:80 ESTABLISHED 16284/nc If you can terminate it with a FIN or an RST in the next hour, I'll post proclaiming your victory. If either the box or my local link drops, we'll call it a draw.
  - tcoppi 14y ago
    There aren't that many combinations you need to brute force in order for it to work. http://www.blackhatlibrary.net/TCP-RST_Injection http://www.blackhatlibrary.net/TCP-RST_Injection has sample code and an explanation of the attack. EDIT: Just downloaded the sample code and it looks like it won't quite work out of the box, but I'm sure someone with more time could through together something with scapy or similar pretty quickly.
    - xyzzy123 14y ago
      Well I did post the source port, which reduces the effort by a factor of tens of thousands... also, the up-thread did say "with a single fake RST or FIN packet"... which is why I mentioned window guessing... Also, crap, I'm an idiot. Let me do that non-natted ;) tcp 0 0 216.240.155.220:4007 60.225.131.226:43972 ESTABLISHED 18694/nc

easytiger 14y ago

Yea you end up with a udp retrans storm on saturated links. It will start losing the rerequests and the rerequest for the lost packets. Some solutions to this use tcp to do the resetting of the connection.

exDM69 14y ago

> Say, you want to design a protocol with reliable delivery and/or loss detection. You will then have ACKs... > So - surprise! - you have no choice but to design a TCP clone. Some use cases don't really want TCP or a TCP clone, for example multi-player games might only use a semi-reliable loss detection system which has ACK's but does not attempt retransmission on failure to deliver. This works a lot better than TCP for that use case, and is not really a "TCP clone".

xyzzy123 14y ago
The really sad thing about this though is that "the internet" no longer works for stuff which isn't TCP. I guess it does, but not for your whole target market. The lowest common denominator is users behind an NTLM proxy. Mass market products don't generally work unless they have a fallback which is "I can make it tunnel through proxies". Which is HTTP or CONNECT :( Which means no teams will do something which isn't TCP anymore, since it's not a thing they can tunnel :( Interested in counterexamples though. This is the same reason why a UDP replacement is a great idea but a waste of time for most people developing a product. This makes me sad.
- ajross 14y ago
  As mentioned upthread, most multiplayer games are still UDP. And indeed, they generally don't work through proxies.
- potkor 14y ago
  You have it the wrong way around. People who cordon themselves off and only leave HTTP working aren't on the Internet. They don't want non-web apps to work. Those networks deliberately rule out networked games, uTP, VoIP, IPsec, WebRTC, etc. If you want to worry about people teetering on the edge of the Internet, you should worry about people behind NAT and its prevalence to the point of people confusing NAT boxes and routers... (Re your last sentence, it's not a UDP replacement, see other comments.)

deleted 14y ago

[deleted]

deleted 14y ago

[deleted]

gritzko 14y ago

Actually, there are also "scavenger" congestion control algorithms that use leftover bandwidth. The most high-profile example is LEDBAT used by BitTorrent's uTP. uTP is indeed a TCP clone, but I had my chance to develop a protocol which was using LEDBAT and it was NOT a TCP clone at all (it later became IETF PPSP). So, TCP eating all the bandwidth is not an issue sometimes.

pjscott 14y ago
uTP is designed to let TCP eat all the bandwidth on saturated links, and generally defer to TCP whenever possible. For BitTorrent, that's exactly what you want. For web browsing, you probably want to be more aggressive. I skimmed the source code, and it looks like they're using the same CUBIC congestion control algorithm that Linux's TCP stack has been using by default since 2007. http://en.wikipedia.org/wiki/CUBIC_TCP http://en.wikipedia.org/wiki/CUBIC_TCP

js2 14y ago

From quic_protocol.h: // Defines for all types of congestion feedback that will be negotiated in QUIC, // kTCP MUST be supported by all QUIC implementations to guarantee 100% // compatibility. enum CongestionFeedbackType { kTCP, // Used to mimic TCP. kInterArrival, // Use additional inter arrival information. kFixRate, // Provided for testing. }; :-)

cbsmith 14y ago

TCP has more than a few other issues as well, not the least of which is that signalling for those windowing semantics doesn't effectively distinguish between congestion and packet loss.

davesims 14y ago

"Say, you want to design a protocol with reliable delivery and/or loss detection. You will then have ACKs, send window and retransmissions." Maybe I'm missing something (likely) but it seems to me any potential replacement for UDP specifically doesn't care about packet loss, acks, etc. The whole point of UDP is speed over reliability, yes?