4 ms·

The other node doesn't depend on the settings, it depends on not receiving a duplicate SYN.

by philh 12y ago

acqq 12y ago
Note what he actually writes: "Our TCP implementation has a variable base SYN retransmit timout, and in this case it was roughly 500ms. So most of the time the page load would fail with our TCP stack, but succeed with an off the shelf one that had a SYN retransmit timeout of 1 second." If his timeout were 1 second, his connection with the other node would work. It's his own choice to insist on "variable base SYN retransmit timeout" even with the nodes which weren't tested with such a setup. He also says: "The options are either to tell a customer that some traffic won't work (which is normally unacceptable, even if the root cause is undeniably on the other end), or to water down a useful feature a little bit to at least fail no more often than the "competition" does." Meaning, the way they use that less common setting is "all or nothing": either all clients or none. With the assumption that that way he will be "competitive." Again, why is then surprising to discover that "the whole world" is not perfect, once you implement your part with the assumption it were? Just because he knows what the "ideal case" is he can't expect that everything he confronts would be ideal.
- viraptor 12y ago
  I don't think it was an issue with the timeout. The other side simply could not handle retransmitted SYN. At all. The 500ms comes up only because that happened to be their RTT. If they used 1 second, this host would work, but another one using the same stack but with 1 second delay would continue to fail. It's not about the other host not having an ideal implementation, or about testing ideal cases. The other host had a bug which cannot cope with some situations. There's nothing they could do about it, apart from tweaking the timeout to some bigger value.
  - jsnell 12y ago
    Exactly. And note that we're not talking outlandish latencies here. A 2G connection, or a combination of a 3G connection + OS X + being on the wrong side of the globe would've done it. Or a single well placed packet loss (losing the SYNACK). Often when you have these kinds of incompatibility issues, they are possible to fix with no cost at all. There's no harm in rearranging or re-aliging your TCP options for maximum compatibility. Or it's possible to notice from either the handshake or later connection behavior that the other end is dodgy, and conditionally disable whatever feature might cause trouble. But you can't possibly do that when we have no information about the other end. That's why I was particularly annoyed by this issue. (And I certainly wasn't expecting everything to work perfectly, like "acqq" claims. That expectation would be quickly beaten out of anyone dealing with arbitrary TCP traffic.)
    - lukego 12y ago
      Could you map all such buggy devices on the internet and treat their known addresses specially? (Scan the whole public IPv4 range every month to find them?) Just an idea :-)
      - viraptor 12y ago
        It's potentially a "scan all ports on every public ipv4 address" job instead. Each port may be redirected to a different machine.
  - acqq 12y ago
    I worked for many years with the locations on the remote islands with the high RTT. You wouldn't believe how much software of the biggest industry players failed to work under that circumstances. My favorite case was when one company first claimed "OK on another side is probably the device from the competition." It was their device on another side too. They "debugged" the case for a bigger part of the year and the case remained unsolved. I'm absolutely not saying that the far side implementation was OK. I concur that the TCP is damn hard. It's just that you can't expect that "everybody else" is perfect. You have to plan to handle the special cases or not be surprised that you can't handle them.
    - viraptor 12y ago
      > You have to plan to handle the special cases or not be surprised that you can't handle them. What do you propose then? You send valid traffic and get a completely broken response - I don't see any space left for handling special cases here. Like they said in the article, it's the first packet and you have no information about the other side yet. This is not something you can avoid/workaround.
      - acqq 12y ago
        As soon as you think "I send a valid packet, everything must work" you're missing the point. Even OP at the end worries about the actual client communication actually not working. At the end, it's not who's "theoretically" right, it's "can you make it work given the real world limitations" that include the implementations not tested with your "clever better than the competition dynamic timeout modification." It can be clever, but be clever more and plan for the cases when it is against the real life limitations. And don't cry foul. It's you who move into less tested territories, expecting to be better than the "competition". The same as I don't complain about the herd reactions here. They are probable, therefore expected.
        viraptor 12y ago
        Sigh... let me say this again clearly, because you keep repeating the same point. This case has nothing to do with timeout modification. It will happen with or without it. If you use lower timeout you will run into the issue more often. If you use higher timeout - less often. But you will run into the issue anyway and there is no way to fix it. So just tell us what is your proposed solution / implied more tested territories. You try to start a connection, send SYN, don't see a response for X (500ms, 1s, whatever, you choose). What do you propose to do that isn't a retransmit (breaks the other side) or choosing new seq numbers (breaks good behaviour on your side by starting two connections and possibly tripping flood detection if you do it too often).

The other node doesn't depend on the settings, it depends on not receiving a duplicate SYN.

by philh 12y ago

The other node doesn't depend on the settings, it depends on not receiving a duplicate SYN.

acqq 12y ago

Note what he actually writes: "Our TCP implementation has a variable base SYN retransmit timout, and in this case it was roughly 500ms. So most of the time the page load would fail with our TCP stack, but succeed with an off the shelf one that had a SYN retransmit timeout of 1 second." If his timeout were 1 second, his connection with the other node would work. It's his own choice to insist on "variable base SYN retransmit timeout" even with the nodes which weren't tested with such a setup. He also says: "The options are either to tell a customer that some traffic won't work (which is normally unacceptable, even if the root cause is undeniably on the other end), or to water down a useful feature a little bit to at least fail no more often than the "competition" does." Meaning, the way they use that less common setting is "all or nothing": either all clients or none. With the assumption that that way he will be "competitive." Again, why is then surprising to discover that "the whole world" is not perfect, once you implement your part with the assumption it were? Just because he knows what the "ideal case" is he can't expect that everything he confronts would be ideal.

viraptor 12y ago
I don't think it was an issue with the timeout. The other side simply could not handle retransmitted SYN. At all. The 500ms comes up only because that happened to be their RTT. If they used 1 second, this host would work, but another one using the same stack but with 1 second delay would continue to fail. It's not about the other host not having an ideal implementation, or about testing ideal cases. The other host had a bug which cannot cope with some situations. There's nothing they could do about it, apart from tweaking the timeout to some bigger value.
- jsnell 12y ago
  Exactly. And note that we're not talking outlandish latencies here. A 2G connection, or a combination of a 3G connection + OS X + being on the wrong side of the globe would've done it. Or a single well placed packet loss (losing the SYNACK). Often when you have these kinds of incompatibility issues, they are possible to fix with no cost at all. There's no harm in rearranging or re-aliging your TCP options for maximum compatibility. Or it's possible to notice from either the handshake or later connection behavior that the other end is dodgy, and conditionally disable whatever feature might cause trouble. But you can't possibly do that when we have no information about the other end. That's why I was particularly annoyed by this issue. (And I certainly wasn't expecting everything to work perfectly, like "acqq" claims. That expectation would be quickly beaten out of anyone dealing with arbitrary TCP traffic.)
  - lukego 12y ago
    Could you map all such buggy devices on the internet and treat their known addresses specially? (Scan the whole public IPv4 range every month to find them?) Just an idea :-)
    - viraptor 12y ago
      It's potentially a "scan all ports on every public ipv4 address" job instead. Each port may be redirected to a different machine.
- acqq 12y ago
  I worked for many years with the locations on the remote islands with the high RTT. You wouldn't believe how much software of the biggest industry players failed to work under that circumstances. My favorite case was when one company first claimed "OK on another side is probably the device from the competition." It was their device on another side too. They "debugged" the case for a bigger part of the year and the case remained unsolved. I'm absolutely not saying that the far side implementation was OK. I concur that the TCP is damn hard. It's just that you can't expect that "everybody else" is perfect. You have to plan to handle the special cases or not be surprised that you can't handle them.
  - viraptor 12y ago
    > You have to plan to handle the special cases or not be surprised that you can't handle them. What do you propose then? You send valid traffic and get a completely broken response - I don't see any space left for handling special cases here. Like they said in the article, it's the first packet and you have no information about the other side yet. This is not something you can avoid/workaround.
    - acqq 12y ago
      As soon as you think "I send a valid packet, everything must work" you're missing the point. Even OP at the end worries about the actual client communication actually not working. At the end, it's not who's "theoretically" right, it's "can you make it work given the real world limitations" that include the implementations not tested with your "clever better than the competition dynamic timeout modification." It can be clever, but be clever more and plan for the cases when it is against the real life limitations. And don't cry foul. It's you who move into less tested territories, expecting to be better than the "competition". The same as I don't complain about the herd reactions here. They are probable, therefore expected.
      - viraptor 12y ago
        Sigh... let me say this again clearly, because you keep repeating the same point. This case has nothing to do with timeout modification. It will happen with or without it. If you use lower timeout you will run into the issue more often. If you use higher timeout - less often. But you will run into the issue anyway and there is no way to fix it. So just tell us what is your proposed solution / implied more tested territories. You try to start a connection, send SYN, don't see a response for X (500ms, 1s, whatever, you choose). What do you propose to do that isn't a retransmit (breaks the other side) or choosing new seq numbers (breaks good behaviour on your side by starting two connections and possibly tripping flood detection if you do it too often).