4 ms·

> But enabling tcp timestamps in general case brings little benefit and wastes 12 bytes of each packet for basically no gain. I disagree; TCP timestamps are aw

> But enabling tcp timestamps in general case brings little benefit and wastes 12 bytes of each packet for basically no gain. I disagree; TCP timestamps are awesome. Linux enables these by defaults. Quick search gives me some measurements from 2012 [1] that indicate that TCP timestamps are enabled on 83% of the top 100k web hosts. You can afford to waste 12 bytes; the bottleneck isn't these 12 bytes but how well you get congestion control to work. And congestion control relies on getting an accurate estimate of the round-trip time [1] https://link.springer.com/chapter/10.1007/978-3-642-36516-4_14 https://link.springer.com/chapter/10.1007/978-3-642-36516-4_... (paywall) Edit: typo Edit: Also, just because 83% of web hosts having it enabled does not imply that it is a good idea to do so in general. They could just all be running the linux defaults and these could be just wrong

djoldman 6y ago
One of the authors of the paper has it linked from his website: https://trammell.ch/publication/ecn-pam-2013/ https://trammell.ch/publication/ecn-pam-2013/
jsnell 6y ago
You can measure RTT continuously, accurately, and even in the presence of packet loss just with selective acks and a little bit of extra bookkeeping in the sender. It's hard to overstate how expensive TCP timestamps are. The thing is that they bloat every single packet including control packets. 2% of the world's bandwidth is being wasted on this. The only reason for anyone to implement TCP timestamps today is that iOS clients have horrible receive window scaling if timestamps are disabled. (Well, that was the only reason a few years ago when I was still in the game of keeping up with the quirks of different TCP stacks.) I wrote more on the subject at the time: https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/ https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/
- toast0 6y ago
  I would guess that Apple could be convinced to fix this, if someone has the right contact. The iOS TCP stack shows a lot of care, generally: they do path MTU probing well, and they've deployed MP-TCP (requires apps to enable it though), among other things I can't remember. Fixing performance if timestamps are disabled seems like something they'd do.
a1369209993 6y ago
> Also, just because 83% of web hosts having it enabled does not imply that it is a good idea to do so in general. Kudos on having the integrity to point this out even when it superficially weakens your argument. Also, a more relevant percentage would be: If you have 64KB packets (which you want, to get the best throughput) 12 bytes is a 0.018% overhead, less than one five-thousandth of your packet. Edit: and apparently it's 0.8% with the default MSS of 1460B. Ugh.
- jsnell 6y ago
  0.8% is still an underestimate. The average size of a TCP packet in real networks is about 500 bytes, due to the ACKs. And timestamps need to be present in the ACKs too.
  - a1369209993 6y ago
    > > Ugh.
ncmncm 6y ago
Small correction: for good congestion control you don't need RTT, you really need the transmission time. RTT is easier to get, but very noisy, and late. By the time you get it, it's obsolete. The way to get good clean flow control is to measure forward transmission time, and have the receiver keep a predictor on the sender updated. Increments in the forward transmission time signal changes in aggregate queue length. Your measurement updates are always about the past, so you need to apply control theory to use them correctly.

djoldman 6y ago

One of the authors of the paper has it linked from his website: https://trammell.ch/publication/ecn-pam-2013/ https://trammell.ch/publication/ecn-pam-2013/

jsnell 6y ago

You can measure RTT continuously, accurately, and even in the presence of packet loss just with selective acks and a little bit of extra bookkeeping in the sender. It's hard to overstate how expensive TCP timestamps are. The thing is that they bloat every single packet including control packets. 2% of the world's bandwidth is being wasted on this. The only reason for anyone to implement TCP timestamps today is that iOS clients have horrible receive window scaling if timestamps are disabled. (Well, that was the only reason a few years ago when I was still in the game of keeping up with the quirks of different TCP stacks.) I wrote more on the subject at the time: https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/ https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/

toast0 6y ago
I would guess that Apple could be convinced to fix this, if someone has the right contact. The iOS TCP stack shows a lot of care, generally: they do path MTU probing well, and they've deployed MP-TCP (requires apps to enable it though), among other things I can't remember. Fixing performance if timestamps are disabled seems like something they'd do.

a1369209993 6y ago

> Also, just because 83% of web hosts having it enabled does not imply that it is a good idea to do so in general. Kudos on having the integrity to point this out even when it superficially weakens your argument. Also, a more relevant percentage would be: If you have 64KB packets (which you want, to get the best throughput) 12 bytes is a 0.018% overhead, less than one five-thousandth of your packet. Edit: and apparently it's 0.8% with the default MSS of 1460B. Ugh.

jsnell 6y ago
0.8% is still an underestimate. The average size of a TCP packet in real networks is about 500 bytes, due to the ACKs. And timestamps need to be present in the ACKs too.
- a1369209993 6y ago
  > > Ugh.

ncmncm 6y ago

Small correction: for good congestion control you don't need RTT, you really need the transmission time. RTT is easier to get, but very noisy, and late. By the time you get it, it's obsolete. The way to get good clean flow control is to measure forward transmission time, and have the receiver keep a predictor on the sender updated. Increments in the forward transmission time signal changes in aggregate queue length. Your measurement updates are always about the past, so you need to apply control theory to use them correctly.