5 ms·

I like the idea mentioned in the article of increasing the number of packets sent in the slow start - as far as I know you could just crank that from the server

I like the idea mentioned in the article of increasing the number of packets sent in the slow start - as far as I know you could just crank that from the server side TCP stack to something much larger, right?

blfr 4y ago
Yeah, now that most of us control the server, is this an option?
- bawolff 4y ago
  Never tried it, but its listed as an option at https://linux.die.net/man/8/ip https://linux.die.net/man/8/ip
  - blfr 4y ago
    Seems fraught with potential issues. Can I do this just for nginx?
    - LinuxBender 4y ago
      Possibly but I've never tried this. One could create a virtual routing table and apply ACL's for the nginx listening ports then apply the initcwnd and initrwnd routing changes to that virtual table.
- LinuxBender 4y ago
  Yes. This is what I use on my hobby sites and home network. About a decade ago I was using 16 when the default changed to 10 from 3. This is with fq_codel+cdg. Most prefer bbr for servers but I have my own quirky use cases. ip route change local 127.0.0.0/8 dev lo initcwnd 128 initrwnd 128 ip route | grep default | while read p; do ip route change $p initcwnd 32 initrwnd 32; done I would suggest performing significant testing from high latency connections before making changes on anything important. i.e. iperf3/nuttcp from a VM in another country. This would also be a good time to get numbers from different congestion control algorithms and default qdiscs. e.g. net.ipv4.tcp_congestion_control and net.core.default_qdisc [1] [Edit] I should also add that changing these values may require different methods depending on the distribution. [2] [1] - https://www.kernel.org/doc/Documentation/sysctl/net.txt https://www.kernel.org/doc/Documentation/sysctl/net.txt [2] - https://serverfault.com/questions/546523/linux-initcwnd-and-initrwnd-via-etc-sysctl-conf https://serverfault.com/questions/546523/linux-initcwnd-and-...
dmurray 4y ago
Or have the server adapt based on page size. Even if starting with 10 packets at a time is optimal when sending an arbitrarily large payload, it's probably not optimal for an 11-packet payload - just send the whole thing at once. I don't know enough about web servers to know why the good ones don't already do this. I suppose it may be reading data from a stream where it doesn't know the size up front.
taneq 4y ago
This was my very first thought - TCP/IP behaviour isn’t carved in stone, just make your server send more data at the start!
- youngtaff 4y ago
  Most CDNs etc. use packet pacing to feed the packets out in a continual stream without overwhelming the network
heisenbit 4y ago
There is a reason for this limit. It is intended to avoid congesting the network and avoid packet loss. Yes, when you always yell your counterpart is a lot less likely to miss your message but don‘t be surprised if people around you get upset.

blfr 4y ago

Yeah, now that most of us control the server, is this an option?

bawolff 4y ago
Never tried it, but its listed as an option at https://linux.die.net/man/8/ip https://linux.die.net/man/8/ip
- blfr 4y ago
  Seems fraught with potential issues. Can I do this just for nginx?
  - LinuxBender 4y ago
    Possibly but I've never tried this. One could create a virtual routing table and apply ACL's for the nginx listening ports then apply the initcwnd and initrwnd routing changes to that virtual table.
LinuxBender 4y ago
Yes. This is what I use on my hobby sites and home network. About a decade ago I was using 16 when the default changed to 10 from 3. This is with fq_codel+cdg. Most prefer bbr for servers but I have my own quirky use cases. ip route change local 127.0.0.0/8 dev lo initcwnd 128 initrwnd 128 ip route | grep default | while read p; do ip route change $p initcwnd 32 initrwnd 32; done I would suggest performing significant testing from high latency connections before making changes on anything important. i.e. iperf3/nuttcp from a VM in another country. This would also be a good time to get numbers from different congestion control algorithms and default qdiscs. e.g. net.ipv4.tcp_congestion_control and net.core.default_qdisc [1] [Edit] I should also add that changing these values may require different methods depending on the distribution. [2] [1] - https://www.kernel.org/doc/Documentation/sysctl/net.txt https://www.kernel.org/doc/Documentation/sysctl/net.txt [2] - https://serverfault.com/questions/546523/linux-initcwnd-and-initrwnd-via-etc-sysctl-conf https://serverfault.com/questions/546523/linux-initcwnd-and-...

dmurray 4y ago

Or have the server adapt based on page size. Even if starting with 10 packets at a time is optimal when sending an arbitrarily large payload, it's probably not optimal for an 11-packet payload - just send the whole thing at once. I don't know enough about web servers to know why the good ones don't already do this. I suppose it may be reading data from a stream where it doesn't know the size up front.

taneq 4y ago

This was my very first thought - TCP/IP behaviour isn’t carved in stone, just make your server send more data at the start!

youngtaff 4y ago
Most CDNs etc. use packet pacing to feed the packets out in a continual stream without overwhelming the network

heisenbit 4y ago

There is a reason for this limit. It is intended to avoid congesting the network and avoid packet loss. Yes, when you always yell your counterpart is a lot less likely to miss your message but don‘t be surprised if people around you get upset.