8 ms·

In my opinion, I think it's correct to be disabled by default. I think Nagle's algorithm does more harm than good if you're unaware of it. I've seen people wri

In my opinion, I think it's correct to be disabled by default. I think Nagle's algorithm does more harm than good if you're unaware of it. I've seen people writing C# applications and wondering why stuff is taking 200ms. Some people don't even realise it's Nagle's algorithm (edit: interacting with Delayed ACKs) and think it's network issues or a performance problem they're introduced. I'd imagine most Go software is deployed in datacentres where the network is high quality and it doesn't really matter too much. Fast data transfer is probably preferred. I think Nagle's algorithm should be an optimisation you can optionally enable (which you can) to more efficiently use the network at the expense of latency. Being more "raw" seems like the sensible default to me.

zamalek 4y ago
From the bottom of the article: > Most people turn to TCP_NODELAY because of the “200ms” latency you might incur on a connection. Fun fact, this doesn’t come from Nagle’s algorithm, but from Delayed ACKs or Corking. Yet people turn off Nagle’s algorithm … :sigh:
- Philip-J-Fry 4y ago
  Yeah but Nagle's Algorithm and Delayed ACKs interaction is what causes the 200ms. Servers tend to enable Nagle's algorithm by default. Clients tend to enabled Delayed ACK by default, and then you get this horrible interaction all because they're trying to be more efficient but stalling eachother. I think Go's behavior is the right default because you can't control every server. But if Nagle's was off by default on servers then we wouldn't need to disabled Delayed ACKs on clients.
  - pclmulqdq 4y ago
    Clients having delayed acks has a very good reason: those ACKs cost data, and clients tend to have much higher download bandwidth than upload bandwidth. Really, clients should probably be delaying acks and nagling packets, while servers should probably be doing neither.
    - charleslmunger 4y ago
      Clients should not be nagling unless the connection is emitting tiny bytes at high frequency. But that's a very odd thing to do, and in most/all cases there's some reasonable buffering occuring higher up in the stack that the nagle's algorithm will only add overhead to. Making things worse are tcp-within-tcp things like http/2. Nagle's algorithm works great for things like telnet but should not be applied as a default to general purpose networking.
      - avianlyric 4y ago
        Why would Nagles algorithm add delay to “reasonable buffering up the stack”? Assuming that buffering is resulting in writes to the TCP stack greater than the packet size, Nagles algorithm won’t add any delay. The only place where Nagles algorithm adds delay is when your doing many tiny writes to a socket, which is exactly the situation you believe Nagles should be applied to.
    - avianlyric 4y ago
      The size of an ACK is minuscule (40 bytes) compared to any reasonable packet size (usually around 1400 bytes). In most client situations where you have high down bandwidth, but limited up, that suggests the vast majority of data is heading towards the client, and client isn’t sending much outbound. In which case your client may end up delaying every ACK to maximum timeout, simply because it doesn’t often send reply data in response to a server response. HTTP is clear example of this. Client issues a request to the server, server replies. Client accepts rely, but never sends any further data to the server. In this case, delaying the client ACK is just a waste of time.
  - Terretta 4y ago
    Part of OPs point is 'most clients' do not have an ideal congestionless/lossless network between them and, well, anything.
    - fragmede 4y ago
      Most clients that OP deals with, anyway. If your code runs exclusively in a data center, like the kind I suspect Google has, then the situation is probably reversed.
      - vitus 4y ago
        If you run all of your code in one datacenter, and it never talks to the outside world, sure. That is a fairly rare usage pattern for production systems at Google, though. Just like anyone else, we have packet drops and congestion within our backbone. We like to tell ourselves that the above is less frequent in our network than the wider internet, but it still exists.
        oefrha 4y ago
        If your DC-DC links are regularly as noisy as shitty apartment WiFi routers competing for air time on a narrow band, fix your DC links.
      - mmis1000 4y ago
        Consider the rising of mobile device. The devices that don't have a good internet is probably everywhere now. It's no longer like 10 years ago. You either have good internet or don't have internet. The devices that have shitty network grow a lot compare to the past.
        Karrot_Kream 4y ago
        Almost every application I've written atop a TCP socket batches up writes into a buffer and then flushes out the buffer. I'd be curious to see how often this doesn't happen.
  - dccoolgai 4y ago
    "Be conservative in what you send and liberal in what you accept" I would cite Postels Law: Nagle's is the "conservative send" side. An ACK is a signal of acceptance, and should be issued more liberally (even though it's also sent, I guess).
sneak 4y ago
> I'd imagine most Go software is deployed in datacentres where the network is high quality The problem is that those datacenters are plugged into the Internet, where the network is not always high quality. TFA mentions the Caddy webserver - this is "datacenter" software designed to talk to diverse clients all over the internet. The stdlib should not tamper with the OS defaults unless the OS defaults are pathological.
- tptacek 4y ago
  That doesn't make much sense. There are all sorts of socket and file descriptor parameters with defaults that are situational; NDELAY is one of them, as is buffer size, nonblockingness, address reuse, &c. Maybe disabling Nagle is a bad default, maybe it isn't, but the appeal to "OS defaults" is a red herring.
  - Teckla 4y ago
    In my opinion, the Principle of Least Surprise applies here. Go is defaulting to surprising (unexpected) behavior.
    - pclmulqdq 4y ago
      In my experience, you usually don't want to be Nagling in code that lives in a datacenter. Go's default is likely set up around that idea.
    - giovannibajo1 4y ago
      I think also “least surprise” depends on your background. In Go, also files don’t buffer by default, contrary to many languages including C. If you call Write() 100 times, you run exactly 100 syscalls. Intermediate Go programmers learn this and that they must explicitly manage buffering (eg: via bufio). I don’t think it’s wrong that sockets follow the same design. It gives me less surprise.
      - incompatible 4y ago
Animats 4y ago
The basic problem, as I've written before[1][2], is that, after I put in Nagle's algorithm, Berkeley put in delayed ACKs. Delayed ACKs delay sending an empty ACK packet for a short, fixed period based on human typing speed, maybe 100ms. This was a hack Berkeley put in to handle large numbers of dumb terminals going in to time-sharing computers using terminal to Ethernet concentrators. Without delayed ACKs, each keystroke sent a datagram with one payload byte, and got a datagram back with no payload, just an ACK, followed shortly thereafter by a datagram with one echoed character. So they got a 30% load reduction for their TELNET application. Both of those algorithms should never be on at the same time. But they usually are. Linux has a socket option, TCP_QUICKACK, to turn off delayed ACKs. But it's very strange. The documentation is kind of vague, but apparently you have to re-enable it regularly.[3] Sigh. [1] https://news.ycombinator.com/item?id=10608356 https://news.ycombinator.com/item?id=10608356 [2] https://developers.slashdot.org/comments.pl?cid=14515105&sid=174457 https://developers.slashdot.org/comments.pl?cid=14515105&sid... [3] https://stackoverflow.com/questions/46587168/when-during-the-socket-lifetime-should-i-set-the-tcp-quickack-option https://stackoverflow.com/questions/46587168/when-during-the...
- rjbwork 4y ago
  Gotta love HN. The man himself shows up to explain.
  - thiht 4y ago
    Imagine being on a math forum discussing Fermat’s theorem and the guy shows up. This is such a cool aspect of CS being a young field: influent people are still alive!
    - Beltalowda 4y ago
      Andrew Wiles showing up would probably be the next best thing.
      - sillysaurusx 4y ago
        Russ also showed up. https://news.ycombinator.com/item?id=34181846 https://news.ycombinator.com/item?id=34181846 Readers might also enjoy his writeup on how google code search worked. https://swtch.com/~rsc/regexp/regexp4.html https://swtch.com/~rsc/regexp/regexp4.html Just discovered it.
silisili 4y ago
Agreed. The post should be titled 'Go enables TCP_NODELAY by default', and a body may or may not even be needed. It's documented, even https://pkg.go.dev/net#TCPConn.SetNoDelay https://pkg.go.dev/net#TCPConn.SetNoDelay To know why would be interesting, I guess. But you should be buffering writes anyways in most cases. And if you refuse to do that, just turn it back off on the socket. This is on the code author.
iofiiiiiiiii 4y ago
> I've seen people writing C# applications and wondering why stuff is taking 200ms I observe that in the most recent generation of its HTTP client (SocketsHttpHandler), .NET also sets NoDelay by default. https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs#L1595 https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...
- kcartlidge 4y ago
  TIL - thanks!

zamalek 4y ago

From the bottom of the article: > Most people turn to TCP_NODELAY because of the “200ms” latency you might incur on a connection. Fun fact, this doesn’t come from Nagle’s algorithm, but from Delayed ACKs or Corking. Yet people turn off Nagle’s algorithm … :sigh:

Philip-J-Fry 4y ago
Yeah but Nagle's Algorithm and Delayed ACKs interaction is what causes the 200ms. Servers tend to enable Nagle's algorithm by default. Clients tend to enabled Delayed ACK by default, and then you get this horrible interaction all because they're trying to be more efficient but stalling eachother. I think Go's behavior is the right default because you can't control every server. But if Nagle's was off by default on servers then we wouldn't need to disabled Delayed ACKs on clients.
- pclmulqdq 4y ago
  Clients having delayed acks has a very good reason: those ACKs cost data, and clients tend to have much higher download bandwidth than upload bandwidth. Really, clients should probably be delaying acks and nagling packets, while servers should probably be doing neither.
  - charleslmunger 4y ago
    Clients should not be nagling unless the connection is emitting tiny bytes at high frequency. But that's a very odd thing to do, and in most/all cases there's some reasonable buffering occuring higher up in the stack that the nagle's algorithm will only add overhead to. Making things worse are tcp-within-tcp things like http/2. Nagle's algorithm works great for things like telnet but should not be applied as a default to general purpose networking.
    - avianlyric 4y ago
      Why would Nagles algorithm add delay to “reasonable buffering up the stack”? Assuming that buffering is resulting in writes to the TCP stack greater than the packet size, Nagles algorithm won’t add any delay. The only place where Nagles algorithm adds delay is when your doing many tiny writes to a socket, which is exactly the situation you believe Nagles should be applied to.
  - avianlyric 4y ago
    The size of an ACK is minuscule (40 bytes) compared to any reasonable packet size (usually around 1400 bytes). In most client situations where you have high down bandwidth, but limited up, that suggests the vast majority of data is heading towards the client, and client isn’t sending much outbound. In which case your client may end up delaying every ACK to maximum timeout, simply because it doesn’t often send reply data in response to a server response. HTTP is clear example of this. Client issues a request to the server, server replies. Client accepts rely, but never sends any further data to the server. In this case, delaying the client ACK is just a waste of time.
- Terretta 4y ago
  Part of OPs point is 'most clients' do not have an ideal congestionless/lossless network between them and, well, anything.
  - fragmede 4y ago
    Most clients that OP deals with, anyway. If your code runs exclusively in a data center, like the kind I suspect Google has, then the situation is probably reversed.
    - vitus 4y ago
      If you run all of your code in one datacenter, and it never talks to the outside world, sure. That is a fairly rare usage pattern for production systems at Google, though. Just like anyone else, we have packet drops and congestion within our backbone. We like to tell ourselves that the above is less frequent in our network than the wider internet, but it still exists.
      - oefrha 4y ago
        If your DC-DC links are regularly as noisy as shitty apartment WiFi routers competing for air time on a narrow band, fix your DC links.
    - mmis1000 4y ago
      Consider the rising of mobile device. The devices that don't have a good internet is probably everywhere now. It's no longer like 10 years ago. You either have good internet or don't have internet. The devices that have shitty network grow a lot compare to the past.
      - Karrot_Kream 4y ago
        Almost every application I've written atop a TCP socket batches up writes into a buffer and then flushes out the buffer. I'd be curious to see how often this doesn't happen.
- dccoolgai 4y ago
  "Be conservative in what you send and liberal in what you accept" I would cite Postels Law: Nagle's is the "conservative send" side. An ACK is a signal of acceptance, and should be issued more liberally (even though it's also sent, I guess).

sneak 4y ago

> I'd imagine most Go software is deployed in datacentres where the network is high quality The problem is that those datacenters are plugged into the Internet, where the network is not always high quality. TFA mentions the Caddy webserver - this is "datacenter" software designed to talk to diverse clients all over the internet. The stdlib should not tamper with the OS defaults unless the OS defaults are pathological.

tptacek 4y ago
That doesn't make much sense. There are all sorts of socket and file descriptor parameters with defaults that are situational; NDELAY is one of them, as is buffer size, nonblockingness, address reuse, &c. Maybe disabling Nagle is a bad default, maybe it isn't, but the appeal to "OS defaults" is a red herring.
- Teckla 4y ago
  In my opinion, the Principle of Least Surprise applies here. Go is defaulting to surprising (unexpected) behavior.
  - pclmulqdq 4y ago
    In my experience, you usually don't want to be Nagling in code that lives in a datacenter. Go's default is likely set up around that idea.
  - giovannibajo1 4y ago
    I think also “least surprise” depends on your background. In Go, also files don’t buffer by default, contrary to many languages including C. If you call Write() 100 times, you run exactly 100 syscalls. Intermediate Go programmers learn this and that they must explicitly manage buffering (eg: via bufio). I don’t think it’s wrong that sockets follow the same design. It gives me less surprise.
    - incompatible 4y ago

Animats 4y ago

The basic problem, as I've written before[1][2], is that, after I put in Nagle's algorithm, Berkeley put in delayed ACKs. Delayed ACKs delay sending an empty ACK packet for a short, fixed period based on human typing speed, maybe 100ms. This was a hack Berkeley put in to handle large numbers of dumb terminals going in to time-sharing computers using terminal to Ethernet concentrators. Without delayed ACKs, each keystroke sent a datagram with one payload byte, and got a datagram back with no payload, just an ACK, followed shortly thereafter by a datagram with one echoed character. So they got a 30% load reduction for their TELNET application. Both of those algorithms should never be on at the same time. But they usually are. Linux has a socket option, TCP_QUICKACK, to turn off delayed ACKs. But it's very strange. The documentation is kind of vague, but apparently you have to re-enable it regularly.[3] Sigh. [1] https://news.ycombinator.com/item?id=10608356 https://news.ycombinator.com/item?id=10608356 [2] https://developers.slashdot.org/comments.pl?cid=14515105&sid=174457 https://developers.slashdot.org/comments.pl?cid=14515105&sid... [3] https://stackoverflow.com/questions/46587168/when-during-the-socket-lifetime-should-i-set-the-tcp-quickack-option https://stackoverflow.com/questions/46587168/when-during-the...

rjbwork 4y ago
Gotta love HN. The man himself shows up to explain.
- thiht 4y ago
  Imagine being on a math forum discussing Fermat’s theorem and the guy shows up. This is such a cool aspect of CS being a young field: influent people are still alive!
  - Beltalowda 4y ago
    Andrew Wiles showing up would probably be the next best thing.
    - sillysaurusx 4y ago
      Russ also showed up. https://news.ycombinator.com/item?id=34181846 https://news.ycombinator.com/item?id=34181846 Readers might also enjoy his writeup on how google code search worked. https://swtch.com/~rsc/regexp/regexp4.html https://swtch.com/~rsc/regexp/regexp4.html Just discovered it.

silisili 4y ago

Agreed. The post should be titled 'Go enables TCP_NODELAY by default', and a body may or may not even be needed. It's documented, even https://pkg.go.dev/net#TCPConn.SetNoDelay https://pkg.go.dev/net#TCPConn.SetNoDelay To know why would be interesting, I guess. But you should be buffering writes anyways in most cases. And if you refuse to do that, just turn it back off on the socket. This is on the code author.

iofiiiiiiiii 4y ago

> I've seen people writing C# applications and wondering why stuff is taking 200ms I observe that in the most recent generation of its HTTP client (SocketsHttpHandler), .NET also sets NoDelay by default. https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs#L1595 https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...

kcartlidge 4y ago
TIL - thanks!