4 ms·

Part of OPs point is 'most clients' do not have an ideal congestionless/lossless network between them and, well, anything.

fragmede 4y ago
Most clients that OP deals with, anyway. If your code runs exclusively in a data center, like the kind I suspect Google has, then the situation is probably reversed.
- vitus 4y ago
  If you run all of your code in one datacenter, and it never talks to the outside world, sure. That is a fairly rare usage pattern for production systems at Google, though. Just like anyone else, we have packet drops and congestion within our backbone. We like to tell ourselves that the above is less frequent in our network than the wider internet, but it still exists.
  - oefrha 4y ago
    If your DC-DC links are regularly as noisy as shitty apartment WiFi routers competing for air time on a narrow band, fix your DC links.
- mmis1000 4y ago
  Consider the rising of mobile device. The devices that don't have a good internet is probably everywhere now. It's no longer like 10 years ago. You either have good internet or don't have internet. The devices that have shitty network grow a lot compare to the past.
  - Karrot_Kream 4y ago
    Almost every application I've written atop a TCP socket batches up writes into a buffer and then flushes out the buffer. I'd be curious to see how often this doesn't happen.
    - mmis1000 4y ago
      Are you replying to the correct people? I think I never mention how you should write a program. I only say that assume user have a good internet connection is a naive idea nowadays. (The gta 5 is the worst example in my opinion, lost of a few udp packets and your whole game exit to main menu. How the f**k the dev assume udp packets never lost?)
      - Karrot_Kream 4y ago
        What I mean to say is that, whether or not your mobile device has bad internet or not shouldn't matter. Most applications are buffering their reads and writes. This makes TCP_NODELAY a non-issue Most importantly buffering doesn't spend a whole bunch of CPU time context switching into the kernel. Even if you are taking advantage of Nagle's, every call to write is a syscall, which calls into the kernel to perform the write. On a mobile device this would tank your battery. This is the main reason writes are buffered in applications.
        vidarh 4y ago
        This is basically the first thing I check if diagnosing performance issues with network apps. Most probably are buffering now, but surprisingly many don't. MySQLs client library for years didn't for example (it's probably been fixed for a decade or more at this point).
jrockway 4y ago
Why does a congestionless network matter here? Nagle's algorithm aggregates writes together in order to fill up a packet. But you can just do that yourself, and then you're not surprised. I find it very rare that anyone is accidentally sending partially-filled packets; they have some data and they want it to be sent now, and are instead surprised by the fact that it doesn't get sent now because their data doesn't happen to be too large to fit in a single packet. Nobody is reading a file a byte at a time and then passing that 1 byte buffer to Write on a socket. (Except... git-lfs I guess?) Nagle's algorithm is super weird as it's saying "I'm sure the programmer did this wrong, here, let me fix it." Then the 99.99% of the time when you're not doing it wrong, the latency it introduces is too high for anything realtime. Kind of a weird tradeoff, but I'm sure it made sense to quickly fix broken telnet clients at the time.
- throwawaylinux 4y ago
  Yeah it seems crazy to have that kind of hack in the entire network stack and on by default just because some interactive remote terminal clients didn't handle that behavior themselves.
- iforgotpassword 4y ago
  Often times when people want to send five structs, they just call send five times. I find delayed acks a lot more weird compared to nagle.
  - another2another 4y ago
    In those cases it would be better to call writev() which was designed to coalesce multiple buffers into one write call. How it sends the data is however up to the implementation, and whether it delays the last send if the TCP buffer isn't entitrely full I'm not sure - but it doesn't make sense to do so, so I would guess not. https://linux.die.net/man/2/writev https://linux.die.net/man/2/writev
- tashbarg 4y ago
  It makes perfect sense if you consider the right abstraction. TCP connections are streams. There are no packets on that abstraction level. You’re not supposed to care about packets. You’re not supposed to know how large a packet even is. The default is an efficient stream of bytes that has some trade-off to latency. If you care about latency, then you can set a flag.

fragmede 4y ago

Most clients that OP deals with, anyway. If your code runs exclusively in a data center, like the kind I suspect Google has, then the situation is probably reversed.

vitus 4y ago
If you run all of your code in one datacenter, and it never talks to the outside world, sure. That is a fairly rare usage pattern for production systems at Google, though. Just like anyone else, we have packet drops and congestion within our backbone. We like to tell ourselves that the above is less frequent in our network than the wider internet, but it still exists.
- oefrha 4y ago
  If your DC-DC links are regularly as noisy as shitty apartment WiFi routers competing for air time on a narrow band, fix your DC links.
mmis1000 4y ago
Consider the rising of mobile device. The devices that don't have a good internet is probably everywhere now. It's no longer like 10 years ago. You either have good internet or don't have internet. The devices that have shitty network grow a lot compare to the past.
- Karrot_Kream 4y ago
  Almost every application I've written atop a TCP socket batches up writes into a buffer and then flushes out the buffer. I'd be curious to see how often this doesn't happen.
  - mmis1000 4y ago
    Are you replying to the correct people? I think I never mention how you should write a program. I only say that assume user have a good internet connection is a naive idea nowadays. (The gta 5 is the worst example in my opinion, lost of a few udp packets and your whole game exit to main menu. How the f**k the dev assume udp packets never lost?)
    - Karrot_Kream 4y ago
      What I mean to say is that, whether or not your mobile device has bad internet or not shouldn't matter. Most applications are buffering their reads and writes. This makes TCP_NODELAY a non-issue Most importantly buffering doesn't spend a whole bunch of CPU time context switching into the kernel. Even if you are taking advantage of Nagle's, every call to write is a syscall, which calls into the kernel to perform the write. On a mobile device this would tank your battery. This is the main reason writes are buffered in applications.
      - vidarh 4y ago
        This is basically the first thing I check if diagnosing performance issues with network apps. Most probably are buffering now, but surprisingly many don't. MySQLs client library for years didn't for example (it's probably been fixed for a decade or more at this point).

jrockway 4y ago

Why does a congestionless network matter here? Nagle's algorithm aggregates writes together in order to fill up a packet. But you can just do that yourself, and then you're not surprised. I find it very rare that anyone is accidentally sending partially-filled packets; they have some data and they want it to be sent now, and are instead surprised by the fact that it doesn't get sent now because their data doesn't happen to be too large to fit in a single packet. Nobody is reading a file a byte at a time and then passing that 1 byte buffer to Write on a socket. (Except... git-lfs I guess?) Nagle's algorithm is super weird as it's saying "I'm sure the programmer did this wrong, here, let me fix it." Then the 99.99% of the time when you're not doing it wrong, the latency it introduces is too high for anything realtime. Kind of a weird tradeoff, but I'm sure it made sense to quickly fix broken telnet clients at the time.

throwawaylinux 4y ago
Yeah it seems crazy to have that kind of hack in the entire network stack and on by default just because some interactive remote terminal clients didn't handle that behavior themselves.
iforgotpassword 4y ago
Often times when people want to send five structs, they just call send five times. I find delayed acks a lot more weird compared to nagle.
- another2another 4y ago
  In those cases it would be better to call writev() which was designed to coalesce multiple buffers into one write call. How it sends the data is however up to the implementation, and whether it delays the last send if the TCP buffer isn't entitrely full I'm not sure - but it doesn't make sense to do so, so I would guess not. https://linux.die.net/man/2/writev https://linux.die.net/man/2/writev
tashbarg 4y ago
It makes perfect sense if you consider the right abstraction. TCP connections are streams. There are no packets on that abstraction level. You’re not supposed to care about packets. You’re not supposed to know how large a packet even is. The default is an efficient stream of bytes that has some trade-off to latency. If you care about latency, then you can set a flag.