4 ms·
> I'd imagine most Go software is deployed in datacentres where the network is high quality The problem is that those datacenters are plugged into the Internet
by sneak 4y ago
> I'd imagine most Go software is deployed in datacentres where the network is high quality
The problem is that those datacenters are plugged into the Internet, where the network is not always high quality. TFA mentions the Caddy webserver - this is "datacenter" software designed to talk to diverse clients all over the internet. The stdlib should not tamper with the OS defaults unless the OS defaults are pathological.
- tptacek 4y agoThat doesn't make much sense. There are all sorts of socket and file descriptor parameters with defaults that are situational; NDELAY is one of them, as is buffer size, nonblockingness, address reuse, &c. Maybe disabling Nagle is a bad default, maybe it isn't, but the appeal to "OS defaults" is a red herring.
- Teckla 4y agoIn my opinion, the Principle of Least Surprise applies here. Go is defaulting to surprising (unexpected) behavior.
- pclmulqdq 4y agoIn my experience, you usually don't want to be Nagling in code that lives in a datacenter. Go's default is likely set up around that idea.
- giovannibajo1 4y agoI think also “least surprise” depends on your background. In Go, also files don’t buffer by default, contrary to many languages including C. If you call Write() 100 times, you run exactly 100 syscalls. Intermediate Go programmers learn this and that they must explicitly manage buffering (eg: via bufio). I don’t think it’s wrong that sockets follow the same design. It gives me less surprise.
- incompatible 4y agoI think my C is getting rusty, but "write" operates on a file descriptor, doesn't it? It's unbuffered. The buffered versions are things like printf and puts.
- blibble 4y agothat Write() doesn't call fsync() though, does it? so there's no buffering going on in the application, but the bytes almost certainly don't hit the disk before Write() returns they've just been staged into an OS buffer, with the OS promising to write them out to the disk at a later time (probably, maybe...? hopefully!) which is exactly the same as a regular TCP socket (with Nagle disabled, i.e. the default, non Go way)
- Ferret7446 4y agoFor userland programming, what matters is the syscall level, as that is expensive (and also the API you have for the kernel). Whether the kernel then does internal buffering is irrelevant and uncontrollable beyond any other syscalls which may or may not be implemented (maybe you're running on a custom kernel that doesn't buffer disk writes?). One write == one syscall, easy. If you want buffering, you add it.
- blibble 4y ago> For userland programming, what matters is the syscall level, as that is expensive which is why pretty much every programming language buffers file output by default even C (other than Go, obviously) > Whether the kernel then does internal buffering is irrelevant everyone that's attempted to write reliable software that cares about what ends up on disk, or the other side of the socket will disagree
- Xorlev 4y agoHave you ever had to chase down strange latency issues? Arguably, this behavior is the least surprising for Go's typical deployment environment.
- lanstin 4y agoTwice I have run into this behavior having known and forgotten it. Chatty non http protocols with a few small messages doing auth or whatever before bulk data flow. Pissed me off and surprised me. Now I make sure my defaults for any framework I am using are no delay, and I make sure to plug my computing device into Ethernet whenever possible.
- squeaky-clean 4y agoOnly if you know that Nagle's algorithm exists and is used everywhere. For anyone with networking experience it's unexpected, but I still remember learning about Nagle's algorithm when trying to fix latency on a game server I was hosting as a teen. That was surprising behavior to me at the time.
- jrockway 4y agoArticles with titles like "How I spent 3 weeks discovering that Nagle's Algorithm exists" are a HN staple. Turning off Nagle follows the principle of least surprise. This article is the first time anyone has ever written about being surprised by Nagle's Algorithm being off.
- Terretta 4y agoYou guys had this convo before, in 2015, on what the interaction of the two settings should be doing: https://news.ycombinator.com/item?id=34180239 https://news.ycombinator.com/item?id=34180239
- fragmede 4y agoI think you meant to link to https://news.ycombinator.com/item?id=10608356 https://news.ycombinator.com/item?id=10608356
- mdolah 4y agoLove this historical find. How did you find this relevant conversation? Was this something with HN search?
- subroutine 4y agoYou haven't committed to memory every hn comment from Ptacek over the last decade?
- drpixie 4y agoAlso, for small packets, disabling consolidation means adding LOTS of packet overhead. You're not sending 1 million * 50 bytes of data, you're sending 1 million * (50 bytes of data + about 80 bytes of TCP+ethernet header). Disabling Nagle makes sense for tiny request/replys (like RPC calls) but it's counterproductive for bulk transfers. I'm not the only one who don't like the thought of a standard library quietly changing standard system behaviour ... so know I have to know the standard routines and their behaviour AND I have to know which platforms/libraries silently reverse things :(
- boring_twenties 4y agoBulk transfer applications should just use larger buffers
- josefx 4y agoWouldn't that still end up with sub optimal network messages unless those large buffers are an exact multiple of the MTU on the network?
- boring_twenties 4y agoHm, I'm not 100% sure about this. If your first buffer is big enough, your next write should be issued before the OS has managed to write it all.