5 ms·
Zero-copy in Go: sendfile, splice, and the cost of io.Copy
- sanxiyn 3mo agoA good reminder. It is surprising first time you encounter it. Same for Rust. As https://doc.rust-lang.org/stable/std/io/fn.copy.html https://doc.rust-lang.org/stable/std/io/fn.copy.html says, std::io::copy can use copy_file_range(2), sendfile(2), or splice(2).
- mike_hock 3mo agoZero-Copy in Go: Why magic is an antipattern, and: performance is observable behavior.
- sanxiyn 3mo agoWhat would you prefer? I do think it is criminal this is not documented (https://pkg.go.dev/io#Copy https://pkg.go.dev/io#Copy), but I think io.Copy is fine as an API.
- throwrioawfo 3mo agoUgh, AI slop writing.
- drivebyhooting 3mo agoDefinitely written by codex.
- badc0ffee 3mo agoThe lack of apostrophe in "dont" makes me think at least part of it was written manually. It's also a totally readable article, really.
- joaohaas 3mo agoInteresting premise for a post, but I had to stop midway due to the AI slop writing adding meaningless information.
- inigyou 3mo agoThis is almost like the expression problem. Copy is a new operation, and you introduced a new type, thus creating a new grid cell nobody from either side could have reasonably known about - except for the fact Copy is in the standard library so you could have known about it but not done anything.
- drivebyhooting 3mo agoHow is the byte counting reader supposed to work in user space without putting the buffer in user space? The article claims there is a way but I want to see what is meant by counting bytes in that case.
- flakes 3mo agosendfile(2) and io.ReaderFrom both return the number of bytes transmitted. The issue is that users are unaware of (or forget about) the optional interface upgrades and fail to define all the methods required for interface upgrades on their wrapper structs. You can definitely make a counting reader with a minimal performance loss, but the proper solution is less obvious than it ideally should be.
- drivebyhooting 3mo agoIsn’t this a symptom of structural/duck typing where interfaces are not declared? In Java for all its faults this wouldn’t happen because you’d be forced to implement all the interfaces.
- flakes 3mo ago> Isn’t this a symptom of structural/duck typing where interfaces are not declared Yes, essentially duck typing. See https://github.com/golang/go/blob/65504872cbca64d77f45828409f5c8c59bf23d54/src/io/io.go#L407-L416 https://github.com/golang/go/blob/65504872cbca64d77f45828409... The logic uses a type assertion to safely verify if the value backing the provided io.Reader interface also implements the io.ReaderFrom interface. If it matches, then it will use the more efficient implementation if rf, ok := dst.(ReaderFrom); ok { return rf.ReadFrom(src) } > In Java for all its faults this wouldn’t happen because you’d be forced to implement all the interfaces. I don't think I would go that far. In Java, many libraries make heavy use of the `instanceof` keyword, which is more or less the same as Go type assertions.
- pjmlp 3mo ago
- sly010 3mo agoBeware, there are versions of go where sendfile is broken and only sends the first 4k of a file on macos.
- majewsky 3mo agoIf this is a thing you need to beware of, you should more importantly beware of using a version of Go that's been outdated for over 1.5 years and EOL for nearly a year. Context: https://github.com/golang/go/issues/70000 https://github.com/golang/go/issues/70000 fixed in 1.23.3 (released 2024-11-06), EOL for 1.23.x was 2025-08-06