4 ms·

I am not all that knowledgeable about rust but I think that if you really want to decrease the numbers of system calls when interacting with files, writev and r

I am not all that knowledgeable about rust but I think that if you really want to decrease the numbers of system calls when interacting with files, writev and readv should be considered. I wonder if BuffWriter/Reader could be made to use it, as it would have the same "atomic" behaviour and would forgo the need to concat strings. Still, the char*s could become invalidated once the system call is made.

trasz 5y ago
This solves the “spatial” part, while the actual problem is the “temporal” part: it’s not about many writes at once, it’s about many writes in different parts of code, spaced apart in time.
vlovich123 5y ago
Typically they are orthogonal concepts. In Rust you could probably do it if you’re accepting ownership of the string but if I recall correctly write takes a slice so the contents are not guaranteed to be valid after the write call (eg the user might reuse the buffer for the next write). It might be interesting to have a version of a Writer trait that handed ownership over in addition to supporting slices and handled things under the hood for you. The API may be too complex to bother with. Buffered I/O is typically a fine middle ground. When you start needing writev you’re changing the I/O model more drastically.
zRedShift 5y ago
FWIW, the Read/Write traits have a vectored read/write method, which File implements using readv/writev on supporting Unix platforms: https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#77 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#77 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#144 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#144 Buf(Reader|Writer) are vectored I/O aware, bypassing the internal buffer whenever the total length of the provided buffers is larger: https://doc.rust-lang.org/src/std/io/buffered/bufreader.rs.html#292 https://doc.rust-lang.org/src/std/io/buffered/bufreader.rs.h... https://doc.rust-lang.org/src/std/io/buffered/bufwriter.rs.html#570 https://doc.rust-lang.org/src/std/io/buffered/bufwriter.rs.h...

trasz 5y ago

This solves the “spatial” part, while the actual problem is the “temporal” part: it’s not about many writes at once, it’s about many writes in different parts of code, spaced apart in time.

vlovich123 5y ago

Typically they are orthogonal concepts. In Rust you could probably do it if you’re accepting ownership of the string but if I recall correctly write takes a slice so the contents are not guaranteed to be valid after the write call (eg the user might reuse the buffer for the next write). It might be interesting to have a version of a Writer trait that handed ownership over in addition to supporting slices and handled things under the hood for you. The API may be too complex to bother with. Buffered I/O is typically a fine middle ground. When you start needing writev you’re changing the I/O model more drastically.

zRedShift 5y ago

FWIW, the Read/Write traits have a vectored read/write method, which File implements using readv/writev on supporting Unix platforms: https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#77 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#77 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#144 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#144 Buf(Reader|Writer) are vectored I/O aware, bypassing the internal buffer whenever the total length of the provided buffers is larger: https://doc.rust-lang.org/src/std/io/buffered/bufreader.rs.html#292 https://doc.rust-lang.org/src/std/io/buffered/bufreader.rs.h... https://doc.rust-lang.org/src/std/io/buffered/bufwriter.rs.html#570 https://doc.rust-lang.org/src/std/io/buffered/bufwriter.rs.h...