8 ms·

Unbuffered I/O Can Make Your Rust Programs Much Slower

45 points by akbarnama 5y ago

ericbarrett 5y ago
I started a Rust project several months ago and quickly became frustrated with the state of basic terminal libraries. None of them seemed to implement the buffering and minimal-refresh of ncurses, so I was forced to rely on one of the many shifting ncurses binding libraries and all the unsafeness that comes with that (or reimplement my own—way out of scope for the little toy I was building). What's the state of the art for raw terminal libraries (not TUI helpers) in Rust nowadays?
- grenoire 5y ago
  Guh, I feel you on this one. I was shopping around for a buffering wrapper but soon realise that I either had to get basically a terminal Qt clone or raw bindings, nothing inbetween...
otterley 5y ago
Unbuffered I/O is slower than buffered I/O regardless of language. It’s not just a Rust phenomenon. Making syscalls like write() every time you want to send even a character of output down a stream is much more latent than just sticking bytes in a buffer. In general, the only time anyone should use unbuffered output is when there’s a real person consuming it - for example, when you’re prompting a user for feedback or input.
- yongjik 5y ago
  > In general, the only time anyone should use unbuffered output is when there’s a real person consuming it - for example, when you’re prompting a user for feedback or input. I'm in the camp that logging should be always unbuffered. If you buffer logs, and your process dies for any reason, you lose your log exactly when you really need it to know what happened.
  - otterley 5y ago
    Buffered I/O is usually not lost when a process exits. Usually buffers are flushed automatically upon exit when the file descriptors are closed. And if a program truly crashes and leaves a core behind, the unflushed buffers should be found in the core file.
    - yongjik 5y ago
      If the process exits in an orderly fashion then, yes, it will flush logs, and also do a bunch of other nice things. The problem arises when it dies in a not-so-nice fashion. (Also, not every environment is running with unlimited core size.)
- silisili 5y ago
  > In general, the only time anyone should use unbuffered output is when there’s a real person consuming it I think this might be a little -too- general, as it's pretty workload dependent. I mean, if you just want to write a blob of bytes once, just write the blob IMO. I don't think using a buffer in those cases ever really hurts, but sometimes makes what should be simple a bit more complicated than necessary.
  - molyss 5y ago
    Even for stuff as simple as handling LOBs, I buffer on both read and writes. Some DB APIs really do “what you tell them to”, even if that means being unbearably slow when unbuffered The worst I’ve seen was actually streaming the LOB on getLength. Since there’s no JDBC API for lob.getContentAsArray(), you need to call lob.getContent(1, lob.length()) (notice that you’d start at index 1…), which by default would stream the LOB twice under the scene!
  - harles 5y ago
    Definitely too general of a statement. I previously worked on real-time mapping and the only way to get the necessary write performance was to go unbuffered and manually split off a block at a time to write. The same was true of reading - it was faster and made sure the # of syscalls in any frame was predictable. As a side note, the real issue here is that unbuffered is the default. Other languages and libraries I’m aware of make it difficult to get in such a mode.
- brundolf 5y ago
  Lots of people are writing Rust code who aren't previously versed in the nuances of optimizing bare-metal code. They're (reasonably) expecting amazing performance, because Rust tends to give you that by default compared with higher-level languages. But repeated calls to println!() is one of the biggest and easiest-to-make performance mistakes for those newcomers, so I think it's fair to call it out. With that said, the OP could definitely have done a better job making it clear that this problem isn't specific to Rust
  - not2b 5y ago
    Developers from other languages, like C, C++, or even Python, would not have expected this particular problem, because it wouldn't have occurred to them that the thing they get from doing a file-open call and that the JSON reader library accepts is a raw file descriptor (for C programmers, the result of 'open' rather than 'fopen') that does unbuffered I/O. In other languages libraries are oriented around buffered I/O structures (a FILE or an iostream).
    - BoorishBears 5y ago
      To me the biggest reason for the surprise is the signature for 'write' I'm used to a raw stream allowing a few specific operations: - read/write/peek one byte - write a byte array, but you must explicitly specify length and offset So seeing a call that simply takes an entire byte string and writes it gives me "buffered I/O vibes". It's hard to quantify, but with how ubiquitous that pattern is I'd be surprised if there are a lot of people who expect similar.
    - not2b 5y ago
      This had been upvoted and now it is downvoted, even though it is obviously true, as the article demonstrates.
  - oconnor663 5y ago
    For what it's worth, I run into the same nuances frequently in Python. For example I often want to run some_script.py | tee log.txt But by default, Python will turn off line buffering when it detects it's not talking to a tty. So unless you know to call flush, or to add the right env var or interpreter flag, you're not going to see any early output from tee. This can be super confusing for newer programmers.
- loeg 5y ago
  Totally agree. In contrast, for example, FILE objects in C are (mostly) buffered by default. So in that sense, Rust defaults are less sensible for some developers who would be better served by buffering, but aren't aware that they need it. But I also think that Rust's default is totally reasonable and it's just something you have to learn about as a systems programmer.
halayli 5y ago
Unbuffered I/O can make any language you use much slower. It has nothing to do with Rust / language choice.
- xxs 5y ago
  "Rust" sets you for the front page of hackernews, though.
- bcrl 5y ago
  Except for the usage patterns where it's faster because you avoid blowing your CPU's caches as much... The trade offs between different ways of performing i/o are decidedly non-trivial.
  - halayli 5y ago
    Everything you mentioned is irrelevant in the context of unbuffered IO. The majority of time is going to be spent on system calls, context switching, and signal handling / checking, and waiting for hardware interrupts to fire and handling them. Also there is a lot more then mentioning cpu cache as this is very generic term. which part of the cpu cache are you talking about? instruction cache? data cache? And why do you have think they matter in this context?
    - bcrl 5y ago
      My comments are perfectly relevant. Using read() and write() still uses buffers, it's just that those buffers happen to be in the kernel. read() and write() still do the kernel's form of memcpy() internally. Truly unbuffered I/O from the kernel's perspective is direct I/O that DMAs data directly into application provided memory. It is important to note that every time buffered I/O is used, it copies data through buffers, and touching those buffers means the memory used for those buffers being touched clearly has to be pulled into the CPU's L1 data cache. In some cases the extra cache traffic for those buffers degrades performance. Instruction caches are relevant to some workloads, but are clearly not what I was referring to since data doesn't go through the instruction cache. glibc further complicates the matter by using mmap() in some cases to directly map the kernel's buffers for data (the page cache in the context of Linux) into userspace. This avoids some of the downsides of buffered I/O by avoiding one of the copies at the expense of page fault overhead. Sometimes it's better for the application itself to use mmap() directly. Direct use of read() and write() makes sense for some applications, especially when dealing with larger chunks of data. Saying unbuffered I/O is bad is a gross oversimplification of the reality when writing high performance applications. Like everything in programming, it's a good rule of thumb, but there is definitely an asterisk next to it.
dmw_ng 5y ago
Alternative title: Poor Interface Design Makes Most Rust Programs Much Slower That programmers need to consider this stuff on each new project is a failure of the standard library design, it's a problem not even C++ suffers from
- mamcx 5y ago
  No. But is very surprising when you see Rust how high-level is as language yet it target system programming. One major philosophy aspect of Rust is that (almost)everything is explicit and you build abstraction from the bottom.
  - deleted 5y ago
    [deleted]
- trasz 5y ago
  Not even C has this problem - stdio handles this just fine, by deciding whether to buffer or not depending on where the data is going to (tty vs a file).
- smolder 5y ago
  No, it's a design choice, not a 'failure'. Hidden behavior has its own pitfalls, and you can argue this all day, but there are already 100 languages which go the other way and cleverly optimize things behind the scenes every chance they get. I wouldn't mind much if they made IO buffered by default, but I don't mind it's not, either, because it's consistent with the design ethos.
- spoiler 5y ago
  In my experience, the most ergonomic implementation is rarely the most performant one in C++, you usually end up having to use some SFINAE monstrosity or sprinkle constexpr if all over the place. std::optional comes specifically to mind as one of the big offenders here. You've also gotta be careful with unexpected and implicit assignment/copy/move constructors. More than half of all the C++ wisdom one needs in their career is when to avoid stdlib and which part of boost to use. Due to C++s religious devition to 80s compatibility, it becomes a minefield of a Don'ts, with a narrow path of Dos. The fact boost is a billion libraries in one speaks in of itself about the abysmal dependency situation in C++. There's also a buncha dep managers that are at each others throats. Also, C++ people like to poke fun about js devs and npm/leftpad, but forget that they use boost for even more rudimentary string operations. Or they implement the same thing over and over again. Or my favourite: copy+paste the same code in multiple projects, and then have divergent implementations of the same thing with different bugs and/or API surface; bonus points if the thing is part of an RPC implementation. As much as the edgy-dev crowd loves to belittle the very concept of dependency management, at least JS/Ruby/Python ecosystems give a shit about DX and made sure the package managers can be used with the same repositories/sources. But hey, at least C++’s IO is buffered by default. Amazing. I got distracted by my rant above: I actually PREFER the default is unbuffered, because it doesn't violate the principle of least surprise (if we remember that Rust is a systems programming language). The fact you expect the default to be buffered is probably due to exposure to C/++.
lor_louis 5y ago
I am not all that knowledgeable about rust but I think that if you really want to decrease the numbers of system calls when interacting with files, writev and readv should be considered. I wonder if BuffWriter/Reader could be made to use it, as it would have the same "atomic" behaviour and would forgo the need to concat strings. Still, the char*s could become invalidated once the system call is made.
- trasz 5y ago
  This solves the “spatial” part, while the actual problem is the “temporal” part: it’s not about many writes at once, it’s about many writes in different parts of code, spaced apart in time.
- vlovich123 5y ago
  Typically they are orthogonal concepts. In Rust you could probably do it if you’re accepting ownership of the string but if I recall correctly write takes a slice so the contents are not guaranteed to be valid after the write call (eg the user might reuse the buffer for the next write). It might be interesting to have a version of a Writer trait that handed ownership over in addition to supporting slices and handled things under the hood for you. The API may be too complex to bother with. Buffered I/O is typically a fine middle ground. When you start needing writev you’re changing the I/O model more drastically.
- zRedShift 5y ago
  FWIW, the Read/Write traits have a vectored read/write method, which File implements using readv/writev on supporting Unix platforms: https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#77 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#77 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#144 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#144 Buf(Reader|Writer) are vectored I/O aware, bypassing the internal buffer whenever the total length of the provided buffers is larger: https://doc.rust-lang.org/src/std/io/buffered/bufreader.rs.html#292 https://doc.rust-lang.org/src/std/io/buffered/bufreader.rs.h... https://doc.rust-lang.org/src/std/io/buffered/bufwriter.rs.html#570 https://doc.rust-lang.org/src/std/io/buffered/bufwriter.rs.h...
qalmakka 5y ago
I kind of understand why Rust made buffered IO explicit and on-demand instead of going down the C route and making it the default, but still I think there's a significant risk lots of programs are going to be suboptimal due to developer ignorance or the expectation of having buffered IO by default like with stdio and iostreams. It would have been IMHO probably less clean but safer to make the standard IO constructs like File buffered by default (which is what users expect in the vast majority of cases) and provide on the side unbuffered variants. Thankfully modern OSes with caches and stuff somewhat mitigate this thing, but it can make IO on less advanced systems way more problematic than it should be.
- brundolf 5y ago
  I think it would go against Rust's philosophy to have hidden (and inconsistent) behavior, like automatically buffering some things but not others. Generally Rust tries to give users explicit and ergonomic control over those sorts of details.
  - lumost 5y ago
    When developing large scale applications ( e.g. any app that needs third-party libraries ). You are going to have to deal with developers who either 1. Didn't know about a nuance. 2. Didn't care about said nuance. Creating magic incantations to achieve typical expected behavior will result in many libraries doing the unexpected behavior. Which means that the typical code that any end user works with will probably do the unexpected thing.
    - brundolf 5y ago
      It's all relative. I'm very glad that JavaScript doesn't try to make you deal with output buffers, but Rust's goals as a language are different.
      - qalmakka 5y ago
        The problem here is that the uncontested ruler of said system languages, C, has a completely opposite behaviour by default. The C standard clearly states that "When opened, a stream is fully buffered if and only if it can be determined not to refer to an interactive device" (§7.21.5.3.8), which means that generations of developers have learned to always expect buffering when opening files `fopen`. Most higher level languages follow the C library when setting their I/O convention, and this has been the trend for most of the last 40 years. The fact that Rust goes in the opposite direction makes sense from a logical standpoint, but as the article itself shows will almost certainly end up causing performance issues. You can rest assured that people *will* forget to buffer their files because it is something they almost never had to care about in the past.
PaulDavisThe1st 5y ago
Alternative Title: Many Problems That Have Affected Software Development Since Denis Ritchie Roamed the Steppes Continue to Affect Rust Programming
- BoorishBears 5y ago
  I can't remember the last time I came across a modern language where the "easy default" is unbuffered I/O. A beginner learning C++ today will be getting familiar with buffered I/O primitives by the time they write a "Hello World" Java mostly keeps the "useful" access patterns (anything more than reading/writing a single byte or manually passing in byte buffers) in buffered wrappers so even a clueless beginner is inclined to use them. People are missing the forest for the trees here, it's about ergonomics and encouraging people to do the right thing by default. Ironic to see this attitude in a Rust comment section of all places, isn't this the same attitude that leads people to dismiss Rust? "Just write correct C code without UB, the major pitfalls haven't changed in the last 4 decades duh?"
  - PaulDavisThe1st 5y ago
    > I can't remember the last time I came across a modern language where the "easy default" is unbuffered I/O. "Hello World" in C from 1986 uses printf() which is buffered. So yeah, it's been a long, long, long time.
  - kccqzy 5y ago
    It's because there's this new trend of high-performance, bufferless, copy-less technologies like io_uring getting mindshare. Beginners who don't understand why these technologies work then blindly remove buffering, or forget to add buffering in the first place.
    - trasz 5y ago
      It’s not really new, it’s a return to how things were done before Unix.
  - spoiler 5y ago
scottlamb 5y ago
I'm surprised serde_json takes a std::io::Read rather than a std::io::BufRead. I wonder why they made this choice. Not only does using plain Read make this mistake possible, but it also means they have to be doing byte-by-byte processing. [1] So not really possible to do SIMD-oriented parsing acceleration like simdjson uses. (I gather simdjson's approach is quite sophisticated, but a simpler version is to scan for the next delimiter via a memchr2 on " and \, then have a routine that validates the characters until then are valid via SIMD methods.) [1] or do larger reads into their own buffer. But they're clearly not doing that given this strace output. And it'd be redundant when folks do use BufRead.
- orlp 5y ago
  Taking in a generic T: Read is the recommended way to accept streaming inputs. It is up to the user whether they want to buffer or not. You can pass in a BufReader into a T: Read no problem, and it will buffer any reads. That the reads internally are buffered is an implementation detail serde_json has no business knowing about. The converse is potentially disastrous - if you read too much from a TCP stream you could hang your program waiting for more input that never comes.
  - scottlamb 5y ago
    > Taking in a generic T: Read is the recommended way to accept streaming inputs. Recommended by whom? When you want to process stuff in bulk, it's better to use BufRead::fill_buf (reusing the BufRead's buffer) rather than copy into your own. For example, I wrote some code which basically skips some escape bytes. [1] It wraps a BufRead and is itself a BufRead. Its caller actually can process bytes straight from the buffer of the BufRead you supply, skipping two layers of copying. > The converse is potentially disastrous - if you read too much from a TCP stream you could hang your program waiting for more input that never comes. You can implement any idea badly, but that doesn't mean the idea itself is bad. There's a difference between taking advantage of buffering and expecting to operate only on full, fixed-sized buffers. [1] https://github.com/dholroyd/h264-reader/blob/60ed66dc4dbfe745ceff80692902273b260f2ba9/src/rbsp.rs#L46 https://github.com/dholroyd/h264-reader/blob/60ed66dc4dbfe74...
- 5y ago
eska 5y ago
Multiple of your “senior” developers didn’t catch this in a “senior” developer’s PR? Have they ever read any of the IO documentation, since this is pointed out literally everywhere in the docs? https://doc.rust-lang.org/std/fs/struct.File.html https://doc.rust-lang.org/std/fs/struct.File.html https://doc.rust-lang.org/std/io/trait.Read.html https://doc.rust-lang.org/std/io/trait.Read.html
- maccard 5y ago
  You don't even need to be familiar with rust to know this - it's a feature of pretty much language. That said, I can guarantee rate I have missed such issues in my code both as the writer and the reviewer. Saying "it's in the documentation" doesn't mean it's inexcusable to make 5he mistake.
- DangitBobby 5y ago
  Look at this guy over here "making mistakes" and "missing things in code review." What a fraud. Btw, pointing to documentation and saying "see, it's documented!" Every time something has lame or broken or unexpected default behavior doesn't make it any less lame or broken or unexpected. Once you feel like you know how to use something, you aren't always checking the docs to get a refresher on the caveats. So tone it down with the accusations and the "helpful" links, everyone else has access to the same Google you do.
- kilburn 5y ago
  Everybody makes mistakes. Being senior doesn't make you infallible, or otherwise we wouldn't spend huge amounts of resources building tools that are essentially mistake-catchers: smarter type systems, linters, static analyzers and so on. Also, senior people are more likely to make the kind of mistake explained in the article than less experienced developers. This is because: - They are more focused on the "big picture" (algorithm correctness, general approach, system architecture, etc.) than on minute details. - They tend to read less documentation because they don't need to reach for it so often. - They have learned that a mistake such as this one can be easily corrected, and hence dedicate their attention to more important matters (avoiding data corruption for instance).
FpUser 5y ago
>"In this post, we will look at a common source of bad performance on Rust code that can trip up even veteran developers, and what you can do when this happens in your programs." "Veteran developers", the sort of who would use Rust after having worked in C/C++/etc would know very well that buffering on very granular IO is a must except some specific situations. Otherwise they're anything but. And it has nothing to do with Rust. This is of course when IO libraries used do not already take care of that.
- DangitBobby 5y ago
  The defensive tone by people in this comment section worried that this is some attack on rust is surprising. Experienced people write bad code sometimes. Rust standard library may or may not encourage people to write poorly performing code sometimes if they aren't paying attention to (or don't know) fundamentals. Chill.
  - FpUser 5y ago
    I am not using Rust as I have nothing to gain from it ROI wise. So there is nothing for me to defend. And I am totally cool, no need to chill.

45 points by akbarnama 5y ago

I started a Rust project several months ago and quickly became frustrated with the state of basic terminal libraries. None of them seemed to implement the buffering and minimal-refresh of ncurses, so I was forced to rely on one of the many shifting ncurses binding libraries and all the unsafeness that comes with that (or reimplement my own—way out of scope for the little toy I was building). What's the state of the art for raw terminal libraries (not TUI helpers) in Rust nowadays?

grenoire 5y ago
Guh, I feel you on this one. I was shopping around for a buffering wrapper but soon realise that I either had to get basically a terminal Qt clone or raw bindings, nothing inbetween...

otterley 5y ago

Unbuffered I/O is slower than buffered I/O regardless of language. It’s not just a Rust phenomenon. Making syscalls like write() every time you want to send even a character of output down a stream is much more latent than just sticking bytes in a buffer. In general, the only time anyone should use unbuffered output is when there’s a real person consuming it - for example, when you’re prompting a user for feedback or input.

yongjik 5y ago
> In general, the only time anyone should use unbuffered output is when there’s a real person consuming it - for example, when you’re prompting a user for feedback or input. I'm in the camp that logging should be always unbuffered. If you buffer logs, and your process dies for any reason, you lose your log exactly when you really need it to know what happened.
- otterley 5y ago
  Buffered I/O is usually not lost when a process exits. Usually buffers are flushed automatically upon exit when the file descriptors are closed. And if a program truly crashes and leaves a core behind, the unflushed buffers should be found in the core file.
  - yongjik 5y ago
    If the process exits in an orderly fashion then, yes, it will flush logs, and also do a bunch of other nice things. The problem arises when it dies in a not-so-nice fashion. (Also, not every environment is running with unlimited core size.)
silisili 5y ago
> In general, the only time anyone should use unbuffered output is when there’s a real person consuming it I think this might be a little -too- general, as it's pretty workload dependent. I mean, if you just want to write a blob of bytes once, just write the blob IMO. I don't think using a buffer in those cases ever really hurts, but sometimes makes what should be simple a bit more complicated than necessary.
- molyss 5y ago
  Even for stuff as simple as handling LOBs, I buffer on both read and writes. Some DB APIs really do “what you tell them to”, even if that means being unbearably slow when unbuffered The worst I’ve seen was actually streaming the LOB on getLength. Since there’s no JDBC API for lob.getContentAsArray(), you need to call lob.getContent(1, lob.length()) (notice that you’d start at index 1…), which by default would stream the LOB twice under the scene!
- harles 5y ago
  Definitely too general of a statement. I previously worked on real-time mapping and the only way to get the necessary write performance was to go unbuffered and manually split off a block at a time to write. The same was true of reading - it was faster and made sure the # of syscalls in any frame was predictable. As a side note, the real issue here is that unbuffered is the default. Other languages and libraries I’m aware of make it difficult to get in such a mode.
brundolf 5y ago
Lots of people are writing Rust code who aren't previously versed in the nuances of optimizing bare-metal code. They're (reasonably) expecting amazing performance, because Rust tends to give you that by default compared with higher-level languages. But repeated calls to println!() is one of the biggest and easiest-to-make performance mistakes for those newcomers, so I think it's fair to call it out. With that said, the OP could definitely have done a better job making it clear that this problem isn't specific to Rust
- not2b 5y ago
  Developers from other languages, like C, C++, or even Python, would not have expected this particular problem, because it wouldn't have occurred to them that the thing they get from doing a file-open call and that the JSON reader library accepts is a raw file descriptor (for C programmers, the result of 'open' rather than 'fopen') that does unbuffered I/O. In other languages libraries are oriented around buffered I/O structures (a FILE or an iostream).
  - BoorishBears 5y ago
    To me the biggest reason for the surprise is the signature for 'write' I'm used to a raw stream allowing a few specific operations: - read/write/peek one byte - write a byte array, but you must explicitly specify length and offset So seeing a call that simply takes an entire byte string and writes it gives me "buffered I/O vibes". It's hard to quantify, but with how ubiquitous that pattern is I'd be surprised if there are a lot of people who expect similar.
  - not2b 5y ago
    This had been upvoted and now it is downvoted, even though it is obviously true, as the article demonstrates.
- oconnor663 5y ago
  For what it's worth, I run into the same nuances frequently in Python. For example I often want to run some_script.py | tee log.txt But by default, Python will turn off line buffering when it detects it's not talking to a tty. So unless you know to call flush, or to add the right env var or interpreter flag, you're not going to see any early output from tee. This can be super confusing for newer programmers.
loeg 5y ago
Totally agree. In contrast, for example, FILE objects in C are (mostly) buffered by default. So in that sense, Rust defaults are less sensible for some developers who would be better served by buffering, but aren't aware that they need it. But I also think that Rust's default is totally reasonable and it's just something you have to learn about as a systems programmer.

halayli 5y ago

Unbuffered I/O can make any language you use much slower. It has nothing to do with Rust / language choice.

xxs 5y ago
"Rust" sets you for the front page of hackernews, though.
bcrl 5y ago
Except for the usage patterns where it's faster because you avoid blowing your CPU's caches as much... The trade offs between different ways of performing i/o are decidedly non-trivial.
- halayli 5y ago
  Everything you mentioned is irrelevant in the context of unbuffered IO. The majority of time is going to be spent on system calls, context switching, and signal handling / checking, and waiting for hardware interrupts to fire and handling them. Also there is a lot more then mentioning cpu cache as this is very generic term. which part of the cpu cache are you talking about? instruction cache? data cache? And why do you have think they matter in this context?
  - bcrl 5y ago
    My comments are perfectly relevant. Using read() and write() still uses buffers, it's just that those buffers happen to be in the kernel. read() and write() still do the kernel's form of memcpy() internally. Truly unbuffered I/O from the kernel's perspective is direct I/O that DMAs data directly into application provided memory. It is important to note that every time buffered I/O is used, it copies data through buffers, and touching those buffers means the memory used for those buffers being touched clearly has to be pulled into the CPU's L1 data cache. In some cases the extra cache traffic for those buffers degrades performance. Instruction caches are relevant to some workloads, but are clearly not what I was referring to since data doesn't go through the instruction cache. glibc further complicates the matter by using mmap() in some cases to directly map the kernel's buffers for data (the page cache in the context of Linux) into userspace. This avoids some of the downsides of buffered I/O by avoiding one of the copies at the expense of page fault overhead. Sometimes it's better for the application itself to use mmap() directly. Direct use of read() and write() makes sense for some applications, especially when dealing with larger chunks of data. Saying unbuffered I/O is bad is a gross oversimplification of the reality when writing high performance applications. Like everything in programming, it's a good rule of thumb, but there is definitely an asterisk next to it.

dmw_ng 5y ago

Alternative title: Poor Interface Design Makes Most Rust Programs Much Slower That programmers need to consider this stuff on each new project is a failure of the standard library design, it's a problem not even C++ suffers from

mamcx 5y ago
No. But is very surprising when you see Rust how high-level is as language yet it target system programming. One major philosophy aspect of Rust is that (almost)everything is explicit and you build abstraction from the bottom.
- deleted 5y ago
  [deleted]
trasz 5y ago
Not even C has this problem - stdio handles this just fine, by deciding whether to buffer or not depending on where the data is going to (tty vs a file).
smolder 5y ago
No, it's a design choice, not a 'failure'. Hidden behavior has its own pitfalls, and you can argue this all day, but there are already 100 languages which go the other way and cleverly optimize things behind the scenes every chance they get. I wouldn't mind much if they made IO buffered by default, but I don't mind it's not, either, because it's consistent with the design ethos.
spoiler 5y ago
In my experience, the most ergonomic implementation is rarely the most performant one in C++, you usually end up having to use some SFINAE monstrosity or sprinkle constexpr if all over the place. std::optional comes specifically to mind as one of the big offenders here. You've also gotta be careful with unexpected and implicit assignment/copy/move constructors. More than half of all the C++ wisdom one needs in their career is when to avoid stdlib and which part of boost to use. Due to C++s religious devition to 80s compatibility, it becomes a minefield of a Don'ts, with a narrow path of Dos. The fact boost is a billion libraries in one speaks in of itself about the abysmal dependency situation in C++. There's also a buncha dep managers that are at each others throats. Also, C++ people like to poke fun about js devs and npm/leftpad, but forget that they use boost for even more rudimentary string operations. Or they implement the same thing over and over again. Or my favourite: copy+paste the same code in multiple projects, and then have divergent implementations of the same thing with different bugs and/or API surface; bonus points if the thing is part of an RPC implementation. As much as the edgy-dev crowd loves to belittle the very concept of dependency management, at least JS/Ruby/Python ecosystems give a shit about DX and made sure the package managers can be used with the same repositories/sources. But hey, at least C++’s IO is buffered by default. Amazing. I got distracted by my rant above: I actually PREFER the default is unbuffered, because it doesn't violate the principle of least surprise (if we remember that Rust is a systems programming language). The fact you expect the default to be buffered is probably due to exposure to C/++.

lor_louis 5y ago

I am not all that knowledgeable about rust but I think that if you really want to decrease the numbers of system calls when interacting with files, writev and readv should be considered. I wonder if BuffWriter/Reader could be made to use it, as it would have the same "atomic" behaviour and would forgo the need to concat strings. Still, the char*s could become invalidated once the system call is made.

trasz 5y ago
This solves the “spatial” part, while the actual problem is the “temporal” part: it’s not about many writes at once, it’s about many writes in different parts of code, spaced apart in time.
vlovich123 5y ago
Typically they are orthogonal concepts. In Rust you could probably do it if you’re accepting ownership of the string but if I recall correctly write takes a slice so the contents are not guaranteed to be valid after the write call (eg the user might reuse the buffer for the next write). It might be interesting to have a version of a Writer trait that handed ownership over in addition to supporting slices and handled things under the hood for you. The API may be too complex to bother with. Buffered I/O is typically a fine middle ground. When you start needing writev you’re changing the I/O model more drastically.
zRedShift 5y ago
FWIW, the Read/Write traits have a vectored read/write method, which File implements using readv/writev on supporting Unix platforms: https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#77 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#77 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#144 https://doc.rust-lang.org/src/std/sys/unix/fd.rs.html#144 Buf(Reader|Writer) are vectored I/O aware, bypassing the internal buffer whenever the total length of the provided buffers is larger: https://doc.rust-lang.org/src/std/io/buffered/bufreader.rs.html#292 https://doc.rust-lang.org/src/std/io/buffered/bufreader.rs.h... https://doc.rust-lang.org/src/std/io/buffered/bufwriter.rs.html#570 https://doc.rust-lang.org/src/std/io/buffered/bufwriter.rs.h...

qalmakka 5y ago

I kind of understand why Rust made buffered IO explicit and on-demand instead of going down the C route and making it the default, but still I think there's a significant risk lots of programs are going to be suboptimal due to developer ignorance or the expectation of having buffered IO by default like with stdio and iostreams. It would have been IMHO probably less clean but safer to make the standard IO constructs like File buffered by default (which is what users expect in the vast majority of cases) and provide on the side unbuffered variants. Thankfully modern OSes with caches and stuff somewhat mitigate this thing, but it can make IO on less advanced systems way more problematic than it should be.

brundolf 5y ago
I think it would go against Rust's philosophy to have hidden (and inconsistent) behavior, like automatically buffering some things but not others. Generally Rust tries to give users explicit and ergonomic control over those sorts of details.
- lumost 5y ago
  When developing large scale applications ( e.g. any app that needs third-party libraries ). You are going to have to deal with developers who either 1. Didn't know about a nuance. 2. Didn't care about said nuance. Creating magic incantations to achieve typical expected behavior will result in many libraries doing the unexpected behavior. Which means that the typical code that any end user works with will probably do the unexpected thing.
  - brundolf 5y ago
    It's all relative. I'm very glad that JavaScript doesn't try to make you deal with output buffers, but Rust's goals as a language are different.
    - qalmakka 5y ago
      The problem here is that the uncontested ruler of said system languages, C, has a completely opposite behaviour by default. The C standard clearly states that "When opened, a stream is fully buffered if and only if it can be determined not to refer to an interactive device" (§7.21.5.3.8), which means that generations of developers have learned to always expect buffering when opening files `fopen`. Most higher level languages follow the C library when setting their I/O convention, and this has been the trend for most of the last 40 years. The fact that Rust goes in the opposite direction makes sense from a logical standpoint, but as the article itself shows will almost certainly end up causing performance issues. You can rest assured that people *will* forget to buffer their files because it is something they almost never had to care about in the past.

PaulDavisThe1st 5y ago

Alternative Title: Many Problems That Have Affected Software Development Since Denis Ritchie Roamed the Steppes Continue to Affect Rust Programming

BoorishBears 5y ago
I can't remember the last time I came across a modern language where the "easy default" is unbuffered I/O. A beginner learning C++ today will be getting familiar with buffered I/O primitives by the time they write a "Hello World" Java mostly keeps the "useful" access patterns (anything more than reading/writing a single byte or manually passing in byte buffers) in buffered wrappers so even a clueless beginner is inclined to use them. People are missing the forest for the trees here, it's about ergonomics and encouraging people to do the right thing by default. Ironic to see this attitude in a Rust comment section of all places, isn't this the same attitude that leads people to dismiss Rust? "Just write correct C code without UB, the major pitfalls haven't changed in the last 4 decades duh?"
- PaulDavisThe1st 5y ago
  > I can't remember the last time I came across a modern language where the "easy default" is unbuffered I/O. "Hello World" in C from 1986 uses printf() which is buffered. So yeah, it's been a long, long, long time.
- kccqzy 5y ago
  It's because there's this new trend of high-performance, bufferless, copy-less technologies like io_uring getting mindshare. Beginners who don't understand why these technologies work then blindly remove buffering, or forget to add buffering in the first place.
  - trasz 5y ago
    It’s not really new, it’s a return to how things were done before Unix.
- spoiler 5y ago

scottlamb 5y ago

I'm surprised serde_json takes a std::io::Read rather than a std::io::BufRead. I wonder why they made this choice. Not only does using plain Read make this mistake possible, but it also means they have to be doing byte-by-byte processing. [1] So not really possible to do SIMD-oriented parsing acceleration like simdjson uses. (I gather simdjson's approach is quite sophisticated, but a simpler version is to scan for the next delimiter via a memchr2 on " and \, then have a routine that validates the characters until then are valid via SIMD methods.) [1] or do larger reads into their own buffer. But they're clearly not doing that given this strace output. And it'd be redundant when folks do use BufRead.

orlp 5y ago
Taking in a generic T: Read is the recommended way to accept streaming inputs. It is up to the user whether they want to buffer or not. You can pass in a BufReader into a T: Read no problem, and it will buffer any reads. That the reads internally are buffered is an implementation detail serde_json has no business knowing about. The converse is potentially disastrous - if you read too much from a TCP stream you could hang your program waiting for more input that never comes.
- scottlamb 5y ago
  > Taking in a generic T: Read is the recommended way to accept streaming inputs. Recommended by whom? When you want to process stuff in bulk, it's better to use BufRead::fill_buf (reusing the BufRead's buffer) rather than copy into your own. For example, I wrote some code which basically skips some escape bytes. [1] It wraps a BufRead and is itself a BufRead. Its caller actually can process bytes straight from the buffer of the BufRead you supply, skipping two layers of copying. > The converse is potentially disastrous - if you read too much from a TCP stream you could hang your program waiting for more input that never comes. You can implement any idea badly, but that doesn't mean the idea itself is bad. There's a difference between taking advantage of buffering and expecting to operate only on full, fixed-sized buffers. [1] https://github.com/dholroyd/h264-reader/blob/60ed66dc4dbfe745ceff80692902273b260f2ba9/src/rbsp.rs#L46 https://github.com/dholroyd/h264-reader/blob/60ed66dc4dbfe74...
5y ago

eska 5y ago

Multiple of your “senior” developers didn’t catch this in a “senior” developer’s PR? Have they ever read any of the IO documentation, since this is pointed out literally everywhere in the docs? https://doc.rust-lang.org/std/fs/struct.File.html https://doc.rust-lang.org/std/fs/struct.File.html https://doc.rust-lang.org/std/io/trait.Read.html https://doc.rust-lang.org/std/io/trait.Read.html

maccard 5y ago
You don't even need to be familiar with rust to know this - it's a feature of pretty much language. That said, I can guarantee rate I have missed such issues in my code both as the writer and the reviewer. Saying "it's in the documentation" doesn't mean it's inexcusable to make 5he mistake.
DangitBobby 5y ago
Look at this guy over here "making mistakes" and "missing things in code review." What a fraud. Btw, pointing to documentation and saying "see, it's documented!" Every time something has lame or broken or unexpected default behavior doesn't make it any less lame or broken or unexpected. Once you feel like you know how to use something, you aren't always checking the docs to get a refresher on the caveats. So tone it down with the accusations and the "helpful" links, everyone else has access to the same Google you do.
kilburn 5y ago
Everybody makes mistakes. Being senior doesn't make you infallible, or otherwise we wouldn't spend huge amounts of resources building tools that are essentially mistake-catchers: smarter type systems, linters, static analyzers and so on. Also, senior people are more likely to make the kind of mistake explained in the article than less experienced developers. This is because: - They are more focused on the "big picture" (algorithm correctness, general approach, system architecture, etc.) than on minute details. - They tend to read less documentation because they don't need to reach for it so often. - They have learned that a mistake such as this one can be easily corrected, and hence dedicate their attention to more important matters (avoiding data corruption for instance).

FpUser 5y ago

>"In this post, we will look at a common source of bad performance on Rust code that can trip up even veteran developers, and what you can do when this happens in your programs." "Veteran developers", the sort of who would use Rust after having worked in C/C++/etc would know very well that buffering on very granular IO is a must except some specific situations. Otherwise they're anything but. And it has nothing to do with Rust. This is of course when IO libraries used do not already take care of that.

DangitBobby 5y ago
The defensive tone by people in this comment section worried that this is some attack on rust is surprising. Experienced people write bad code sometimes. Rust standard library may or may not encourage people to write poorly performing code sometimes if they aren't paying attention to (or don't know) fundamentals. Chill.
- FpUser 5y ago
  I am not using Rust as I have nothing to gain from it ROI wise. So there is nothing for me to defend. And I am totally cool, no need to chill.