4 ms·

The solution is that buffered accesses should almost always flush after a threshold number of bytes or after a period of time if there is at least one byte, “th

by Veserv 2y ago

The solution is that buffered accesses should almost always flush after a threshold number of bytes or after a period of time if there is at least one byte, “threshold or timeout”. This is pretty common in hardware interfaces to solve similar problems. In this case, the library that buffers in userspace should set appropriate timers when it first buffers the data. Good choices of timeout parameter are: passed in as argument, slightly below human-scale (e.g. 1-100 ms), proportional to {bandwidth / threshold} (i.e. some multiple of the time it would take to reach the threshold at a certain access rate), proportional to target flushing overhead (e.g. spend no more than 0.1% time in syscalls). Also note this applies for both writes and reads. If you do batched/coalesced reads then you likely want to do something similar. Though this is usually more dependent on your data channel as you need some way to query or be notified of “pending data” efficiently which your channel may not have if it was not designed for this use case. Again, pretty common in hardware to do interrupt coalescing and the like.

asveikau 2y ago
I think doing those timeouts transparently would be tricky under the constraints of POSIX and ISO C. It would need to have some cooperation from the application layer
- jart 2y ago
  The only way you'd be able to do it is by having functions like fputc() call clock_gettime(CLOCK_MONOTONIC_COARSE) which will impose ~3ns overhead on platforms like x86-64 Linux which have a vDSO implementation. So it can be practical sort of although it'd probably be smarter to just use line buffered or unbuffered stdio. In practice even unbuffered i/o isn't that bad. It's the default for stderr. It actually is buffered in practice, since even in unbuffered mode, functions like printf() still buffer internally. You just get assurances whatever it prints will be flushed by the end of the call.
  - asveikau 2y ago
    That's just for checking the clock. You'd also need to have a way of getting called back when the timeout expires, after fputc et al are long gone from the stack and your program is busy somewhere else, or maybe blocked. Timeouts are usually done with signals (a safety nightmare, so no thanks) or an event loop. Hence my thought that you can't do it really transparently while keeping current interfaces.
    - jart 2y ago
      Signals aren't a nightmare it's just that fflush() isn't defined by POSIX as being asynchronous signal safe. You could change all your stdio functions to block signals while running, but then you'd be adding like two system calls to every fputc() call. Smart thing to do would probably be creating a thread with a for (;;) { usleep(10000); fflush(stdout); } loop.
      - asveikau 2y ago
        Signals are indeed a nightmare. Your example of adding tons of syscalls to make up for lack of safety shows that you understand that to be true. And no, creating threads to solve this fringe problem in a spin loop with a sleep is not what I'd call "smart". It's unnecessary complexity and in most cases, totally wasted work.
        jart 2y ago
        The smartest thing to do is still probably not buffering. What's wrong with the thread? It would take maybe 15 lines of code to implement. It would be correct without rarely occurring bugs. It doesn't need signals or timers. It wouldn't add overhead to stdio calls. It's a generalized abstraction. You won't need to change your program's event loop code. Create the thread with a tiny 64kb stack and what's not to like? Granted, it would rub me the wrong way if libc did this by default, since I wouldn't want mystery threads appearing in htop for my hello world programs. But for an app developer, this is a sure fire solution.
        klempner 2y ago
toast0 2y ago
I think this is the right approach, but any libc setting automatic timers would lead to a lot of tricky problems because it would change expectations. I/O errors could occur at any point, instead of only when you write. Syscalls everywhere could be interrupted by a timer, instead of only where the program set timers, or when a signal arrives. There's also a reasonable chance of confusion when the application and libc both set timer, depending on how the timer is set (although maybe this isn't relevant anymore... kernel timer apis look better than I remember). If the application specifically pauses signals for critical sections, that impacts the i/o timers, etc. There's a need to be more careful in accessing i/o structures because of when and how signals get handled.
- nine_k 2y ago
  I don't follow. Using a pipe sets an expectation of some amount of asynchronicity, because we only control one end of the pipe. I don't see a dramatic difference between an error occurring because of the process on the other end is having trouble, or because of a timeout handler is trying to push the bytes. On the reading end, the error may occur at the attempt to read the pipe. On the writing end, the error may be signaled at the next attempt to write to or close the pipe. In either case, a SIGPIPE can be sent asynchronously. What scenario am I missing?
  - toast0 2y ago
    > In either case, a SIGPIPE can be sent asynchronously. My expectation (and I think this is an accurate expecation) is that a) read does not cause a SIGPIPE, read on a widowed pipe returns a zero count read as indication of EOF. b) write on a widowed pipe raises SIGPIPE before the write returns. c) write to a pipe that is valid will not raise SIGPIPE if the pipe is widowed without being read from. Yes, you could get a SIGPIPE from anywhere at anytime, but unless someone is having fun on your system with random kills, you won't actually get one except immediately after a write to a pipe. With a timer based asynchronous write, this changes to potentially happening any time. This could be fine if it was well documented and expected, but it would be a mess to add it into the libcs at this point. Probably a mess to add it to basic output buffering in most languages.
- 2y ago
vlovich123 2y ago
Typical Linux alarms are based on signals and are very difficult to manage and rescheduling them may have a performance impact since it requires thunking into the kernel. If you use io_uring with userspace timers things can scale much better, but it still requires you to do tricks if you want to support a lot of fast small writes (eg > ~1 million writes per second timer management starts to show up more and more and you have to do some crazy tricks I figured out to get up to 100M writes per second)
- Veserv 2y ago
  You do not schedule a timeout on each buffered write. You only schedule one timeout on the transition from empty to non-empty that is retired either when the timeout occurs or when you threshold flush (you may choose to not clear on threshold flush if timeout management is expensive). So, you program at most one timeout per timeout duration/threshold flush. The point is to guarantee data gets flushed promptly which only fails when not enough data gets buffered. The timeout is a fallback to bound the flush latency.
  - vlovich123 2y ago
    Yes that can work but as I said that has trade offs. If you flush before the buffer is full, you’re sacrificing throughput. Additionally the timer firing has additional performance degradation especially if you’re in libc land and only have a sigalarm available. So when an additional write is added, you want to push out the timer. But arming the timer requires reading the current time among other things and at rates of 10-20Mhz and up reading the current wall clock gets expensive. Even rdtsc approaches start to struggle at 20-40Mhz. You obviously don’t want to do it on every write but you want to make sure that you never actually trigger the timer if you’re producing data at a relatively fast enough clip to otherwise fill the buffer within a reasonable time. Source: I implemented write coalescing in my nosql database that can operate at a few gigahertz for 8 byte writes/s into an in memory buffer. Once the buffer is full or a timeout occurs, a flush to disk is triggered and I net out at around 100M writes/s (sorting the data for the LSM is one of the main bottlenecks). By comparison DBs like RocksDB can do ~2M writes/s and SQLite can do ~800k.
YZF 2y ago
I would tend to disagree. The buffering here is doing what's it's supposed to be. The mix of something that's supposed to be interactive with a contract that's not meant to be interactive is the source of the problem (tail following into a pipe). There's no real problem to solve here. The "hardware" analog is a tank of water that accumulates rainwater and you only move it somewhere when it fills up. I'm not sure what examples you have in mind but time based flushes aren't common in hardware AFAIK. The proposed fix makes the contract a lot more complicated.
- dullcrisp 2y ago
  How is there no problem to solve when the post clearly identifies a problem, as well as why the behavior is confusing to a user? “The system is working as it was designed,” is always true but unhelpful.
  - YZF 2y ago
    I get that. But it's like complaining why your pliers aren't good at unscrewing a bolt. I'm willing to accept the UX isn't great but `tail -f | grep` is like `vim | grep` or `rogue | grep` (I'm exaggerating to try and make a point). `tail -f` is also if I'm not mistaken a much more recent development vs. the origins of the Unix command line pipes. So sure, it would maybe be a better UX to be able to combine things and have them work, but there is fundamental tension between building something that's optimized for moving chunks of data and building things that's interactive. And trying to force one into the other, in my humble opinion, is not the solution.
    - kiitos 2y ago
      > `tail -f | grep` is like `vim | grep` or `rogue | grep` I think this position is user-hostile. `vim` and `rogue` are fully user-interactive programs. The same is not true of `tail -f`, which by default appears to users as a stream of lines. I understand why, at a technical level, `tail -f | grep` doesn't work in the way that's expected here. But it should! At least, when invoked from a user-interactive shell session -- in that context, a "chunk of data" is clearly expected to be a newline-delimited line, not a buffer of some implicitly-defined size.
worksonmymach 2y ago
I prefer a predictable footgun. Your idea is good but it would need to be another flag, so you have to know it exists. Not knowing the semantics is the issue, rather than the semantics themselves.

by Veserv 2y ago

asveikau 2y ago

I think doing those timeouts transparently would be tricky under the constraints of POSIX and ISO C. It would need to have some cooperation from the application layer

jart 2y ago
The only way you'd be able to do it is by having functions like fputc() call clock_gettime(CLOCK_MONOTONIC_COARSE) which will impose ~3ns overhead on platforms like x86-64 Linux which have a vDSO implementation. So it can be practical sort of although it'd probably be smarter to just use line buffered or unbuffered stdio. In practice even unbuffered i/o isn't that bad. It's the default for stderr. It actually is buffered in practice, since even in unbuffered mode, functions like printf() still buffer internally. You just get assurances whatever it prints will be flushed by the end of the call.
- asveikau 2y ago
  That's just for checking the clock. You'd also need to have a way of getting called back when the timeout expires, after fputc et al are long gone from the stack and your program is busy somewhere else, or maybe blocked. Timeouts are usually done with signals (a safety nightmare, so no thanks) or an event loop. Hence my thought that you can't do it really transparently while keeping current interfaces.
  - jart 2y ago
    Signals aren't a nightmare it's just that fflush() isn't defined by POSIX as being asynchronous signal safe. You could change all your stdio functions to block signals while running, but then you'd be adding like two system calls to every fputc() call. Smart thing to do would probably be creating a thread with a for (;;) { usleep(10000); fflush(stdout); } loop.
    - asveikau 2y ago
      Signals are indeed a nightmare. Your example of adding tons of syscalls to make up for lack of safety shows that you understand that to be true. And no, creating threads to solve this fringe problem in a spin loop with a sleep is not what I'd call "smart". It's unnecessary complexity and in most cases, totally wasted work.
      - jart 2y ago
        The smartest thing to do is still probably not buffering. What's wrong with the thread? It would take maybe 15 lines of code to implement. It would be correct without rarely occurring bugs. It doesn't need signals or timers. It wouldn't add overhead to stdio calls. It's a generalized abstraction. You won't need to change your program's event loop code. Create the thread with a tiny 64kb stack and what's not to like? Granted, it would rub me the wrong way if libc did this by default, since I wouldn't want mystery threads appearing in htop for my hello world programs. But for an app developer, this is a sure fire solution.
        klempner 2y ago

toast0 2y ago

I think this is the right approach, but any libc setting automatic timers would lead to a lot of tricky problems because it would change expectations. I/O errors could occur at any point, instead of only when you write. Syscalls everywhere could be interrupted by a timer, instead of only where the program set timers, or when a signal arrives. There's also a reasonable chance of confusion when the application and libc both set timer, depending on how the timer is set (although maybe this isn't relevant anymore... kernel timer apis look better than I remember). If the application specifically pauses signals for critical sections, that impacts the i/o timers, etc. There's a need to be more careful in accessing i/o structures because of when and how signals get handled.

nine_k 2y ago
I don't follow. Using a pipe sets an expectation of some amount of asynchronicity, because we only control one end of the pipe. I don't see a dramatic difference between an error occurring because of the process on the other end is having trouble, or because of a timeout handler is trying to push the bytes. On the reading end, the error may occur at the attempt to read the pipe. On the writing end, the error may be signaled at the next attempt to write to or close the pipe. In either case, a SIGPIPE can be sent asynchronously. What scenario am I missing?
- toast0 2y ago
  > In either case, a SIGPIPE can be sent asynchronously. My expectation (and I think this is an accurate expecation) is that a) read does not cause a SIGPIPE, read on a widowed pipe returns a zero count read as indication of EOF. b) write on a widowed pipe raises SIGPIPE before the write returns. c) write to a pipe that is valid will not raise SIGPIPE if the pipe is widowed without being read from. Yes, you could get a SIGPIPE from anywhere at anytime, but unless someone is having fun on your system with random kills, you won't actually get one except immediately after a write to a pipe. With a timer based asynchronous write, this changes to potentially happening any time. This could be fine if it was well documented and expected, but it would be a mess to add it into the libcs at this point. Probably a mess to add it to basic output buffering in most languages.
2y ago

vlovich123 2y ago

Typical Linux alarms are based on signals and are very difficult to manage and rescheduling them may have a performance impact since it requires thunking into the kernel. If you use io_uring with userspace timers things can scale much better, but it still requires you to do tricks if you want to support a lot of fast small writes (eg > ~1 million writes per second timer management starts to show up more and more and you have to do some crazy tricks I figured out to get up to 100M writes per second)

Veserv 2y ago
You do not schedule a timeout on each buffered write. You only schedule one timeout on the transition from empty to non-empty that is retired either when the timeout occurs or when you threshold flush (you may choose to not clear on threshold flush if timeout management is expensive). So, you program at most one timeout per timeout duration/threshold flush. The point is to guarantee data gets flushed promptly which only fails when not enough data gets buffered. The timeout is a fallback to bound the flush latency.
- vlovich123 2y ago
  Yes that can work but as I said that has trade offs. If you flush before the buffer is full, you’re sacrificing throughput. Additionally the timer firing has additional performance degradation especially if you’re in libc land and only have a sigalarm available. So when an additional write is added, you want to push out the timer. But arming the timer requires reading the current time among other things and at rates of 10-20Mhz and up reading the current wall clock gets expensive. Even rdtsc approaches start to struggle at 20-40Mhz. You obviously don’t want to do it on every write but you want to make sure that you never actually trigger the timer if you’re producing data at a relatively fast enough clip to otherwise fill the buffer within a reasonable time. Source: I implemented write coalescing in my nosql database that can operate at a few gigahertz for 8 byte writes/s into an in memory buffer. Once the buffer is full or a timeout occurs, a flush to disk is triggered and I net out at around 100M writes/s (sorting the data for the LSM is one of the main bottlenecks). By comparison DBs like RocksDB can do ~2M writes/s and SQLite can do ~800k.

YZF 2y ago

I would tend to disagree. The buffering here is doing what's it's supposed to be. The mix of something that's supposed to be interactive with a contract that's not meant to be interactive is the source of the problem (tail following into a pipe). There's no real problem to solve here. The "hardware" analog is a tank of water that accumulates rainwater and you only move it somewhere when it fills up. I'm not sure what examples you have in mind but time based flushes aren't common in hardware AFAIK. The proposed fix makes the contract a lot more complicated.

dullcrisp 2y ago
How is there no problem to solve when the post clearly identifies a problem, as well as why the behavior is confusing to a user? “The system is working as it was designed,” is always true but unhelpful.
- YZF 2y ago
  I get that. But it's like complaining why your pliers aren't good at unscrewing a bolt. I'm willing to accept the UX isn't great but `tail -f | grep` is like `vim | grep` or `rogue | grep` (I'm exaggerating to try and make a point). `tail -f` is also if I'm not mistaken a much more recent development vs. the origins of the Unix command line pipes. So sure, it would maybe be a better UX to be able to combine things and have them work, but there is fundamental tension between building something that's optimized for moving chunks of data and building things that's interactive. And trying to force one into the other, in my humble opinion, is not the solution.
  - kiitos 2y ago
    > `tail -f | grep` is like `vim | grep` or `rogue | grep` I think this position is user-hostile. `vim` and `rogue` are fully user-interactive programs. The same is not true of `tail -f`, which by default appears to users as a stream of lines. I understand why, at a technical level, `tail -f | grep` doesn't work in the way that's expected here. But it should! At least, when invoked from a user-interactive shell session -- in that context, a "chunk of data" is clearly expected to be a newline-delimited line, not a buffer of some implicitly-defined size.

worksonmymach 2y ago

I prefer a predictable footgun. Your idea is good but it would need to be another flag, so you have to know it exists. Not knowing the semantics is the issue, rather than the semantics themselves.