4 ms·

> It absolutely is. Sure, you can safely call fread() from multiple threads, but they will wait for each other to finish, they will not be reading from the unde

> It absolutely is. Sure, you can safely call fread() from multiple threads, but they will wait for each other to finish, they will not be reading from the underlying file simultaneously. What I took "synchronous" to mean was not whether FILE serializes access by multiple readers/writers (it does), but how the I/O operations are carried out. They are carried out a-synchronously in the sense that each fread() does not cause a corresponding read() syscall. Any fread() can cause 0, 1, or more such calls. In general, the hope is that it causes close to 0 system calls on average (that's the idea behind the buffering). In some sense, the multiple threads will not be reading from the underlying file _at all_. To understand what I mean, imagine an implementation of FILE where the I/O is carried out by a worker thread, and connected to the FILE structure using a in-process queue and some thread synchronization (Mutex, Condition variable or similar). fread() only reads from the queue, and synchronizes to the condition variable only when not enough data is in the queue (just returning a short read instead of synchronizing and waiting for more data would be a more useful default option, but that is not how the simplistic FILE API is specified). In any case it never calls the OS for file I/O. > But, in a non-concurrent context, you can always tell which fread() failed and why I don't think you can do that unless the specs define precisely when the actually OS calls (read(2)) happen. Btw. you might also see errors from filesystem operations that are not related to the particular file you were operating on - but due to some H/W problem, quota or filesystem full state, etc. What you're up to is not leading anywhere. Anyway, my point is that this is almost completely meaningless, and definitely not how you are encouraged to use this API. Even the OS doesn't make particular guarantees what call an error that was returned to you is related to. For example, an error "resulting" from a write() might be visible only on close(). (Or in the worst case you never get to see it, but let's ignore that here). What you _can_ do is call explicit synchronization routines (there is fflush() in case of FILE, but if you're getting serious you need to drop stdio and interface with the particular OS that you're running on). This lets you make inferences like, "everything I've written up to now is stored on disk". Notice as well how your example of "tell which fread() failed" breaks down in the case of fwrite(), unless you do fflush() after every single fwrite(), which you absolutely are not advised to do because the temperature in my office has been already sufficient this summer. There is an interesting asymmetry in fread() vs fwrite() with the usual implementations, which stems from the fact that blocking I/O is relatively "impure". A successful fread() implies that the data that crossed the FILE API has crossed the kernel/userspace boundary in the current call or in an earlier one. While an fwrite() implies the opposite - the data is transferred in the current or one of the next calls. It would make a lot more sense if the OS would read/write the FILE buffers asynchronously, just like the worker thread example above. These days we're moving more and more to more explicitly decoupled asynchronous I/O, e.g. io_uring, and with these APIs there is a perfect symmetry between reading and writing, it's only a buffer exchange in both cases. Even when you call fflush(), there is no guarantee that the data was written to disk. In general the data is only pushed to OS buffers, and will be committed to disk asynchronously. Consequently, there is no way to tell "which fwrite() failed". In conclusion, this is a hopeless undertaking. What you're trying to do is resulting in unmaintainable and brittle code. Not too mention REALLY SLOW because whenever you're introducing synchronization everything is grinding to a halt. The tasks that we do on machines are nowhere near instant. Better imagine a computer as a house full of pipes connected to each other. All you can do in any particular location is pull a bit here, push a little there.

simiones 4y ago
I still think that most of this is irrelevant. Especially for fread(), all I care about when I'm doing error handling is "if I didn't get all the data I requested for, why didn't I?" - if it's ferror() I may want to retry reading from the same place where my fread() began, but if it's feof() I probably don't. In this case, I can only meaningfully handle the error if I know exactly which of my fread()s failed for which reason. Sure, the actual IO that caused that error may have happened 3 hours ago on a different thread, and 100 fread()s may have been able to read data from the buffer since then without caring about the IO error. When the 101st one actually sees it can't read enough, now I have to care, and I want to find out as easily as possible, and atomically. I don't want my (buffered) read to fail and then be unable to find out why because some other thread reset my context. Now, for fwrite() the issue is indeed quite different, since my code doesn't depend on the result of the operation in the same way (I am ultimately invoking it for its side effect, not for its result, unlike fread() ). And you're right, by its very nature, fwrite() can't meaningfully report all errors I would care about (and even fflush() can't, as the creators of Postgres found out the hard way). But even here, the problem is not related in any way to how you chose to report exceptions, and ferror() is not in any way reliable for ensuring transactional access, any more than exceptions would be. In fact, the newer async IO operations in both Windows and Linux (I/O Rings and io_uring) also return the result in place, not on some context object. Looking more closely at the Linux API, each CQE (completion queue event) has a `res` field that represents the error code from the results of 1 specific SQE (submission queue entry); this entirely replaces the state-based global errno variable that was sometimes used for other system calls. So, the pattern here is: start an async IO task, check the error returned in case it wasn't able to even start, wait for it to complete, check the error this specific task produced when it completed. The same could be done with a hypothetical safe write operation: initiate a call to definitely_write(), and get back some kind of handle that allowed us to wait until those particular bytes were physically written to disk (or at the very least to the disk driver, since those now often do their own level of buffering...). The wait() operation on this handle would then return an error code if the write failed for any reason. Of course, in the case of an actual IO error, it's very likely that many different such handles will return the same error when someone waits for them to finish, since the error itself was not specific to some specific byte. And also of course, it is possible that the bytes from one particular definitely_write() that returns an error on wait() have actually been written to disk - the error just means that the IO layer can't guarantee this to us. But even so, there's no reason and no benefit to decoupling the error reporting from the operations themselves, especially if the error state is transient - and all modern IO APIs, low or high level, recognize this.

simiones 4y ago

I still think that most of this is irrelevant. Especially for fread(), all I care about when I'm doing error handling is "if I didn't get all the data I requested for, why didn't I?" - if it's ferror() I may want to retry reading from the same place where my fread() began, but if it's feof() I probably don't. In this case, I can only meaningfully handle the error if I know exactly which of my fread()s failed for which reason. Sure, the actual IO that caused that error may have happened 3 hours ago on a different thread, and 100 fread()s may have been able to read data from the buffer since then without caring about the IO error. When the 101st one actually sees it can't read enough, now I have to care, and I want to find out as easily as possible, and atomically. I don't want my (buffered) read to fail and then be unable to find out why because some other thread reset my context. Now, for fwrite() the issue is indeed quite different, since my code doesn't depend on the result of the operation in the same way (I am ultimately invoking it for its side effect, not for its result, unlike fread() ). And you're right, by its very nature, fwrite() can't meaningfully report all errors I would care about (and even fflush() can't, as the creators of Postgres found out the hard way). But even here, the problem is not related in any way to how you chose to report exceptions, and ferror() is not in any way reliable for ensuring transactional access, any more than exceptions would be. In fact, the newer async IO operations in both Windows and Linux (I/O Rings and io_uring) also return the result in place, not on some context object. Looking more closely at the Linux API, each CQE (completion queue event) has a `res` field that represents the error code from the results of 1 specific SQE (submission queue entry); this entirely replaces the state-based global errno variable that was sometimes used for other system calls. So, the pattern here is: start an async IO task, check the error returned in case it wasn't able to even start, wait for it to complete, check the error this specific task produced when it completed. The same could be done with a hypothetical safe write operation: initiate a call to definitely_write(), and get back some kind of handle that allowed us to wait until those particular bytes were physically written to disk (or at the very least to the disk driver, since those now often do their own level of buffering...). The wait() operation on this handle would then return an error code if the write failed for any reason. Of course, in the case of an actual IO error, it's very likely that many different such handles will return the same error when someone waits for them to finish, since the error itself was not specific to some specific byte. And also of course, it is possible that the bytes from one particular definitely_write() that returns an error on wait() have actually been written to disk - the error just means that the IO layer can't guarantee this to us. But even so, there's no reason and no benefit to decoupling the error reporting from the operations themselves, especially if the error state is transient - and all modern IO APIs, low or high level, recognize this.