5 ms·
> NOOOO it's not. It absolutely is. Sure, you can safely call fread() from multiple threads, but they will wait for each other to finish, they will not be read
by simiones 4y ago
> NOOOO it's not.
It absolutely is. Sure, you can safely call fread() from multiple threads, but they will wait for each other to finish, they will not be reading from the underlying file simultaneously.
Also, it's not really possible to use this API thread-safely, especially in the presence of errors. For example, the following code may show exactly the problem I was talking about:
void thread1() {
int* buf1 = calloc(1000, sizeof(int));
while (true) {
int read = fread(buf1, sizeof(int), 1000, file);
if (read < 1000) {
printf("couldn't read whole object: ")
if (ferror(file)) {
printf("read error");
} else if (feof(file)) {
printf("end of file");
} else {
printf("no reason???"); //will be occasionally printed
}
} else {
printf("success");
}
}
}
void thread2() {
while (true) {
fseek(file, 0, 0);
}
}
> FILE streams are buffered, so you never know which fread() call was the "source" of the error. And there is absolutely no point at all in trying to relate errors on a file handle to the n'th or (n+1)'th call.
The buffering is irrelevant. Sure, it's possible that ferror(f) is true even if your data was read successfully (a confusing situation in itself). But, in a non-concurrent context, you can always tell which fread() failed and why - the first time that you get back less bytes than you requested, if the file is not at the end, a read error was encountered (possibly some time ago, as you say).
> Yes it does, unless you're willing to throw away the gains you got from concurrent operation (or with those fancy messy async frameworks, throw away at least a part of the gains). Retrieving the error or other return value requires synchronization.
If you don't care about the result, than you don't even need to handle errors. If you do care about the result, returning the error in the exact same way as the success result adds 0 extra overhead/synchronization. However, returning the result in one place and the errors in another (like fread() does) adds mental overhead.
> Better than checking each call individually, isn't it? Less code and faster to execute, because requests can be processed asynchronously. This is very much like the FILE API.
It's easier to follow than manually checking each call individually, but I would say it's overall much worse:
1. If you forget to check for mctx.bad() before using the buffers, you'll have a very hard to track error. I would have to be very mindful of this during code review.
2. If you wanted to do something like initializing an array of buffers, you would have to check if mctx.bad() multiple times (once after malloc'ing the array, and again after malloc'ing all the buffers).
3. If I want to sequence these operations (write 1337 to buffer2 only if I successfully wrote 666 to buffer1) or need all of the data to be written or the whole operation is useless (and I don't want to wait to write 2GB to some file only to then find out that the 1B write to the registry failed), I actually have to write more code than if I had exceptions, since I have to add checks after each operation to interrupt the others.
In contrast, here is how this would look like with C# async/await and typical exceptions:
using(var mctx = malloc_context()) {
var buffer1 = mctx.malloc_buffer(size1); //throws exception if it fails
var buffer2 = mctx.malloc_buffer(size2);
var buffers = mctx.malloc_buffer_array(100); //your solution was not allocating this, so didn't care about errors
for (var i = 0; i < 100; i++) {
buffers[i] = mctx.malloc_buffer(42);
}
//assuming we want the behavior you suggest, but buffer_write is sync
Task.WaitAll(new Task[] {
Task.Factory.StartNew(() => {
try {
buffer1.write(data, 666);
catch(IOException e1) {
//handle very specific problem?
}
}),
Task.Factory.StartNew(() => buffer2.write(data, 1337))
}); //throws AggregateException if any of the tasks throws IOException
//assuming we want to write all data and fail as soon as one thing fails:
buffer1.write(data, 666);
buffer2.write(data, 1337);
} //error or no error, everything is cleaned up and an error is signalled to higher layers if this failed
If you want asynchronous behavior, you should make it explicitly asynchronous.
- jstimpfle 4y ago> It absolutely is. Sure, you can safely call fread() from multiple threads, but they will wait for each other to finish, they will not be reading from the underlying file simultaneously. What I took "synchronous" to mean was not whether FILE serializes access by multiple readers/writers (it does), but how the I/O operations are carried out. They are carried out a-synchronously in the sense that each fread() does not cause a corresponding read() syscall. Any fread() can cause 0, 1, or more such calls. In general, the hope is that it causes close to 0 system calls on average (that's the idea behind the buffering). In some sense, the multiple threads will not be reading from the underlying file _at all_. To understand what I mean, imagine an implementation of FILE where the I/O is carried out by a worker thread, and connected to the FILE structure using a in-process queue and some thread synchronization (Mutex, Condition variable or similar). fread() only reads from the queue, and synchronizes to the condition variable only when not enough data is in the queue (just returning a short read instead of synchronizing and waiting for more data would be a more useful default option, but that is not how the simplistic FILE API is specified). In any case it never calls the OS for file I/O. > But, in a non-concurrent context, you can always tell which fread() failed and why I don't think you can do that unless the specs define precisely when the actually OS calls (read(2)) happen. Btw. you might also see errors from filesystem operations that are not related to the particular file you were operating on - but due to some H/W problem, quota or filesystem full state, etc. What you're up to is not leading anywhere. Anyway, my point is that this is almost completely meaningless, and definitely not how you are encouraged to use this API. Even the OS doesn't make particular guarantees what call an error that was returned to you is related to. For example, an error "resulting" from a write() might be visible only on close(). (Or in the worst case you never get to see it, but let's ignore that here). What you _can_ do is call explicit synchronization routines (there is fflush() in case of FILE, but if you're getting serious you need to drop stdio and interface with the particular OS that you're running on). This lets you make inferences like, "everything I've written up to now is stored on disk". Notice as well how your example of "tell which fread() failed" breaks down in the case of fwrite(), unless you do fflush() after every single fwrite(), which you absolutely are not advised to do because the temperature in my office has been already sufficient this summer. There is an interesting asymmetry in fread() vs fwrite() with the usual implementations, which stems from the fact that blocking I/O is relatively "impure". A successful fread() implies that the data that crossed the FILE API has crossed the kernel/userspace boundary in the current call or in an earlier one. While an fwrite() implies the opposite - the data is transferred in the current or one of the next calls. It would make a lot more sense if the OS would read/write the FILE buffers asynchronously, just like the worker thread example above. These days we're moving more and more to more explicitly decoupled asynchronous I/O, e.g. io_uring, and with these APIs there is a perfect symmetry between reading and writing, it's only a buffer exchange in both cases. Even when you call fflush(), there is no guarantee that the data was written to disk. In general the data is only pushed to OS buffers, and will be committed to disk asynchronously. Consequently, there is no way to tell "which fwrite() failed". In conclusion, this is a hopeless undertaking. What you're trying to do is resulting in unmaintainable and brittle code. Not too mention REALLY SLOW because whenever you're introducing synchronization everything is grinding to a halt. The tasks that we do on machines are nowhere near instant. Better imagine a computer as a house full of pipes connected to each other. All you can do in any particular location is pull a bit here, push a little there.