4 ms·

Now tell me what exactly was wrong with fopen()/fread()/ferror()? > They simply don't work, at all, in any kind of concurrent application, even a single-threa

Now tell me what exactly was wrong with fopen()/fread()/ferror()? > They simply don't work, at all, in any kind of concurrent application, even a single-threaded one. What am I missing here? Why does this make no sense at all to me? For concurrent programming, containing errors / handling them asynchronously is basically a requirement. It's how distributed systems work.

simiones 4y ago
fopen()/fread()/ferror() work decently well only because fread() is an inherently synchronous operation. Still, returning the error from fread() would be infinitely better, except that C doesn't support multiple return values in any nice to use form. But, if it would have made sense to call fread() on the same file handle concurrently, you would have to add synchronization logic (externally or internally) before checking the error flag, and even then, there would be no way to signal two different errors occurring in two different concurrent fread() calls - one error would always overwrite the other. > For concurrent programming, containing errors / handling them asynchronously is basically a requirement. Sure, the concurrency implementation ultimately has to somehow store errors, but this doesn't have to be exposed to the client API. For example, C#'s async/await handle exceptions like you would expect: an async function can throw an exception, and await will throw the exception to the caller of that await (or send an AggregateException if this is caused by something like AwaitAny/AwaitAll). The same thing can be done with error results. Alternatively, in CSP style, error values can just be regular events that a process may continue from (somewhat equivalent to sending an error value on a `chan` in Go). As I said, in all of these cases, the runtime does have to do what you are suggesting, as you would if you were writing interrupt handlers in an embedded program. But I don't think there is any good reason to casually do this in normal code, as it adds significant synchronization requirements to what may otherwise be/seem single-threaded code. For example, imagine if malloc() had a corresponding merror() like fread() to indicate why an allocation failed. If you called malloc() and received a null pointer, you would probably see a "no error" state before you got a chance to check the result of merror() in a concurrent program, since there's a good chance something else had successfully allocated memory before you got a chance to read an error. To avoid this, you would have to explicitly guard with a mutex from before malloc() to after merror(), instead of relying on the internal locks in malloc() (and thus also lose all of the optimization opportunities for thread-local allocation).
- jstimpfle 4y ago
  > fread() is an inherently synchronous operation NOOOO it's not. It's the opposite. FILE streams are buffered, and the typical case is that fread() just copies from its internal buffer, where the data in the buffer had already been transferred from the OS at some earlier point. While it is not normally done, I think fread() could be implemented with a worker in a separate thread no problem. Think of it as an asynchronous API. > if it would have made sense to call fread() on the same file handle concurrently, you would have to add synchronization logic (externally or internally) before checking the error flag, and even then, there would be no way to signal two different errors occurring in two different concurrent fread() calls - one error would always overwrite the other. FILE streams are in fact protected by mutexes internally - it is safe to call e.g. fwrite() from different threads simultaneously (you can also circumvent the mutexes using e.g. fputc_unlocked() if you know it's safe to do so). Whether doing that makes sense beyond the quick debug printf is another question. FILE streams are buffered, so you never know which fread() call was the "source" of the error. And there is absolutely no point at all in trying to relate errors on a file handle to the n'th or (n+1)'th call. But if there was, the only thing you would have to do is to pass the function calls some context to discern between the calls. Essentially this is state segregation, which is not the point of the FILE API, which is why it doesn't do that. > Sure, the concurrency implementation ultimately has to somehow store errors, but this doesn't have to be exposed to the client API. Yes it does, unless you're willing to throw away the gains you got from concurrent operation (or with those fancy messy async frameworks, throw away at least a part of the gains). Retrieving the error or other return value requires synchronization. > imagine if malloc() had a corresponding merror() like fread() to indicate why an allocation failed You're not getting the point. malloc() API is fine for what it does (general purpose memory allocation in global context, which is not exactly what you should use in a well developed system, but like printf it gets you started...), and I'm not suggesting there should be a global merror() kind of thing. But checkout the following: mctx := malloc_context() buffer1 := malloc_buffer(mctx, size1) buffer2 := malloc_buffer(mctx, size2) for i in 0...100: buffers[i] := malloc_buffer(mctx, 42) if mctx.bad(): mctx.release() return # go on using the buffers... Maybe even like follows?: buffer_write(buffer1, data, 666) buffer_write(buffer2, data, 1337) if buffer1.bad(): # handle very specific problem? if mctx.bad(): # handle problem with any buffer? Better than checking each call individually, isn't it? Less code and faster to execute, because requests can be processed asynchronously. This is very much like the FILE API.

simiones 4y ago

fopen()/fread()/ferror() work decently well only because fread() is an inherently synchronous operation. Still, returning the error from fread() would be infinitely better, except that C doesn't support multiple return values in any nice to use form. But, if it would have made sense to call fread() on the same file handle concurrently, you would have to add synchronization logic (externally or internally) before checking the error flag, and even then, there would be no way to signal two different errors occurring in two different concurrent fread() calls - one error would always overwrite the other. > For concurrent programming, containing errors / handling them asynchronously is basically a requirement. Sure, the concurrency implementation ultimately has to somehow store errors, but this doesn't have to be exposed to the client API. For example, C#'s async/await handle exceptions like you would expect: an async function can throw an exception, and await will throw the exception to the caller of that await (or send an AggregateException if this is caused by something like AwaitAny/AwaitAll). The same thing can be done with error results. Alternatively, in CSP style, error values can just be regular events that a process may continue from (somewhat equivalent to sending an error value on a `chan` in Go). As I said, in all of these cases, the runtime does have to do what you are suggesting, as you would if you were writing interrupt handlers in an embedded program. But I don't think there is any good reason to casually do this in normal code, as it adds significant synchronization requirements to what may otherwise be/seem single-threaded code. For example, imagine if malloc() had a corresponding merror() like fread() to indicate why an allocation failed. If you called malloc() and received a null pointer, you would probably see a "no error" state before you got a chance to check the result of merror() in a concurrent program, since there's a good chance something else had successfully allocated memory before you got a chance to read an error. To avoid this, you would have to explicitly guard with a mutex from before malloc() to after merror(), instead of relying on the internal locks in malloc() (and thus also lose all of the optimization opportunities for thread-local allocation).

jstimpfle 4y ago
> fread() is an inherently synchronous operation NOOOO it's not. It's the opposite. FILE streams are buffered, and the typical case is that fread() just copies from its internal buffer, where the data in the buffer had already been transferred from the OS at some earlier point. While it is not normally done, I think fread() could be implemented with a worker in a separate thread no problem. Think of it as an asynchronous API. > if it would have made sense to call fread() on the same file handle concurrently, you would have to add synchronization logic (externally or internally) before checking the error flag, and even then, there would be no way to signal two different errors occurring in two different concurrent fread() calls - one error would always overwrite the other. FILE streams are in fact protected by mutexes internally - it is safe to call e.g. fwrite() from different threads simultaneously (you can also circumvent the mutexes using e.g. fputc_unlocked() if you know it's safe to do so). Whether doing that makes sense beyond the quick debug printf is another question. FILE streams are buffered, so you never know which fread() call was the "source" of the error. And there is absolutely no point at all in trying to relate errors on a file handle to the n'th or (n+1)'th call. But if there was, the only thing you would have to do is to pass the function calls some context to discern between the calls. Essentially this is state segregation, which is not the point of the FILE API, which is why it doesn't do that. > Sure, the concurrency implementation ultimately has to somehow store errors, but this doesn't have to be exposed to the client API. Yes it does, unless you're willing to throw away the gains you got from concurrent operation (or with those fancy messy async frameworks, throw away at least a part of the gains). Retrieving the error or other return value requires synchronization. > imagine if malloc() had a corresponding merror() like fread() to indicate why an allocation failed You're not getting the point. malloc() API is fine for what it does (general purpose memory allocation in global context, which is not exactly what you should use in a well developed system, but like printf it gets you started...), and I'm not suggesting there should be a global merror() kind of thing. But checkout the following: mctx := malloc_context() buffer1 := malloc_buffer(mctx, size1) buffer2 := malloc_buffer(mctx, size2) for i in 0...100: buffers[i] := malloc_buffer(mctx, 42) if mctx.bad(): mctx.release() return # go on using the buffers... Maybe even like follows?: buffer_write(buffer1, data, 666) buffer_write(buffer2, data, 1337) if buffer1.bad(): # handle very specific problem? if mctx.bad(): # handle problem with any buffer? Better than checking each call individually, isn't it? Less code and faster to execute, because requests can be processed asynchronously. This is very much like the FILE API.