4 ms·
> Blocking simply means that the specification does not guarantee an upper bound on the completion time. I don't think that's a commonly-accepted (or useful) d
by loeg 5y ago
> Blocking simply means that the specification does not guarantee an upper bound on the completion time.
I don't think that's a commonly-accepted (or useful) definition of "blocking." By that definition, getpid(2) is blocking.
> I think this is an instance of confusing what should be with what is.
Who is doing the confusing? I said "should be." Are you saying they're fast now but should be slow? Why?
> The reality is that system calls aren’t “fast” and they can’t portably or dynamically be guaranteed to be fast.
This isn't a portable program; it's a Linux program. The problem isn't that close can't be portably guaranteed to complete in some time bound; it's that Linux is adding what is essentially an extra usleep(100000), with very high probability, for the devfs synthetic filesystem in Linux.
This is entirely an own-goal; Linux has historically explicitly aimed to complete system calls quickly, when that does not break other functionality. It is a bug that can be fixed, e.g., with the proposed patch(es).
POSIX does not mandate that close blocks on anything other than removing the index from the fd table -- it's even allowed to leave associated IO in-flight and silently ignore errors. It makes little sense for a synthetic filesystem without real IO to block close so grossly.
- CyberRabbi 5y ago> I don't think that's a commonly-accepted (or useful) definition of "blocking." By that definition, getpid(2) is blocking. When it comes to expecting a specific duration, getpid() is blocking. If you run getpid() in a tight loop and then have performance issues you can’t reasonably blame the system. > This isn't a portable program; it's a Linux program But the interface is a portable interface > POSIX does not mandate that close blocks on anything other than removing the index from the fd table And what if the fd-table is a very large hash table with high collision rate? How do you then specify how quickly close() should complete? 1ms/open fd? 10ms/open fd? Etc. It should be clear that the problem here is that the author of the code had a faulty understanding of the system in which their code runs. Today the issue was close() just happened to be too “slow.” If the amount of input devices were higher, let’s say 2x more, then the same issue would have manifested even if close() were 2x “faster.” No matter how fast you make close() there is a situation in which this issue would manifest itself. I.e. the application has a design flaw.
- loeg 5y ago> Today the issue was close() just happened to be too “slow.” If the amount of input devices were higher, let’s say 2x more, then the same issue would have manifested even if close() were 2x “faster.” No matter how fast you make close() there is a situation in which this issue would manifest itself. Close, on an fd for which no asynchronous IO has occurred, should be 10000x faster, or more. It’s unlikely a user will have even 100 real input devices. I agree the algorithm leaves something to be desired, but the only reason it is user-visible is the performance bug in Linux. I’ve worked on performance in both userspace and the kernel and I think you’re fundamentally way off-base in a way we’ll never reconcile.
- CyberRabbi 5y ago> I agree the algorithm leaves something to be desired, but the only reason it is user-visible is the performance bug in Linux. The only reason it wasn’t user-visible was luck. Robust applications don’t depend on luck. Something tells me you’ll think twice before calling close() in a time-sensitive context in your future performance engineering endeavors. That’s because both you and I now know that no implementation of POSIX makes any guarantee on the runtime of close() nor will likely do so in the future. That’s just reality kicking in. Welcome to the club :)
- slavik81 5y agoThere's no guarantee for the runtime of any function. It's perfectly valid for the OS to swap your program instructions to disk, and then take seconds or even minutes to load it back. It's effectively impossible to avoid depending on what you call "luck". The OS does not provide nearly enough guarantees to build useful interactive applications without also depending on other reasonable performance expectations.
- CyberRabbi 5y ago> It's perfectly valid for the OS to swap your program instructions to disk, and then take seconds or even minutes to load it back. It’s not valid to swap your program instructions to disk if you call mlock() on your executable pages. Indeed, performance sensitive applications do just that. https://man7.org/linux/man-pages/man2/mlock.2.html https://man7.org/linux/man-pages/man2/mlock.2.html > It's effectively impossible to avoid depending on what you call "luck". The OS does not provide nearly enough guarantees to build useful interactive applications without also depending on other reasonable performance expectations. This is all self-evidently false. You likely wrote your comment on a POSIX-based interactive application. It just takes knowledge of how the system works and what the specifications are. Well-designed programs are hard to come by but they do exist.
- dagmx 5y agoCyberRabbi's definition of blocking is correct and what I've always seen commonly accepted. Blocking means you don't know how long it'll take, and you want to wait for it to finish. The only safe assumption is that you cannot guarantee how long it'll take. getpid is accurately therefore a blocking call. You don't know how long it'll take. You can profile and make best guesses, but you can never assuredly say how long it'll take.
- vanviegen 5y agoI'd say that the commonly accepted definition for a blocking call is one that may depend on I/O to complete, releasing control of the CPU core while waiting. By that definition, getpid() is definitely nonblocking, though it doesn't have an upper bound in execution time. POSIX does not offer hard realtime guarantees. close() in general would probably be blocking (as a filesystem may need to do I/O), but I'd expect it to behave nonblocking in most cases, especially when operating on virtual files opened read-only. Unfortunately, I don't think those kinds of behavioral details are documented.
- dagmx 5y agoA function that sleeps for 5 seconds is blocking. No IO involved. Blocking just means that you're blocking your current code till you return out of the called function. Anything else regarding a function call is an assumption unless you know the exact implementation.
- loeg 5y agoEvery operation in a non-RTOS is blocking by this definition, even local function calls that don’t enter the kernel, because the kernel may switch to another thread at any time. It’s utterly useless as a definition. Much more common is to divide system calls into ones that call depend on some external actor and those that don’t. Eg, recv() on a socket, blocking on a futex held by some other process, or waiting on IO to some disk controller. Getpid() is synchronous but does not block.
- CyberRabbi 5y ago