8 ms·

My (admittedly quick) reading of this is that the issues stem, primarily, if not exclusively, from using multiple file descriptors for the same file, necessitat

by mmt 8y ago

My (admittedly quick) reading of this is that the issues stem, primarily, if not exclusively, from using multiple file descriptors for the same file, necessitated by using/supporting threading for concurrency. If that's the case, then perhaps it's not so much that PAL was broken by design but that the original design merely didn't anticipate threading. I'm not sure how the advisory locking scheme compares in terms of timeline with threading support, but Unix file locking in general easily predates even SMP.

Hello71 8y ago
I don't think that's correct. According to Wikipedia, SMP dates to approximately the early 1960s, whereas according to the FreeBSD man pages, flock and fcntl were both introduced in 4.2BSD, released in 1983, two decades later. It's at most fair to say that Unix file locking came around the same time as Unix SMP.
- deleted 8y ago
  [deleted]
- throwaway76543 8y ago
  "Unix SMP" is not synonymous with a system threading interface -- processes give parallelism as well and predate the introduction of threading interfaces. The pthread interface came about in the 1990s. The unix interface is broken with respect to threads in many places besides this. Signals. Forking. Problems abound because the concept was bolted on decades later.
  - mmt 8y ago
    > processes give parallelism as well and predate the introduction of threading interfaces Right, but the reverse isn't true, which is what I was getting at. That is, availability of hardware parallelism was a prerequisite for anyone wanting threading in the first place.
    - MaxBarraclough 8y ago
      I don't get you. Are you saying that the only reason people would want threading capabilities is to exploit multicore/multiprocessor architectures? That isn't true. Having multiple threads for GUI applications makes sense for responsiveness even on a single-core machine.
      - mmt 8y ago
        > Are you saying that the only reason people would want threading capabilities is to exploit multicore/multiprocessor architectures? That's exactly what I was saying, though, specifically, kernel threading support. > Having multiple threads for GUI applications makes sense for responsiveness even on a single-core machine. I had never heard this before, but I'm hardly an expert. Is there something you could point to, ideally pre-MP (or pre-MP-popularity), that advocates for kernel thread support being needed for improved GUI responsiveness.
        MaxBarraclough 8y ago
        Strictly speaking you're right, it's not really about kernel threads. It's about concurrency. Proper pre-emptive threads in the kernel aren't the only way to achieve that: Java used to use 'green threads' rather than kernel threads. Its advice on using worker threads to keep your Swing application nice and responsive [0], would still have applied. (I believe this is still the case today for GTK programming in Python, which still uses green threads.) Most GUI toolkits use C-based languages and compile to native code. Cooperative multithreading isn't a popular model in C-family languages, but I can't see a good technical reason that it wouldn't work (after all, it boils down to something akin to green threads with finer control). A well-designed Qt app (in C++) will use worker threads to keep the UI thread responsive [1]. It doesn't matter if the worker threads run on a different core, or on the same core through time-division multiplexing. It's not a question of compute-throughput. What matters is that the UI thread doesn't get blocked for long. [0] https://docs.oracle.com/javase/tutorial/uiswing/concurrency/index.html https://docs.oracle.com/javase/tutorial/uiswing/concurrency/... [1] https://doc.qt.io/archives/qq/qq27-responsive-guis.html https://doc.qt.io/archives/qq/qq27-responsive-guis.html
  - kazinator 8y ago
    signals, forking, and simple things like chdir. A huge area that is broken w.r.t. threading is the whole internationalization stack: setlocale and all. What if I want to write a server such that each request is executed in a potentially different locale (because it services globally distributed users?)
    - throwaway76543 8y ago
      Ah yes, I was thinking about the system call interfaces but if we start looking at libc damn near nothing was reentrant until relatively recently (the late 90s and early 00s count as recent right? ... right?) Just take a gander at all those _r functions where an extra parameter needed to be added for reentrancy.
      - kazinator 8y ago
        Some of them turned out to be hyper-corrective duds, like readdir_r. No need for that one, since the buffer it needs can be allocated in the DIR object. You'd never want multiple threads concurrently doing readdir_r from the same DIR stream; there is no credible use case for it. (Or, maybe, optimization? Since it can store the dirent-s directly into some space designated by the caller, eliminating a copy operation from a program that wants to retain all those dirent-s.)
        throwaway76543 8y ago
        Is this because, as I recall, a DIR object is opaque while a FILE is not? Allowing the former to be expanded without breaking the API? edit: Nevermind, that shouldn't matter should it? The important thing is that they're both by reference.
        loeg 8y ago
        FILE can be opaque. Some implementations made the (arguable) mistake of implementing e.g. fileno() as a macro that accesses FILE internals, and this means they can't change that layout without breaking libc ABI of existing programs. But you can implement all POSIX stdio FILE APIs with an opaque FILE.
- mmt 8y ago
  As the sibling comment implies, I don't mean SMP anywhere in computing, I mean in the Unix world, which didn't even exist in the early 1960s. I'll grant that my early Unix knowledge is far from comprehensive, primarily skewed toward hardware at certain universities and in the database industry, as well as mostly BSD, with little from the "AT&T side" before SVR3. Given that, I'd be hard-pressed to name a Unix port that supported SMP prior to 1983. And I didn't mean to equate SMP with threads, only that thread support wouldn't make much sense without hardware concurrency, of which multi-processor hardware was an early practical example in the Unix world. As such, it was merely a (failed) rhetorical device to illustrate how designers of filesystem locking might not even have a notion such as intra-process concurrency on their mental radar. Even if you are, pedantically, correct, it doesn't refute my overall suggestion.
zAy0LfpBZLC8mAC 8y ago
Well, threading is a major problem area, but it is most definitely broken by design in any case, in that it has sort-of "spooky action at a distance" semantics, which is just completely uncomposable. Imagine you call a function from a library that simply scans a directory for files with a specific magic value, say ... if any other part of your program happens to hold a POSIX lock on any of those files (could even be another unrelated library), that will release those locks. That's just braindead.
- mmt 8y ago
  I can't claim to speak for the designers, but my intuition (having had my first Unix exposure in the early/mid-80s) is that their assumptions were along the lines that it would be used consistent with the part of the Unix philosophy stating: > Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features". This seems inconsistent with the (now totally common) programming styles that use multiple, unrelated libraries or have a single program do something that requires keeping files with locks open, while going off on a tangent to scan a directory. Back then, I suspect the standard practice would be to fork() if not outright delegate the task to existing tools like find/xargs. I certainly agree that hidden side effects like closing a duplicate fd causing all the locks to disappear is undesirable, but it might not even have occurred to them that anyone would care. Maybe they thought about it and the alternative would be too memory intensive (remember when memory was actually scarce?). It would be interesting if anyone had pointers to something like actual mailing list archives. Characterizing it as "just braindead" as uncharitable, when it's decades later, without at least being able to point to contemporaneous pressures and constraints.
  - zAy0LfpBZLC8mAC 8y ago
    Well, take a simpler case: A program that takes a lock and then reads a file specified by the user on the command line. There are tons of perfectly reasonable scenarios that are hilariously complicated to implement correctly, even ones that fit within the more traditional unix approach. Yes, maybe this is a result of some sort of tradeoff, but I would say it almost certainly was not a sensible tradeoff even back then. I don't see how using a sensible strategy could be much worse in terms of any resources used (I mean, really, it's a matter of where you put the pointers to the lock structures and where you call the automatic unlock handler), and the costs of using the semantics as they are now correctly are so high it's just not a useful abstraction really. If anything, this smells like one of those tradeoffs where a terrible abstraction that is incredibly difficult to use correctly was chosen simply because it was less code to write, while thus causing every user of the abstraction to have to write tons more code if they want to have correct code. Though if you ask me this looks much more like someone not thinking it through. Locks are for coordinating concurrency, the unit of concurrency is the process, so locks are held by processes. Oh, and we have to clean up locks that aren't released by the process! Well, let's do that when the file is closed ... OK, done!
iforgotpassword 8y ago
That design might be for portability, since pread() and pwrite() ought to be thread safe, so one global FD per file for your multi threaded app should be enough, at least on Linux and freebsd. It does however not help if some other part of your application "accidentally" opens that file too (some lib for example).
Aaargh20318 8y ago
> If that's the case, then perhaps it's not so much that PAL was broken by design but that the original design merely didn't anticipate threading. I don't think it's even that. It's simply because the API describes the contract between the kernel and processes, not the kernel and threads. It's a case of determining who is responsible for what. In this case the process is responsible for coordinating file locks with the kernel, not individual threads. From the kernel's perspective, 2 requests to the locking API from different threads still come from the same actor: the process. From the perspective of the locking API threads are a process's private business. A developer failing to understand the abstraction levels in an API is not the fault of the API's design, but it could be caused by documentation.
- Dylan16807 8y ago
  Being unable to control a lock from multiple threads isn't a great API either. Per file descriptor^H^Hion is closer to what I'd call ideal. It gives you a locking context that can map m:n to threads. Even if it is a bit fussy with things like fseek.

by mmt 8y ago

Hello71 8y ago

I don't think that's correct. According to Wikipedia, SMP dates to approximately the early 1960s, whereas according to the FreeBSD man pages, flock and fcntl were both introduced in 4.2BSD, released in 1983, two decades later. It's at most fair to say that Unix file locking came around the same time as Unix SMP.

deleted 8y ago
[deleted]
throwaway76543 8y ago
"Unix SMP" is not synonymous with a system threading interface -- processes give parallelism as well and predate the introduction of threading interfaces. The pthread interface came about in the 1990s. The unix interface is broken with respect to threads in many places besides this. Signals. Forking. Problems abound because the concept was bolted on decades later.
- mmt 8y ago
  > processes give parallelism as well and predate the introduction of threading interfaces Right, but the reverse isn't true, which is what I was getting at. That is, availability of hardware parallelism was a prerequisite for anyone wanting threading in the first place.
  - MaxBarraclough 8y ago
    I don't get you. Are you saying that the only reason people would want threading capabilities is to exploit multicore/multiprocessor architectures? That isn't true. Having multiple threads for GUI applications makes sense for responsiveness even on a single-core machine.
    - mmt 8y ago
      > Are you saying that the only reason people would want threading capabilities is to exploit multicore/multiprocessor architectures? That's exactly what I was saying, though, specifically, kernel threading support. > Having multiple threads for GUI applications makes sense for responsiveness even on a single-core machine. I had never heard this before, but I'm hardly an expert. Is there something you could point to, ideally pre-MP (or pre-MP-popularity), that advocates for kernel thread support being needed for improved GUI responsiveness.
      - MaxBarraclough 8y ago
        Strictly speaking you're right, it's not really about kernel threads. It's about concurrency. Proper pre-emptive threads in the kernel aren't the only way to achieve that: Java used to use 'green threads' rather than kernel threads. Its advice on using worker threads to keep your Swing application nice and responsive [0], would still have applied. (I believe this is still the case today for GTK programming in Python, which still uses green threads.) Most GUI toolkits use C-based languages and compile to native code. Cooperative multithreading isn't a popular model in C-family languages, but I can't see a good technical reason that it wouldn't work (after all, it boils down to something akin to green threads with finer control). A well-designed Qt app (in C++) will use worker threads to keep the UI thread responsive [1]. It doesn't matter if the worker threads run on a different core, or on the same core through time-division multiplexing. It's not a question of compute-throughput. What matters is that the UI thread doesn't get blocked for long. [0] https://docs.oracle.com/javase/tutorial/uiswing/concurrency/index.html https://docs.oracle.com/javase/tutorial/uiswing/concurrency/... [1] https://doc.qt.io/archives/qq/qq27-responsive-guis.html https://doc.qt.io/archives/qq/qq27-responsive-guis.html
- kazinator 8y ago
  signals, forking, and simple things like chdir. A huge area that is broken w.r.t. threading is the whole internationalization stack: setlocale and all. What if I want to write a server such that each request is executed in a potentially different locale (because it services globally distributed users?)
  - throwaway76543 8y ago
    Ah yes, I was thinking about the system call interfaces but if we start looking at libc damn near nothing was reentrant until relatively recently (the late 90s and early 00s count as recent right? ... right?) Just take a gander at all those _r functions where an extra parameter needed to be added for reentrancy.
    - kazinator 8y ago
      Some of them turned out to be hyper-corrective duds, like readdir_r. No need for that one, since the buffer it needs can be allocated in the DIR object. You'd never want multiple threads concurrently doing readdir_r from the same DIR stream; there is no credible use case for it. (Or, maybe, optimization? Since it can store the dirent-s directly into some space designated by the caller, eliminating a copy operation from a program that wants to retain all those dirent-s.)
      - throwaway76543 8y ago
        Is this because, as I recall, a DIR object is opaque while a FILE is not? Allowing the former to be expanded without breaking the API? edit: Nevermind, that shouldn't matter should it? The important thing is that they're both by reference.
        loeg 8y ago
        FILE can be opaque. Some implementations made the (arguable) mistake of implementing e.g. fileno() as a macro that accesses FILE internals, and this means they can't change that layout without breaking libc ABI of existing programs. But you can implement all POSIX stdio FILE APIs with an opaque FILE.
mmt 8y ago
As the sibling comment implies, I don't mean SMP anywhere in computing, I mean in the Unix world, which didn't even exist in the early 1960s. I'll grant that my early Unix knowledge is far from comprehensive, primarily skewed toward hardware at certain universities and in the database industry, as well as mostly BSD, with little from the "AT&T side" before SVR3. Given that, I'd be hard-pressed to name a Unix port that supported SMP prior to 1983. And I didn't mean to equate SMP with threads, only that thread support wouldn't make much sense without hardware concurrency, of which multi-processor hardware was an early practical example in the Unix world. As such, it was merely a (failed) rhetorical device to illustrate how designers of filesystem locking might not even have a notion such as intra-process concurrency on their mental radar. Even if you are, pedantically, correct, it doesn't refute my overall suggestion.

zAy0LfpBZLC8mAC 8y ago

Well, threading is a major problem area, but it is most definitely broken by design in any case, in that it has sort-of "spooky action at a distance" semantics, which is just completely uncomposable. Imagine you call a function from a library that simply scans a directory for files with a specific magic value, say ... if any other part of your program happens to hold a POSIX lock on any of those files (could even be another unrelated library), that will release those locks. That's just braindead.

mmt 8y ago
I can't claim to speak for the designers, but my intuition (having had my first Unix exposure in the early/mid-80s) is that their assumptions were along the lines that it would be used consistent with the part of the Unix philosophy stating: > Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features". This seems inconsistent with the (now totally common) programming styles that use multiple, unrelated libraries or have a single program do something that requires keeping files with locks open, while going off on a tangent to scan a directory. Back then, I suspect the standard practice would be to fork() if not outright delegate the task to existing tools like find/xargs. I certainly agree that hidden side effects like closing a duplicate fd causing all the locks to disappear is undesirable, but it might not even have occurred to them that anyone would care. Maybe they thought about it and the alternative would be too memory intensive (remember when memory was actually scarce?). It would be interesting if anyone had pointers to something like actual mailing list archives. Characterizing it as "just braindead" as uncharitable, when it's decades later, without at least being able to point to contemporaneous pressures and constraints.
- zAy0LfpBZLC8mAC 8y ago
  Well, take a simpler case: A program that takes a lock and then reads a file specified by the user on the command line. There are tons of perfectly reasonable scenarios that are hilariously complicated to implement correctly, even ones that fit within the more traditional unix approach. Yes, maybe this is a result of some sort of tradeoff, but I would say it almost certainly was not a sensible tradeoff even back then. I don't see how using a sensible strategy could be much worse in terms of any resources used (I mean, really, it's a matter of where you put the pointers to the lock structures and where you call the automatic unlock handler), and the costs of using the semantics as they are now correctly are so high it's just not a useful abstraction really. If anything, this smells like one of those tradeoffs where a terrible abstraction that is incredibly difficult to use correctly was chosen simply because it was less code to write, while thus causing every user of the abstraction to have to write tons more code if they want to have correct code. Though if you ask me this looks much more like someone not thinking it through. Locks are for coordinating concurrency, the unit of concurrency is the process, so locks are held by processes. Oh, and we have to clean up locks that aren't released by the process! Well, let's do that when the file is closed ... OK, done!

iforgotpassword 8y ago

That design might be for portability, since pread() and pwrite() ought to be thread safe, so one global FD per file for your multi threaded app should be enough, at least on Linux and freebsd. It does however not help if some other part of your application "accidentally" opens that file too (some lib for example).

Aaargh20318 8y ago

> If that's the case, then perhaps it's not so much that PAL was broken by design but that the original design merely didn't anticipate threading. I don't think it's even that. It's simply because the API describes the contract between the kernel and processes, not the kernel and threads. It's a case of determining who is responsible for what. In this case the process is responsible for coordinating file locks with the kernel, not individual threads. From the kernel's perspective, 2 requests to the locking API from different threads still come from the same actor: the process. From the perspective of the locking API threads are a process's private business. A developer failing to understand the abstraction levels in an API is not the fault of the API's design, but it could be caused by documentation.

Dylan16807 8y ago
Being unable to control a lock from multiple threads isn't a great API either. Per file descriptor^H^Hion is closer to what I'd call ideal. It gives you a locking context that can map m:n to threads. Even if it is a bit fussy with things like fseek.