5 ms·
The author intends the list to be read as 'rights' or 'entitlements' of user land code: > The philosophy here is to let the kernel do as much work as possible.
by eru 3y ago
The author intends the list to be read as 'rights' or 'entitlements' of user land code:
> The philosophy here is to let the kernel do as much work as possible. At my most pessimistic, I trust the kernel developers more than a trust myself. More practically, it’s stupid to spend CPU time locking around an operation that’s already atomic.
From a system's point of view, this list is also a list of demands on file system design. And you can tell that POSIX was invented before distributed computing really was a thing.
Some of the POSIX demands are really quite costly to satisfy in a distributed, networked filesystem, and often unnecessarily degrade performance.
- throwawaaarrgh 3y agoIt's impossible to reliably satisfy anything in a distributed networked anything. There aren't costs as much as tradeoffs. Reliable, fast, accurate: pick one.
- bmer 3y agoWhat's the difference between "reliable" and "accurate"? In any case, I have seen enough of these "A, B, C : pick N" (where 0 < N < 3) mantras shot down (usually by re-examining whether the constraints/assumptions of the original mantra apply in a specific situation, or just through plain algorithmic engineering) that whenever someone drops something like this, I mostly tune it out. In the case of distributed network <anything>, implementing programs in languages that help you manage important invariants automatically (and preferably not at run-time, but at compile-time) is key in creating robust systems that are both reliable, and fast. But if you have such tools, and you are trying to also meet a set of standards that were designed without any care for distributed computing, then...yeah.
- abwizz 3y agomaybe helpful https://en.wikipedia.org/wiki/CAP_theorem https://en.wikipedia.org/wiki/CAP_theorem
- robertlagrant 3y agoI think they probably are misremembering consistency, partition-tolerance, availability; pick 2.
- throwawaaarrgh 3y agoNo, I'm familiar with CAP. My theorem is different. Reliable: A system can be available but not reliable; if a machine is down 6 minutes every hour, it's 90% availabile but less than 1 hour reliable. A networked distributed system can be reliable but not accurate if you keep it online while it slowly accrues errors, such as the mentioned link/unlink race condition. The machine still works, it just has occasional corruption. It tends not to be very fast if it's reliable. Fast: You can do whatever you want quickly. Think UDP, or filesystems that don't commit their log journal frequently. Tends not to be reliable or accurate. Accurate: Whatever you do has no corrupted state. Think ACID. Most systems don't do this because it's expensive/complicated.
- robertlagrant 3y agoApologies - fair enough. I can't quite tell if that is a genuine set of three alternatives or not[0], but I sort of see your point. Couple of unsure places: > Think ACID. Most systems don't do this because it's expensive/complicated. Is this true? A lot of systems are built on databases. > Accurate: Whatever you do has no corrupted state How does accuracy work if the system is distributed and there's a partition, and two node get different data? Presumably it's reliable if they're different, as accuracy is a different item? > Fast: You can do whatever you want quickly. Think UDP, or filesystems that don't commit their log journal frequently. Tends not to be reliable or accurate. Why wouldn't this be reliable?
- throwawaaarrgh 3y agoMost databases aren't ACID compliant, and even if they can be, they're often used in a non-ACID way. > How does accuracy work if the system is distributed and there's a partition, and two node get different data? You don't allow writes during a partition. All nodes (or a quorum) must confirm on each write, and if that fails then a partition is detected. Everything that was accidentally written before you detected partition is thrown away. Usually you just have one writer node for simplicity. > Fast: > Why wouldn't this be reliable? If you prioritize reliability you will need a slower method to ensure reliability. TCP is slower than UDP so it can be reliable.
- eru 3y agoThat's a trite observation that doesn't actually _say_ anything. But formulated in that language, my comment becomes: POSIX picks less than useful trade-offs for distributed file systems. An extreme example: POSIX's atime gives you something you seldom need, but imposes a huge burden. That's such a bad trade-off, that approximately no-one implements the original spec, even if nothing is distributed at all.
- labster 3y agoIt’s not a bad idea, atime, just that our hardware never developed any tech where writes are as cheap as reads. Or even close. I wonder if that’s theoretically possible.
- eru 3y agoWhen they were coming up with 'atime' and thought it was a good idea, it was because with their tech writes were about as cheap as reads.
- yjftsjthsd-h 3y ago> just that our hardware never developed any tech where writes are as cheap as reads. Or even close. I wonder if that’s theoretically possible. I had it in my head that modern flash actually does read in a way that involves writing, and we just paper over that for the sake of the abstractions we use. Am I wildly misremembering?
- jlokier 3y agoYou might be thinking of DRAM, which does work that way. Reads discharge the memory array capacitance so the data has to be amplified and rewritten.
- bravetraveler 3y agoAs far as I know, reads are reads. Though, writes aren't quite so simple Sometimes blocks may have to be read and re-written elsewhere to make room... and this is what makes using TRIM and having cache on-disk so important.
- cryptonector 3y ago> Some of the POSIX demands are really quite costly to satisfy in a distributed, networked filesystem, and often unnecessarily degrade performance. Especially things like `stat(2)`, which complicate things for distributed filesystems like Lustre. Or the lack of a readdir()+stat().