5 ms·

> Linux lacks a lot of core abstraction properties No, it's worse than that: It has too many of them, leading to a mess of special cases that you have to deal

by ori_b 7y ago

> Linux lacks a lot of core abstraction properties No, it's worse than that: It has too many of them, leading to a mess of special cases that you have to deal with. What happens when you have a socket in your file system, and you export it over NFS? Lacking abstraction properties is fixable -- you can add them. But removing them, especially if they're widely used, is incredibly hard.

pjmlp 7y ago
To be fair that is a general UNIX issue.
jstimpfle 7y ago
Making good abstractions is hard. On Unix I sometimes wish I could unwrap the stream abstraction, but nevertheless I think it is one of the few abstractions that have really stood the test of time. Why wouldn't a a socket exported over NFS just work seamlessly?
- ori_b 7y ago
  Because it's a kernel data structure thing that exists in the file system. The remote machine doesn't have access to it.
  - jstimpfle 7y ago
    That's true for regular files as well.
    - ori_b 7y ago
      No, the remote machine has access to file structures via NFS, which is complicated enough. NFS doesn't have protocol-level special cases for forwarding operations for the local sockets, handling setsockopt(), various socket ioctls -- which, mind you, are often machine specific, where the data sent in ioctl is ABI dependent. I'm not even sure how you'd do that sort of thing, since NFS is a stateless protocol. And then you would need to repeat the exercise for these special cases for all of the other special types of node, like /dev. What does it even mean to mmap the video memory of a video card on a remote machine? And then you'd need to fix the assumptions of all the software that assumes local semantics ("the connection doesn't drop, and poll() is always accurate"). On top of that, you'd need to run on a treadmill to add support for new ioctls. Do you really want to implement the server side of a protocol that can handle all of the complexity of all you can do on all file systems, with all ioctls, across all node types? How many implementations providing resources via this protocol do you think would exist?
      - jstimpfle 7y ago
        What does it mean to mmap a file on an NFS server? Isn't it a connection drop when a local process dies, too? What happens when a disk is suddenly removed? > On top of that, you'd need to run on a treadmill to add support for new ioctls. Absolutely, it'd be a lot of work. So it's a better idea to not implement many of these things and instead simply return an error.
        ori_b 7y ago
        > What does it mean to mmap a file on an NFS server? It means you have issues around synchronization and performance, if you use it as anything other than a private read only mapping. And some things are just impossible, like a shared memory ringbuffer. Which is exactly what you do with the memory you mmap from a video card: submit commands to the command ringbuffer. > So it's a better idea to not implement many of these things and instead simply return an error. And now you need to start writing multiple code paths in user code, testing which calls work and which don't, one of which will be broken due to lack of testing. And when you guess wrong at the needed operations, software often goes off the rails instead of failing gracefully. Failure modes like blindly retrying forever, or assuming the wrong state of the system and destroying data. Too many complicated abstractions break the ability to do interesting things with a system. It's death by a thousand edge cases. On plan 9, you have 9p. https://9p.io/magic/man2html/5/0intro https://9p.io/magic/man2html/5/0intro That, and process creation/control/namespace management, are the only ways to do anything with the system. There are few edge cases. Implementing a complete, correct server is a matter of hours, not weeks.
        jstimpfle 7y ago
        > like a [remote] shared memory ringbuffer. Technically just as possible, only very slow... Performance is abstracted out by the VFS. You need to stay sane through other measures, like having your software configured right, etc. > And now you need to start writing multiple code paths in user code I don't think the number of paths is increased. Any software should handle calls that fail - if only by bailing out. That's acceptable for any operation that just can't complete due to failed assumptions - whether it's about file permissions or that the resource must be "performant" / not on an NFS share, etc. > 9p. Now what is the point? How's that different or better? They actually are much more into sharing resources over the network... which means less possible assumptions about availability/reliability/performance. I doubt they can make the shared ringbuffer work better.

by ori_b 7y ago

pjmlp 7y ago

To be fair that is a general UNIX issue.

jstimpfle 7y ago

Making good abstractions is hard. On Unix I sometimes wish I could unwrap the stream abstraction, but nevertheless I think it is one of the few abstractions that have really stood the test of time. Why wouldn't a a socket exported over NFS just work seamlessly?

ori_b 7y ago
Because it's a kernel data structure thing that exists in the file system. The remote machine doesn't have access to it.
- jstimpfle 7y ago
  That's true for regular files as well.
  - ori_b 7y ago
    No, the remote machine has access to file structures via NFS, which is complicated enough. NFS doesn't have protocol-level special cases for forwarding operations for the local sockets, handling setsockopt(), various socket ioctls -- which, mind you, are often machine specific, where the data sent in ioctl is ABI dependent. I'm not even sure how you'd do that sort of thing, since NFS is a stateless protocol. And then you would need to repeat the exercise for these special cases for all of the other special types of node, like /dev. What does it even mean to mmap the video memory of a video card on a remote machine? And then you'd need to fix the assumptions of all the software that assumes local semantics ("the connection doesn't drop, and poll() is always accurate"). On top of that, you'd need to run on a treadmill to add support for new ioctls. Do you really want to implement the server side of a protocol that can handle all of the complexity of all you can do on all file systems, with all ioctls, across all node types? How many implementations providing resources via this protocol do you think would exist?
    - jstimpfle 7y ago
      What does it mean to mmap a file on an NFS server? Isn't it a connection drop when a local process dies, too? What happens when a disk is suddenly removed? > On top of that, you'd need to run on a treadmill to add support for new ioctls. Absolutely, it'd be a lot of work. So it's a better idea to not implement many of these things and instead simply return an error.
      - ori_b 7y ago
        > What does it mean to mmap a file on an NFS server? It means you have issues around synchronization and performance, if you use it as anything other than a private read only mapping. And some things are just impossible, like a shared memory ringbuffer. Which is exactly what you do with the memory you mmap from a video card: submit commands to the command ringbuffer. > So it's a better idea to not implement many of these things and instead simply return an error. And now you need to start writing multiple code paths in user code, testing which calls work and which don't, one of which will be broken due to lack of testing. And when you guess wrong at the needed operations, software often goes off the rails instead of failing gracefully. Failure modes like blindly retrying forever, or assuming the wrong state of the system and destroying data. Too many complicated abstractions break the ability to do interesting things with a system. It's death by a thousand edge cases. On plan 9, you have 9p. https://9p.io/magic/man2html/5/0intro https://9p.io/magic/man2html/5/0intro That, and process creation/control/namespace management, are the only ways to do anything with the system. There are few edge cases. Implementing a complete, correct server is a matter of hours, not weeks.
        jstimpfle 7y ago
        > like a [remote] shared memory ringbuffer. Technically just as possible, only very slow... Performance is abstracted out by the VFS. You need to stay sane through other measures, like having your software configured right, etc. > And now you need to start writing multiple code paths in user code I don't think the number of paths is increased. Any software should handle calls that fail - if only by bailing out. That's acceptable for any operation that just can't complete due to failed assumptions - whether it's about file permissions or that the resource must be "performant" / not on an NFS share, etc. > 9p. Now what is the point? How's that different or better? They actually are much more into sharing resources over the network... which means less possible assumptions about availability/reliability/performance. I doubt they can make the shared ringbuffer work better.