4 ms·

This [0] mail from Linus might shed some light over this. [0]: https://lkml.org/lkml/2009/3/25/632 https://lkml.org/lkml/2009/3/25/632

josephg 6y ago
Thanks. It looks like I have a deep philosophical disagreement with Linus: > Theory and practice sometimes clash. And when that happens, theory loses. Every single time. I think he's wrong. Compiler / PL theory works. Distributed systems theory (TLA+, etc) allow us to write correct software. I've used fuzzers to find and fix bugs in databases and concurrency algorithms, and the result is robust, working code. Theory can add complexity, but complexity added in one place can remove complexity from another. (Eg TCP is complex but reliable, in order delivery in the network layer makes the application layer much simpler and free of bugs.) Linus's argument is that users will keep writing open(); write(); close();. I think he's half right - thats the right level of complexity the API should have, with no tricky, buggy fsync barriers. But we've known how to write simple, correct APIs for modifying data like this for decades. The answer is transactions. Keeping the kernel as it is just keeps us in a dark age of buggy software.
- BlueTemplar 6y ago
  Wait, isn't TCP(/IP) the quick and dirty practical implementation that beat the much more theoretical OSI model ? (And the same thing happened to GNU/Linux, so Linus would know something about it !)
  - msla 6y ago
    > Wait, isn't TCP(/IP) the quick and dirty practical implementation that beat the much more theoretical OSI model? Yes, it is. It even has its own (four-layer) model but the IETF People aren't nearly as interested in layering as the OSI people were: Layering Considered Harmful https://tools.ietf.org/html/rfc3439#section-3 https://tools.ietf.org/html/rfc3439#section-3 Point being, this is why people who know history get a bit perturbed when people insist on shoving TCP/IP into the OSI Model, or, worse, insist that TCP/IP "implements" OSI somehow. OSI had its chance, it lost, and now we use POP and SMTP for email, not X.400.
  - josephg 6y ago
    Hah good question. When I was learning it in school, TCP/IP sure seemed complicated with frame numbers and window sizes and whatnot. It seems much more simple to me now. I suspect that everything sounds like "useless, complex theory" until you understand it and use it regularly, and then it mysteriously transforms into "obvious, simple practical knowledge". Are transactional databases useless theory or simple and pragmatic? Hash tables? Nagle's algorithm? CRDTs? SSA compilers? Your answer probably says more about your experience than it does about the technologies themselves. And thats why I don't trust Linus's rant. He's familiar with the garbage fire of the current posix filesystem APIs. I suspect deep down he's just arguing from status quo bias and doesn't like the "useless theory" of transactional filesystems because he hasn't seen the practical side of them and he hasn't thought through how they could be implemented efficiently in the kernel. And thats fine; but it might mean if we want transactional filesystems to happen in linux, it might make sense to start by implementing them in a BSD.
    - BlueTemplar 6y ago
      Funnily enough, I don't know that much about networking myself, I was mostly repeating what our networking teacher told us to illustrate the importance of practice vs theory. Another Linus example would be git, which he allegedly threw together in a couple of weeks, compared to pijul, which seems to have required years of work, but now is going to have a very hard time to catch up...
bsder 6y ago
The problem with that post is that it assumes that all application writers are writing crappy C. Linus has tunnel vision about this, and it wasn't true in 2009 and is becoming increasingly untrue. The people who need better tools to deal with this are the language implementers who almost always abstract over this. And since most language implementers target Windows and OS X as well as Linux, they are generally more aware of what better could look like than Linus.
- josephg 6y ago
  > The people who need better tools to deal with this are the language implementers who almost always abstract over this. Do they? What languages abstract away the filesystem or the OS's file abstraction?
  - bsder 6y ago
    Java? Python? Common Lisp? Others? I'm not dropping to direct system calls of each OS, so there is clearly some level of abstraction.
    - josephg 6y ago
      Barely. I don’t know about clisp but for the most part Java and python just provide thin shims over the posix syscalls. They don’t give you atomic transactional filesystem semantics.

josephg 6y ago

Thanks. It looks like I have a deep philosophical disagreement with Linus: > Theory and practice sometimes clash. And when that happens, theory loses. Every single time. I think he's wrong. Compiler / PL theory works. Distributed systems theory (TLA+, etc) allow us to write correct software. I've used fuzzers to find and fix bugs in databases and concurrency algorithms, and the result is robust, working code. Theory can add complexity, but complexity added in one place can remove complexity from another. (Eg TCP is complex but reliable, in order delivery in the network layer makes the application layer much simpler and free of bugs.) Linus's argument is that users will keep writing open(); write(); close();. I think he's half right - thats the right level of complexity the API should have, with no tricky, buggy fsync barriers. But we've known how to write simple, correct APIs for modifying data like this for decades. The answer is transactions. Keeping the kernel as it is just keeps us in a dark age of buggy software.

BlueTemplar 6y ago
Wait, isn't TCP(/IP) the quick and dirty practical implementation that beat the much more theoretical OSI model ? (And the same thing happened to GNU/Linux, so Linus would know something about it !)
- msla 6y ago
  > Wait, isn't TCP(/IP) the quick and dirty practical implementation that beat the much more theoretical OSI model? Yes, it is. It even has its own (four-layer) model but the IETF People aren't nearly as interested in layering as the OSI people were: Layering Considered Harmful https://tools.ietf.org/html/rfc3439#section-3 https://tools.ietf.org/html/rfc3439#section-3 Point being, this is why people who know history get a bit perturbed when people insist on shoving TCP/IP into the OSI Model, or, worse, insist that TCP/IP "implements" OSI somehow. OSI had its chance, it lost, and now we use POP and SMTP for email, not X.400.
- josephg 6y ago
  Hah good question. When I was learning it in school, TCP/IP sure seemed complicated with frame numbers and window sizes and whatnot. It seems much more simple to me now. I suspect that everything sounds like "useless, complex theory" until you understand it and use it regularly, and then it mysteriously transforms into "obvious, simple practical knowledge". Are transactional databases useless theory or simple and pragmatic? Hash tables? Nagle's algorithm? CRDTs? SSA compilers? Your answer probably says more about your experience than it does about the technologies themselves. And thats why I don't trust Linus's rant. He's familiar with the garbage fire of the current posix filesystem APIs. I suspect deep down he's just arguing from status quo bias and doesn't like the "useless theory" of transactional filesystems because he hasn't seen the practical side of them and he hasn't thought through how they could be implemented efficiently in the kernel. And thats fine; but it might mean if we want transactional filesystems to happen in linux, it might make sense to start by implementing them in a BSD.
  - BlueTemplar 6y ago
    Funnily enough, I don't know that much about networking myself, I was mostly repeating what our networking teacher told us to illustrate the importance of practice vs theory. Another Linus example would be git, which he allegedly threw together in a couple of weeks, compared to pijul, which seems to have required years of work, but now is going to have a very hard time to catch up...

bsder 6y ago

The problem with that post is that it assumes that all application writers are writing crappy C. Linus has tunnel vision about this, and it wasn't true in 2009 and is becoming increasingly untrue. The people who need better tools to deal with this are the language implementers who almost always abstract over this. And since most language implementers target Windows and OS X as well as Linux, they are generally more aware of what better could look like than Linus.

josephg 6y ago
> The people who need better tools to deal with this are the language implementers who almost always abstract over this. Do they? What languages abstract away the filesystem or the OS's file abstraction?
- bsder 6y ago
  Java? Python? Common Lisp? Others? I'm not dropping to direct system calls of each OS, so there is clearly some level of abstraction.
  - josephg 6y ago
    Barely. I don’t know about clisp but for the most part Java and python just provide thin shims over the posix syscalls. They don’t give you atomic transactional filesystem semantics.