8 ms·
Linus Torvalds on userspace filesystems
- barrkel 15y agoAn approach for this kind of problem: suppose you have a very simple language composed of simple, verifiable instructions, like: block read, block write, read-offset, write-offset, simple arithmetic, simple branching; you could compose programs for various file-system operations and pass them over to the kernel for execution en masse, rather than needing a chatty interface. This basic idea is something I've applied to a file format that stored objects, but where the costs of serializing and deserializing a whole object graph was prohibitive for a request/response round-trip. For any given object path, foo.bar[3].baz, a "program" could be compiled that could be interpreted over the file contents and the answer retrieved. All object paths were available for compilation ahead of time (it was a custom language, long story), so this approach could be far more efficient than any serialization story.
- pjscott 15y agoThat reminds me of Singularity OS, an experimental OS from Microsoft Research, which has no distinction between kernel and user space, thanks to writing everything in a variant of .NET bytecode. http://en.wikipedia.org/wiki/Singularity_(operating_system) http://en.wikipedia.org/wiki/Singularity_(operating_system)
- barrkel 15y agoThere have been Java OSes too, with the idea that verifiable bytecode does away with the need for "expensive" memory protection etc. Running everything in one memory space is error-prone, but it's the default way of the monolithic kernel. Having services delegated out to processes with separate memory spaces is architecturally more stable; a single bug in an obscure driver in a less actively maintained corner of the kernel shouldn't lead to a system-wide vulnerability. The approach I mention preserves the ability to keep things separate (with appropriate controls and verification at the boundary); the Singularity / Java OS approach seems rather different. Snoracle says JavaOS itself is obsolete, FWIW.
- btilly 15y agoYet another variant on the idea was the AS 400. It had no memory protection and didn't need it, since there was no way to address anything that you weren't supposed to. Given the impressive stability and security record of the AS 400, they had a point.
- barrkel 15y agoYes, when you have typed pointers at the hardware level, it can be even better than page protection.
- swdunlop 15y agoOnly because Snoracle hasn't ported Oracle to it, yet.
- derleth 15y agoI'm pretty sure Lisp OSes (you know, on the bytecoded Lisp Machines from companies such as Symbolics and TI) worked like that, and I know the CISC variants of the AS/400 architecture worked like that as well. (I don't know the RISC AS/400 or iSeries that well.) An interesting aspect of that is what it does to your security model: In the AS/400 world, where applications are compiled to bytecode and then to machine code, the software that compiles to machine code is trusted to never emit dangerous machine code, as there are no further checks on application behavior. In the Lisp Machine world, anyone who can load microcode is God. In Singularity OS, the .Net runtime is effectively the microcode and the same remarks about Lisp Machines apply.
- DrJokepu 15y agoI wonder how they really achieve that considering that the x86 CPU in protected mode definitely makes the distinction (rings 0 and 3, respectively)?
- sparky 15y agoSounds similar to the Untrusted Deterministic Functions used in exokernels to let userspace do god-knows-what to underlying hardware resources in an efficient but provably safe way, using a correct-by-construction language [1,2]. Another tool in this space is Native Client; any chance it could be adapted to allow an even smaller, safer x86 subset, so that you could give the kernel pre-compiled 'callbacks' that it could execute on, e.g., receiving a network packet, with 0 context switches? Probably not without a lot of work, since NaCl relies a lot on virtual memory for memory isolation, but it's an idea. [1] http://portal.acm.org/citation.cfm?id=266644 http://portal.acm.org/citation.cfm?id=266644 [2] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.120.4319&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.120...
- barrkel 15y agoNice links. The UDFs look like a very similar idea to mine, taken further and generalized; I make no claim to originality, it seemed obvious to my compiler-oriented eye.
- sparky 15y ago>> I make no claim to originality Oh, I didn't interpret it as such either; just some interesting related work :) Another peripherally-related idea is Active Messages [1,2]. I see it as the analog to UDFs or other rich executable messages for efficient node-to-node communication in large clusters. It's sort of like RPC if you squint hard enough; you send little executable messages that run on a remote node, and can send you back some data if you want. I guess the point is that having rich executable messages allows you to get around long physical latencies between sender and receiver, as well as more mundane intra-node delays due to userspace/kernelspace context switches. [1] http://www.cs.cornell.edu/home/tve/thesis/ http://www.cs.cornell.edu/home/tve/thesis/ (particularly chapter 4, it's a bit wordy but has lots of detail). [2] http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA204210&Location=U2&doc=GetTRDoc.pdf http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA204210&Locati... (there's lots of other published work on J-Machine from Dally's group, too)
- 15y ago
- michael_dorfman 15y agoIt's a bit amusing to see Linus, all these years after his famous dispute with Tanenbaum, still fighting microkernels. Personally, I think that the Minix source code shows that microkernels are quite viable, and that Linus's disdain is misguided.
- j_baker 15y agoI'm not an OS expert, so perhaps I'm just being an idiot, but doesn't that argument go both ways (and probably more to Linux)? I mean, couldn't one say that the Linux source code proves that microkernels aren't really necessary?
- __rkaup__ 15y agoI hear Hurd is making some progress...
- shadowfox 15y agoI guess no thread on Microkernels is complete without bringing up Hurd :P There are at least a few microkernel implementations beside Minix that are in use/more or less stable/shows some viability. (E.g. QNX, L4 family and to some extent EROS family) They have received nowhere as much attention as something like Linux. So the usability is not really up there.
- 15y ago
- gyom 15y agoI'm beginning to suspect that people report cases of Linus insulting people regardless of whether the discussion was interesting or not. Linus insults someone on a forum -> first page on HN !! Just a thought.
- deleted 15y ago[deleted]
- msbarnett 15y agoIt was never news, for just about any definition of 'news'.
- dkarl 15y agoYes, if the discussion has some fascinating relevance for the general HN readership, I can't figure it out. Linus doesn't come off very well, trying to settle a discussion with common sense and sarcasm while the other people on the thread use facts and examples to paint a more complicated picture. People who think that userspace filesystems are realistic for anything but toys are just misguided. I use sshfs all the time. Much of the software I use every day only needs to meet "toy" standards to be useful to me. What is Linus on about here?
- icebraining 15y agoYes, but would you rely on it as the filesystem for a public server? His point isn't that userspace filesystems are useless, but they are much slower, so the existence of a Fuse filesystem is not an argument for not including a kernel space filesystem in the kernel.
- swdunlop 15y agoLinus wasn't very clear, but the issue has more to do with the cost of context switches between the kernel and a userland process to support a userland filesystem. These switches are relatively expensive; a good example is the relative throughput of a LUKS/DM-CRYPT encrypted volume compared to a Truecrypt volume with equivalent cipher selections. LUKS' throughput is much higher, simply because it does not add at least two more kernel<->userspace transitions on every read and write.
- larrik 15y ago"That's like saying you should do a microkernel - it may sound nice on paper, but it's a damn stupid idea for people who care more about some idea than they care about reality." He just had to sneak in a dig at Tanenbaum.
- yuvadam 15y agoContext: http://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate http://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_deba... And the original thread circa 1992: http://groups.google.com/group/comp.os.minix/browse_thread/thread/c25870d7a41696d2/f447530d082cd95d?tvc=2 http://groups.google.com/group/comp.os.minix/browse_thread/t...
- shubber 15y agoAnd, intended or not, at Stallman
- btilly 15y agoI have it on good authority that Stallman now agrees that going with a microkernel was a mistake.
- vog 15y agoIs there any public statement or other public source for that statement? In particular, it would be very interesting to read about the concrete reasons which lead him to this conclusion.
- btilly 15y agoSorry, I got that from private communication with Thomas Bushnell, who was the principle architect on the project for a long time. His claim is that they had two choices, go with the microkernel, or start with the BSD code that was legally in the clear and rewrite the parts that were under a legal cloud at the time. Thomas preferred the latter, Stallman the former. As events proved, the BSD approach would have been fine (particularly since the legal issues eventually got cleared up), while the microkernel approach ran into much larger unexpected roadblocks than anticipated.
- j_baker 15y agoCorrect me if I'm wrong, but this is nothing new. We've known about these limitations on FUSE for quite some time, haven't we?
- walexander 15y agoAnecdotally, a project I worked on involved evaluating some encrypted file systems on linux and encFS (on FUSE) was at an order of magnitude worse performance than kernel solutions. Not to say it could never happen, but the tradeoff on performance is just no where near worth it right now.
- marshray 15y agoI'd suspected performance might not be the primary purpose when I came across the Python packages for implementing filesystems. Still, it seems like a lot of the interesting filesystem ideas have to do with non-root users authenticating to something else over some network connection. This type of thing is simply a more natural fit for user space. Keeping the high-level stateful protocol stuff out of the kernel is usually a good idea except when performance is all that matters (i.e. there will be full-time developers tuning it and cleaning up the inevitable security and crash bugs).
- lukeschlather 15y agoThe main thing I use fuse for is sshfs, where trying to optimize inside the kernel is unlikely to give noticeable gains relative to network latency. And Linus acknowledges: > fuse works fine if the thing being exported is some random low-use > interface to a fundamentally slow device
- leif 15y agoI don't care what he says, I'm still waiting for an inclusion of some kind of union mount functionality in mainline. I don't need anything fancy, just one ro dir and one rw dir. If he can't even get me that in-kernel, I'll stick with unionfs in fuse, thank you very much. Don't tell me about aufs, UnionMount, or overlayfs: aufs isn't up to date because it's not mainlined (for no good reason I can find), and UnionMount and overlayfs are too far out of mainline and too much in development to find reasonable packages. Linus: when you can back up your claims with working code, I'll start listening. ;-)
- jamespo 15y agoIsn't it down to you to prove him wrong rather than vice versa?
- leif 15y agoyou misunderstand me. I have a working solution, using fuse. if he thinks that's bad, it's on him to provide a correct in-kernel solution, then show me it's faster
- caf 15y agoRead the whole thread (or even just the immediate parent email, from Andrew Morton). Linus wasn't saying "don't use fuse for this", he was pushing back against a suggestion that the existence of the fuse implementation was a reason not to merge a kernel implementation.
- leif 15y agoI didn't read the parent, thanks.
- rwmj 15y agoThe current status of this seems to be summed up in this LWN article: https://lwn.net/Articles/447650/ https://lwn.net/Articles/447650/
- 15y ago
- andrewcooke 15y agothis reply is a bit more sane - http://www.spinics.net/lists/linux-fsdevel/msg46080.html http://www.spinics.net/lists/linux-fsdevel/msg46080.html glad i don't work with torvlads.
- pragmatic 15y agoIs there a website or book with the collected ideas/opinions of Linus and the design of linux? "that's like saying you should do a microkernel - it may sound nice on paper, but it's a damn stupid idea for people who care more about some idea than they care about reality." So I look up this microkernel thing and find this old debate: http://www.dina.dk/~abraham/Linus_vs_Tanenbaum.html http://www.dina.dk/~abraham/Linus_vs_Tanenbaum.html It would be great to have a resource that explains the what and the why behind the linux architecture. Also arguments, errr I mean debates, between high caliber smart people would be cool. There is a book "Just for Fun: The Story of an Accidental Revolutionary" that I ordered, but I have a feeling it doesn't go into the technical depth I would like. I could be wrong, just ordered it, will find out.
- uriel 15y agocough Plan 9[1] cough [1] http://doc.cat-v.org/plan_9/4th_edition/papers/names http://doc.cat-v.org/plan_9/4th_edition/papers/names
- borism 15y agoIt's sad Linus doesn't want to learn.
- sambeau 15y agoInteresting in light of Go: - New languages are needed for writing distributed/parallel applications `Needed', no. `Helpful', perhaps. The jury's still out.
- uriel 15y agoNote that the Plan 9 team at the time were already working on Alef ( http://doc.cat-v.org/plan_9/2nd_edition/papers/alef/ http://doc.cat-v.org/plan_9/2nd_edition/papers/alef/ ) and would later develop Limbo ( http://doc.cat-v.org/inferno/4th_edition/limbo_language/ http://doc.cat-v.org/inferno/4th_edition/limbo_language/ ) as the language for the Inferno distributed system ( http://doc.cat-v.org/inferno/ http://doc.cat-v.org/inferno/ )
- treetrouble 15y agosshfs isn't a toy
- 16s 15y agoIf you've ever implemented real-world code that solves a problem you'll find that there are some people (mostly who have not implemented much practical code but read a lot about it) who want to tell you how you should have done it. I think Linus got sick of hearing these suggestions and began defending his implementation decisions based on his experience and the running code.
- api 15y agohttp://www.opendedup.org/ http://www.opendedup.org/ Not a toy.
- masto 15y agoLinus is increasingly becoming a crazy old man. I find his diatribes largely irrelevant.