3 ms·
"The advantage of doing it in the userspace code is that it can be done in a simpler, more specialized way." No, the advantage of doing it in userspace code is
by jerf 16y ago
"The advantage of doing it in the userspace code is that it can be done in a simpler, more specialized way."
No, the advantage of doing it in userspace code is that you can paper over the deficiencies of the layers under the one you are working in with significant manual effort. Node.js introduced the concurrency problems when it selected Javascript as one of the layers, it doesn't get much credit in my mind for then solving them at great effort and with horrible damage done to the resulting program structures. The problems Node.js solves are not fundamental to programming, they are fundamental to Javascript.
Pick something like Erlang and the problem never exists in the first place. You don't have to paper over the deficiencies in the lower levels, because the levels below the code you're writing aren't deficient for concurrency in the first place.
I should point out that in general this is not necessarily a bad thing; alas, there's always some way your lower layers are deficient, and it's far worse when they make it impossible to paper over the problem. Still, you will never end up with simpler code. More specialized, oh my yes, but certainly not simpler. And the wisdom of picking a layer that is fundamentally deficient for your core target problem then papering over it seems pretty limited to me.
- sedachv 16y agoThis is what makes me want to work on operating systems, and in particular exokernel operating systems. There's no reason why OS threads should be expensive, other than hugepages.
- amalcon 16y agoPick something like Erlang and the problem never exists in the first place. You don't have to paper over the deficiencies in the lower levels, because the levels below the code you're writing aren't deficient for concurrency in the first place. Now, be fair: Erlang papers over the deficiencies of the lower layers (namely, the kernel) with significant manual effort. It's just that someone else has already gone to this effort. BEAM is an event-based server behind the scenes, with lots of syntactic sugar to make it look multithreaded. The PLT Scheme webserver is another example of this. I also didn't mean to suggest that the code itself is necessarily simpler. It can be, but (as the node.js example proves) it is certainly not always. The scheduling algorithm, on the other hand, is typically much simpler. Namely, it's typically "round-robin cooperative multitasking." No serious operating system since Windows for Workgroups has actually tried to use round-robin cooperative multitasking. All the other "thread stuff" is typically simplified as well: smaller stacks (if any at all), simpler context-switching code, that sort of thing. The actual encoding of the business logic? That depends on the problem.
- jerf 16y ago"Now, be fair: Erlang papers over the deficiencies of the lower layers (namely, the kernel) with significant manual effort." Since the various Xpolls were added to the kernel, I would disagree. Even based on a select loop the kernel has still been happy to do many things asynchronously for a long time now, it's just a bit of a klunky API. What stopped it from being easy was that C had no good concurrency story except "lots of manual effort", so nobody could really effectively use those things in C. (The kernel has actually had the "lots of manual effort" applied to it.) The vast majority of concurrency failure has been layered in at the high-level language point. There is a large class of "better C than C" languages (or the implementations thereof as the case may be) that simply shoot you before you can even think about fine-grained concurrency, CPython, Perl, Ruby, Javascript, etc. And another class of languages that permit it but don't really help much, like C(++), Java, C#, and anything where you might seriously use a semaphore directly. This is the core error that Node.js hype and its partisans make, mistaking the deficiencies of a set of languages for deficiencies in programming itself. It's not even in the OS; the OS doesn't mind threadlets/green threads/OS processes, it's all in the high-level languages. The kernel may not have provided you a threadlet system out of the box, but the Erlang VM isn't particularly fighting the kernel either, it's building on it. In this context, that's not particularly what I mean by deficiency. Missing things can be added, I'm talking about things where the underlying layers actively fight you. Also, yes, in some respect I'm still really responding more to the hype than directly to you. Saying that BEAM is an event-based server behind the scenes is basically the point I think needs to be made more clearly. You can be "asynchronous" and "event based" without having to embed the asnchronony and eventedness visibly in every function and nearly every line in the code base.
- wmf 16y agothe OS doesn't mind threadlets/green threads/OS processes Except for the blocking system calls and blocking page faults that block all your green threads.
- jerf 16y agoTo the extent those can be papered over, they have been by most good runtimes; the remainder is evenly applied to all languages and runtimes under discussion because nobody can escape from them at all, no matter how awesome the top layers are. This is what I was referencing when I offhandedly mentioned the fact that some layers can make papering over them actually impossible; if the kernel can not be convinced to do it with any series of syscalls, or worse yet the hardware itself can not be convinced, you lose, game over. Another example, Haskell's very strong type system can do a fairly good job of making really damned sure you don't escape from the system, which is useful for making an STM that is actually usable precisely because the smallest escape hatch tends to bring it down. But the downside is that a library or something can actually make it impossible to hack around a problem (with any reasonable degree of effort).