5 ms·
> Why do people still continue to push the myth that using event driven style with callbacks and a huge unmaintainable mess is the only alternative to native th
by rdtsc 14y ago
> Why do people still continue to push the myth that using event driven style with callbacks and a huge unmaintainable mess is the only alternative to native threads?
Performance.
Before it used to be the famous C10K problem. And the best solution was a solution based on a select/epoll/kpoll loop. That was then (5-10 years ago). HAProxy is still build this way. Nginx is build this way as well. That works ok for smallish applications that are only IO bounded.
Now there is also node.js. People want to use Javascript on the server with node.js (which is fine). But node.js is a single threaded application and asynchronous programming is the main way to do it. So all the advantages we are hearing about are well, people like to admire the technology they use because it makes them feel better about themselves (so it is not always causally the other way -- the pick technologies they like, sometimes they are forced to pick and then they force themselves to like it).
> Userland threading has been around for a very long time. The only difference between using an event driven style and using a userland thread lib is that with userland threading the complexity of managing the state of the various computations is done by the thread lib instead of by you.
Yes. And it turns out that in most practical application userland threading is actually built on an asynchronous IO select-like call. Think of Python's greenlet based concurrency mechanisms (gevent and eventlet). There there is the advantage of using threads and the advantage of using small per/thread storage and switching costs. So I think that this the most sane way to handle a highly concurrent, complex, application.
Ok, I lied. The most sane way is to also provide data isolation & CPU based concurrency. For that you need Erlang (or maybe Elixir which runs on Erlang BEAM VM).
- xyzzy123 14y agoWell callback style is closest to the metal, greenlets look nicer (at the expense of this opaque dispatch loop) and IMHO both them cause pain over time :/ It's like a deal with the devil for performance... I guess I am disagreeing with the grandparent post. Maybe in Haskell it works (or is free), but C/C++, it works, but you can see you're paying for it.
- papsosouid 14y agoPaying for it how? You get a simple model to work with and reason about, and the performance of an async event loop. Where does the problem arise?
- xyzzy123 14y agoOK. Can you block on a sysv semaphore and a socket at the same time? How do you integrate event loops using these two primitives at the same time? (EDIT: hint your runtime can't, because the kernel doesn't provide a way to do that. I guess I should be more explicit).
- papsosouid 14y agoI am not blocking or using event loops, that's the point. When I call read on a socket, the IO manager handles saving my context, doing the async read, yielding, and then restoring context and resuming when there's data to be read. The entire point is that green threads are equivalent to event loops, but taking advantage of abstraction to move the complexity away from the thousands of individual programmers, and onto the language authors.
- xyzzy123 14y ago(I was going to opt out of this, but I can spend another 1/2 hour on it_). What you are saying is, you're a client of someone else's nonblocking event loop. So all the classical stuff regarding nonblock io loops apply. E.g: http://talkingcode.co.uk/2008/12/02/haskell-gtk-and-multi-threading/ http://talkingcode.co.uk/2008/12/02/haskell-gtk-and-multi-th... But substitute Gtk for Qt, or anything where you're the sucker dealing with an external concurrency model or event loop. Hilariously, they end up having the same debate we are here, just in Haskell. Seriously, there's no pixie dust for this.
- papsosouid 14y agoThat link is a good example of exactly why we need to push for languages to take concurrency seriously. That's an example of how a haskell user has to deal with the issue when interfacing with a C lib that rolled their own event loop. For most languages people have to deal with that issue all the time, for their own code, entirely in that one language. I have no idea why you are talking about pixie dust. My point was that explicitly using event loops instead of abstracting it into a green thread library is dumb, and we need to start having higher expectations from languages rather than accepting backwards shit like node. Not "green threads are magic".
- rdtsc 14y ago> It's like a deal with the devil for performance Well put. You usually pay for it. Not familiar with Haskell too much to tell. It might provide the solution to trick the devil. The devil usually wins the bets though ;-)
- papsosouid 14y ago>Performance. Is identical. >And the best solution was a solution based on a select/epoll/kpoll loop But the underlying mechanism does not need to dictate what interface is exposed to be used. >And it turns out that in most practical application userland threading is actually built on an asynchronous IO select-like call. That is precisely the point. Presenting it as though the only options are "use native threads" and "use callbacks and event loops" is the myth. Use userland threads, you get the same performance and scalability as an event driven style, as it uses the exact same async calls under the hood, but you get a programming style that is readable and maintainable. Using an event loop is like rolling your own while loop out of setjmp/longjmp. Yes, the underlying mechanism of a loop is the same, but abstraction is pretty nice. >For that you need Erlang Why?
- rdtsc 14y ago> For that you need Erlang Because the other side of highly concurrent applications is increase complexity. Both threads and async-based _large_ applications become a mess, fast. No talking about HAProxy type small server, but complicated business rules, state machines and so on. Shared state is hard to reason in non-concurrent application, large concurrent ones, especially with shared data structures, become complicated. That is why light-weight isolated process based concurrency is the best IMHO. Erlang is at the for-front of that. It also has a completely concurrent garbage collector (it won't stop the world) since each process also has a private heap. But as the infomercial guy says, "But wait, that's not all". Add hot code reloading and being able to schedule all these light-weight processes on any number of CPU in an m:n fashion is what takes the cake. Aha, so is it all rainbows and unicorns. No. You pay for it by getting a sequential slow down. So if you benchmark mandelbrot or matrix multiplication on C++ vs Erlang, C++ will win. Safety and isolation doesn't come for free. So pick and choose based on what requirements you have. There is the syntax issue that people don't like so those are some trade-offs.
- papsosouid 14y agoNone of that explains why you need erlang. Any language can do message passing. The whole point is that we should be expecting these capabilities from every language, rather than pretending using an event loop is in any way reasonable.