5 ms·

Why do people still continue to push the myth that using event driven style with callbacks and a huge unmaintainable mess is the only alternative to native thre

Why do people still continue to push the myth that using event driven style with callbacks and a huge unmaintainable mess is the only alternative to native threads? Userland threading has been around for a very long time. The only difference between using an event driven style and using a userland thread lib is that with userland threading the complexity of managing the state of the various computations is done by the thread lib instead of by you. The sooner people stop promoting the false dichotomy, the sooner people will realize every language should do concurrency as well as haskell, and the sooner that will actually happen.

jiggy2011 14y ago
Don't you trade for a different type of complexity in yield/resume? Also you will lost some of the scalability advantages because you have to store context for all of the idle fibers.
- batterseapower 14y ago
  In Haskell at least you do not have to explicitly yield. Instead, any call to the garbage collector is treated as a possible yield point (so it's possible to hog a CPU if you write a non-allocating loop). Not sure what to say about your second point: you do have to store context for the other fibers, but only in the same way you have to pay to store the context of an OS thread or pending callback.
- papsosouid 14y ago
  >Don't you trade for a different type of complexity in yield/resume? That depends on the library in question. Generally the event/callback style code people are writing are only "yielding" at calls to the non-blocking version of otherwise blocking functions, like read type functions, or accept. Most userland threading libs I've seen make the assumption that you want that, and so you don't need to yield yourself. If you did want that level of control, it is still simpler than the event style of achieving the same result. >Also you will lost some of the scalability advantages because you have to store context for all of the idle fibers. The overhead is small, especially if you know you don't need much stack and set the thread stack size lower.
xyzzy123 14y ago
In practical terms what happens is that your clients need to have a real good mental model of what will block and what won't. Also it turns out event driven programming is fine when you do it, but wait 'till you have curl handles and say mysql handles and blah 'andles and foo andles. Fork andles! Threading sucks. Mixing event loops also really sucks. The developer centric way of doing it (e.g. screw performance I just want it to be correct and look nice) is really to flip to message passing and separate "logical processes". That's our unfortunate state of the art. EDIT: I didn't mention that I think all the userland threading libs I know of are crap. Which ones do you think are good? P.S I'm not going to be sarcastic or critical about it, I would really like to have portable, usable LWP in C or C++.
- papsosouid 14y ago
  >In practical terms what happens is that your clients need to have a real good mental model of what will block and what won't. I don't understand. Who are "clients" in this context? You need to know what calls will block and what calls won't regardless of style (event vs thread) unless you are using OS threads or processes. >Threading sucks. Why? >Which ones do you think are good? The one that comes with ghc.
  - xyzzy123 14y ago
    I think perhaps the communications problem here is a difference in context. I'm coming from a C/C++ background. I get some Haskell but you'll have to explain specifics to me (I'm certainly willing to change my mind!). Are you using a framework, or developing one? Who gets to choose the concurrency strategy? As a framework provider, if you provide LWPs to ppl (like provide open and read and all that stuff for them) so they can write sequential code as if it were sequential, but it's actually nonblock, what happens in C/C++ is that when they need to integrate third-party stuff, life gets really complicated. Clients are the clients of the API I'm talking about, which is a hypothetical "lightweight thread" based API where everything just works as per original post. >>Threading sucks. >Why? Well, in an ideal model of computation we wouldn't need to synchronise things. This is the great attraction of likesay a node-based model where we run our stuff as synchronous events. Once you accept threading you need to care about who does what to which at what time. Also you need to figure out e.g. how your dependencies are thread-safe and under what conditions. You might have berkeley db where contexts are not generally thread-safe but handles can be if you set the right flags, for example. >>Which ones do you think are good? >The one that comes with ghc. That's cool :) I should probably learn more about it. The really cynical part of my mind is going "please tell me about this magical solution to integrating different event loops and concurrency models" and the less cynical part is open to your suggestions.
rdtsc 14y ago
> Why do people still continue to push the myth that using event driven style with callbacks and a huge unmaintainable mess is the only alternative to native threads? Performance. Before it used to be the famous C10K problem. And the best solution was a solution based on a select/epoll/kpoll loop. That was then (5-10 years ago). HAProxy is still build this way. Nginx is build this way as well. That works ok for smallish applications that are only IO bounded. Now there is also node.js. People want to use Javascript on the server with node.js (which is fine). But node.js is a single threaded application and asynchronous programming is the main way to do it. So all the advantages we are hearing about are well, people like to admire the technology they use because it makes them feel better about themselves (so it is not always causally the other way -- the pick technologies they like, sometimes they are forced to pick and then they force themselves to like it). > Userland threading has been around for a very long time. The only difference between using an event driven style and using a userland thread lib is that with userland threading the complexity of managing the state of the various computations is done by the thread lib instead of by you. Yes. And it turns out that in most practical application userland threading is actually built on an asynchronous IO select-like call. Think of Python's greenlet based concurrency mechanisms (gevent and eventlet). There there is the advantage of using threads and the advantage of using small per/thread storage and switching costs. So I think that this the most sane way to handle a highly concurrent, complex, application. Ok, I lied. The most sane way is to also provide data isolation & CPU based concurrency. For that you need Erlang (or maybe Elixir which runs on Erlang BEAM VM).
- xyzzy123 14y ago
  Well callback style is closest to the metal, greenlets look nicer (at the expense of this opaque dispatch loop) and IMHO both them cause pain over time :/ It's like a deal with the devil for performance... I guess I am disagreeing with the grandparent post. Maybe in Haskell it works (or is free), but C/C++, it works, but you can see you're paying for it.
  - papsosouid 14y ago
    Paying for it how? You get a simple model to work with and reason about, and the performance of an async event loop. Where does the problem arise?

jiggy2011 14y ago

Don't you trade for a different type of complexity in yield/resume? Also you will lost some of the scalability advantages because you have to store context for all of the idle fibers.

batterseapower 14y ago
In Haskell at least you do not have to explicitly yield. Instead, any call to the garbage collector is treated as a possible yield point (so it's possible to hog a CPU if you write a non-allocating loop). Not sure what to say about your second point: you do have to store context for the other fibers, but only in the same way you have to pay to store the context of an OS thread or pending callback.
papsosouid 14y ago
>Don't you trade for a different type of complexity in yield/resume? That depends on the library in question. Generally the event/callback style code people are writing are only "yielding" at calls to the non-blocking version of otherwise blocking functions, like read type functions, or accept. Most userland threading libs I've seen make the assumption that you want that, and so you don't need to yield yourself. If you did want that level of control, it is still simpler than the event style of achieving the same result. >Also you will lost some of the scalability advantages because you have to store context for all of the idle fibers. The overhead is small, especially if you know you don't need much stack and set the thread stack size lower.

xyzzy123 14y ago

In practical terms what happens is that your clients need to have a real good mental model of what will block and what won't. Also it turns out event driven programming is fine when you do it, but wait 'till you have curl handles and say mysql handles and blah 'andles and foo andles. Fork andles! Threading sucks. Mixing event loops also really sucks. The developer centric way of doing it (e.g. screw performance I just want it to be correct and look nice) is really to flip to message passing and separate "logical processes". That's our unfortunate state of the art. EDIT: I didn't mention that I think all the userland threading libs I know of are crap. Which ones do you think are good? P.S I'm not going to be sarcastic or critical about it, I would really like to have portable, usable LWP in C or C++.

papsosouid 14y ago
>In practical terms what happens is that your clients need to have a real good mental model of what will block and what won't. I don't understand. Who are "clients" in this context? You need to know what calls will block and what calls won't regardless of style (event vs thread) unless you are using OS threads or processes. >Threading sucks. Why? >Which ones do you think are good? The one that comes with ghc.
- xyzzy123 14y ago
  I think perhaps the communications problem here is a difference in context. I'm coming from a C/C++ background. I get some Haskell but you'll have to explain specifics to me (I'm certainly willing to change my mind!). Are you using a framework, or developing one? Who gets to choose the concurrency strategy? As a framework provider, if you provide LWPs to ppl (like provide open and read and all that stuff for them) so they can write sequential code as if it were sequential, but it's actually nonblock, what happens in C/C++ is that when they need to integrate third-party stuff, life gets really complicated. Clients are the clients of the API I'm talking about, which is a hypothetical "lightweight thread" based API where everything just works as per original post. >>Threading sucks. >Why? Well, in an ideal model of computation we wouldn't need to synchronise things. This is the great attraction of likesay a node-based model where we run our stuff as synchronous events. Once you accept threading you need to care about who does what to which at what time. Also you need to figure out e.g. how your dependencies are thread-safe and under what conditions. You might have berkeley db where contexts are not generally thread-safe but handles can be if you set the right flags, for example. >>Which ones do you think are good? >The one that comes with ghc. That's cool :) I should probably learn more about it. The really cynical part of my mind is going "please tell me about this magical solution to integrating different event loops and concurrency models" and the less cynical part is open to your suggestions.

rdtsc 14y ago

> Why do people still continue to push the myth that using event driven style with callbacks and a huge unmaintainable mess is the only alternative to native threads? Performance. Before it used to be the famous C10K problem. And the best solution was a solution based on a select/epoll/kpoll loop. That was then (5-10 years ago). HAProxy is still build this way. Nginx is build this way as well. That works ok for smallish applications that are only IO bounded. Now there is also node.js. People want to use Javascript on the server with node.js (which is fine). But node.js is a single threaded application and asynchronous programming is the main way to do it. So all the advantages we are hearing about are well, people like to admire the technology they use because it makes them feel better about themselves (so it is not always causally the other way -- the pick technologies they like, sometimes they are forced to pick and then they force themselves to like it). > Userland threading has been around for a very long time. The only difference between using an event driven style and using a userland thread lib is that with userland threading the complexity of managing the state of the various computations is done by the thread lib instead of by you. Yes. And it turns out that in most practical application userland threading is actually built on an asynchronous IO select-like call. Think of Python's greenlet based concurrency mechanisms (gevent and eventlet). There there is the advantage of using threads and the advantage of using small per/thread storage and switching costs. So I think that this the most sane way to handle a highly concurrent, complex, application. Ok, I lied. The most sane way is to also provide data isolation & CPU based concurrency. For that you need Erlang (or maybe Elixir which runs on Erlang BEAM VM).

xyzzy123 14y ago
Well callback style is closest to the metal, greenlets look nicer (at the expense of this opaque dispatch loop) and IMHO both them cause pain over time :/ It's like a deal with the devil for performance... I guess I am disagreeing with the grandparent post. Maybe in Haskell it works (or is free), but C/C++, it works, but you can see you're paying for it.
- papsosouid 14y ago
  Paying for it how? You get a simple model to work with and reason about, and the performance of an async event loop. Where does the problem arise?