8 ms·

JavaScript Concurrency Model and Event Loop

139 points by abhikandoi2000 9y ago

iso-8859-1 9y ago
This doesn't mention server-side JavaScript at all. There are lots of blocking routines in Node.js, the standard library is full of them. But it is interesting that it doesn't mention the oft repeated meme "JavaScript is single-threaded". Would be nice with an example showing parallel number-crunching without WebWorkers. Is that possible?
- girvo 9y ago
  I haven't read the article, but my simplistic understanding is that Node.js for your _app_ code is single-threaded, but the async IO that happens is done via an internal thread-pool. So, number crunching in your code will pretty much always block the rest of your code (but your existing IO routines will continue, until they complete, and their callbacks can't run until your number crunching is done). I am very tired, and not thinking straight, so I've probably messed up the explanation a bit. Lately, I've been playing with libev and thread-pools together in Nim; so you write async/await stuff everywhere with futures, including number crunching, which gets delegated out to the thread pool and yields back to your main thread when it's done. It's quite nice, but I forgot how complex this stuff gets!
  - elmigranto 9y ago
    > the async IO that happens is done via an internal thread-pool Not quite, since using threads internally (thread per IO operation) would defeat a purpose of async system, which is created specifically to not pay threading cost (context switches, etc.). In layman terms, kernel will simply invoke your C callback function when "stuff happens", and libuv propagates that to JS in platform-agnostic manner. There are still threads and thread pools in V8 and node for stuff like calculating PI (or, more realistically, password hashing and other number crunching), but that is mostly unrelated to async IO itself. Some google queries: select, poll, epoll, kqueue…
    - paulddraper 9y ago
      > using threads internally (thread per IO operation) would defeat a purpose of async system Yes and no. Having single threaded JS (no matter if runtime uses epoll) allows for JS concurrency concerns to be limited to a small number of places, i.e no pre-emptive. This makes concurrency in JS easier to reason about.
    - zbjornson 9y ago
      Are you sure the thread pool isn't used for IO? I thought that was it's primary use. This explanation is a bit old but I think still correct: http://stackoverflow.com/a/20346545 http://stackoverflow.com/a/20346545 and http://dailyjs.com/post/libuv http://dailyjs.com/post/libuv
      - elmigranto 9y ago
        That is true only for file system calls. Everything else is using non-blocking OS-provided mechanism. http://docs.libuv.org/en/v1.x/design.html http://docs.libuv.org/en/v1.x/design.html
- jordache 9y ago
  js is single threaded. parallel number crunching is not possible. You can fake it by kicking off multiple executions on a data set and chunk the work load across the data set, so they progress incrementally and equivalently. The data might be small enough (or browser fast enough) to give the impression of things being done in parallel.
  - munchor 9y ago
    OP referred to Web Workers[1] which do allow real parallel number crunching (at minimum on a different thread, might even be in a different core). It's a somewhat "new" feature of JavaScript but most modern browsers already implement it[2]. [1] https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers... [2] http://caniuse.com/#search=webworkers http://caniuse.com/#search=webworkers
    - jordache 9y ago
      do you read? OP asked for an example sans web workers. Plus web workers do not change the fact that JS is single threaded.
- tyingq 9y ago
  >Would be nice with an example showing parallel number-crunching without WebWorkers. Is that possible? Generators and yield for some types of crunching. Not parallel, but at least not a tight spinning loop.
- Pherdnut 9y ago
  Not without making calls to c-bound processes that plug into the event loop or two independent JS run-times talking to each other which is also possible on the web through iframes or in Node via spawning child processes. Single-threaded is desired behavior in the case of web ui and of Node, which is multi-process c/++ under the hood and JS responding to message queues. It's pretty simple. Functions block. The messaging queue that triggers function calls in event-driven, setTimeout, XHR, talking to another JS runtime cases doesn't. So yes, you have to be careful to not lock up your entire server by processing images with JS function calls in the same runtime. But the huge benefit you get out of this tradeoff is that you don't ever have to worry about two things trying to change something at the same time. Everybody is locked until your functions are finished and with async that doesn't typically take very long. The synchronous lib calls in Node are mostly for convenience. I use them all the time when doing simple file I/O stuff where I just want to read something from a file, do something to it and stick it in a var or a new file. In a server or more complex application, however, you always want to use async callbacks.
tejohnso 9y ago
> A very interesting property of the event loop model is that JavaScript, unlike a lot of other languages, never blocks. That seems like a dangerous, misleading statement.
- kyberias 9y ago
  Indeed, nothing in Javascript _language_ indicates that is has to be single-threaded. Then the author continues by comparing this with C and talks about threads. Again, nothing in C programming language indicates that it's multi-threaded.
  - tyingq 9y ago
    C is low level enough that you can mutex around non thread safe stuff, without big changes to the underlying implementation. I suppose the JS language spec itself doesn't rule out multi-threaded, but I imagine that would be a fairly big endeavor to re-do the guts of any implementation. The current limitations of web workers seems to confirm that.
    - saurik 9y ago
      With a concurrency-safe garbage collector (which isn't even hard to come by) I would argue the exact same is true of JavaScript. It really isn't a language detail and I would argue it would be a weekend project to get real multithreading working for JavaScript (as in, to the point where it works as well as in C, which is to say it might require tons of mutex locks everywhere).
      - pcwalton 9y ago
        It's already been implemented, in fact, in the venerable Rhino JS engine.
      - aardvark179 9y ago
        It can be done, and has been in Rhino, Nashorn, and graal.js. The problem isn't in the language per se but in the libraries provided as standard (mutexes, thread safe hashes/property access etc.) and in the implementation. It's much easier to build a JIT when you don't have to worry about concurrent threads of execution, and converting a single threaded JIT to a robust multithreaded one can take a lot of work.
Lerc 9y ago
In my time I have encountered quite a few descriptions of how the event loop works in JavaScript. What I would like to find is _why_ do they work like this. What is so important that this is the way it must be done. We live with browsers that lock up for a while until they figure out that some code has inadvertantly done a while(true). Promises and similar callback amelioration techniques took ages to turn up, when without the event model they might never have been required at all. Is there a reason for this? Is it a good reason?
- pavlov 9y ago
  The reason is that JavaScript runs within the same event loop as the browser GUI itself. The history of JavaScript is that it started as a sort of "quick and dirty" scripting language at Netscape. Creating a completely separate event model wasn't possible in those circumstances, so JavaScript ended up being shaped by the single-threaded lowest common denominator of Win32 and other platforms that Netscape ran on.
- skybrian 9y ago
  The tradeoff is that JavaScript programmers don't have to understand concurrency or deal with locking. Threads in Java (which became popular around the same time) are trickier both to implement and to use. Early multithreading was buggy, even at the OS level. You also don't gain a lot from multithreading in typical UI event handling. Most event handlers have to finish quickly anyway or the user will notice lag in updating the display. A UI that responds to events slowly and out of order would confuse users, even if it had no concurrency bugs (which is unlikely).
- jayflux 9y ago
  Because it's a great (quick and easy) way of making sure JavaScript and the browser's internal rendering functions don't run at the same time and cause data races. Asynchronous callbacks share the same queue as gui tasks, and those tasks can run whilst JS is waiting on an action or an I/O task to finish. As the UI uses the same thread as JS, if JS blocks, this would lock up the whole window (clicking on things would do nothing), so it's important functions don't take too long or suspend the thread. There was an experiment within servo to see if rendering and JS could run in separate threads entirely, but I'm not sure how that went? https://github.com/servo/servo/wiki/Design https://github.com/servo/servo/wiki/Design
twic 9y ago
One thing this doesn't touch on is the different handling of microtasks and macrotasks: https://jakearchibald.com/2015/tasks-microtasks-queues-and-schedules/ https://jakearchibald.com/2015/tasks-microtasks-queues-and-s... http://www.c-sharpcorner.com/article/overview-of-micro-tasks-in-knockoutjs/ http://www.c-sharpcorner.com/article/overview-of-micro-tasks...
- nevir 9y ago
  It's very important that JS devs be aware of the distinction there too - especially as Promises gain traction
- eriknstr 9y ago
  >In what order should the logs appear? Firefox 53 produced a log that matched what I expected but then I ran it a few more times and then it happened in another order which I then read was the order they were supposed to appear. >Microsoft Edge, Firefox 40, iOS Safari and desktop Safari 8.0.8 log setTimeout before promise1 and promise2 - although it appears to be a race condition. This is really weird, as Firefox 39 and Safari 8.0.7 get it consistently right. So Firefox 39 may have gotten it consistently right on his machine, Firefox 40 did not on his machine and Firefox 53 still does not on my machine.
- Touche 9y ago
  And within macro and micro tasks there are several different queues that have different priorities.
bsaul 9y ago
Side topic : The event loop architecture is also heavily used in iOS / Cocoa, although it is often not well understood by developper. Each thread has an event loop, including the main UI thread, and many weird behaviors can be understood better once you know a bit about them. Which made me wonder if a simple implementation of agent based concurrency in swift server couldn't simply be one agent - one event loop, plus a way to prevent direct calls across agent boundaries. Server is not iOS, but i suppose some language facilities should already be there and make it easier to implement. /side topic
- pacaro 9y ago
  Also win32/16 and X both have event loops at the core
  - toast0 9y ago
    Xlib has an event loop at the core, but if you write your client directly to the socket protocol, you can arrange it however you like.
    - pacaro 9y ago
      You could probably do the same with PeekMessage in win32, maybe call it periodically in a green threading library, or similar. But the most common mechanism is an event loop. I'm sure people have developed X applications that talk the protocol directly, but it isn't the typical way to do it.
  - Matthias247 9y ago
    The interesting question is more or less which GUI frameworks are not build around a singlethreaded event loop. QT and GTK/glib are too. Adobe AIR too. I think the Java frameworks (AWT, Swing) also.
- bschwindHN 9y ago
  Android also has event loops at its core but the Android API seems to do a pretty crappy job at exposing it in a nice way.
jlward4th 9y ago
Misleading title. There was actually nothing about concurrency in the article.
- thereIsCon 9y ago
  +1 If the article had used I/O (http request) as an example of event handling (message queuing), it would have added some concurrency context.
  - abhikandoi2000 9y ago
    I agree.
  - Pherdnut 9y ago
    setTimeout uses the same message queuing system as XHR, DOM events, or communicating with stuff that has its own stack like iframes or workers. That's why a function with a timeout of 0 will not fire until after it's enclosing function has popped. Functions block. Waiting for messages doesn't. That's the model for JS concurrency.
jordache 9y ago
practically speaking, I think for the most part, when folks talk about blocking execution, they are referring to the browser locking up while the long running process is running. To gracefully establish expectation in the user - A processing indicator will buy the application some time before the user gets impatient. The setTimeout 0 pattern is useful to delay the long running process enough for the browser to redraw the UI to throw up the processing indicator. One can further achieve another state in the processing indicator by leveraging time delayed CSS animation which runs in a thread that's parallel to JS. So you could use CSS to augment the content of the processing dialog while the blocking JS is running.
migstopheles 9y ago
This reminded me of Phil Roberts' excellent talk a couple of years ago at ScotlandJS: "Help, I'm stuck in an event loop". Well worth the 20 minute watch. https://vimeo.com/96425312 https://vimeo.com/96425312

This doesn't mention server-side JavaScript at all. There are lots of blocking routines in Node.js, the standard library is full of them. But it is interesting that it doesn't mention the oft repeated meme "JavaScript is single-threaded". Would be nice with an example showing parallel number-crunching without WebWorkers. Is that possible?

girvo 9y ago
I haven't read the article, but my simplistic understanding is that Node.js for your _app_ code is single-threaded, but the async IO that happens is done via an internal thread-pool. So, number crunching in your code will pretty much always block the rest of your code (but your existing IO routines will continue, until they complete, and their callbacks can't run until your number crunching is done). I am very tired, and not thinking straight, so I've probably messed up the explanation a bit. Lately, I've been playing with libev and thread-pools together in Nim; so you write async/await stuff everywhere with futures, including number crunching, which gets delegated out to the thread pool and yields back to your main thread when it's done. It's quite nice, but I forgot how complex this stuff gets!
- elmigranto 9y ago
  > the async IO that happens is done via an internal thread-pool Not quite, since using threads internally (thread per IO operation) would defeat a purpose of async system, which is created specifically to not pay threading cost (context switches, etc.). In layman terms, kernel will simply invoke your C callback function when "stuff happens", and libuv propagates that to JS in platform-agnostic manner. There are still threads and thread pools in V8 and node for stuff like calculating PI (or, more realistically, password hashing and other number crunching), but that is mostly unrelated to async IO itself. Some google queries: select, poll, epoll, kqueue…
  - paulddraper 9y ago
    > using threads internally (thread per IO operation) would defeat a purpose of async system Yes and no. Having single threaded JS (no matter if runtime uses epoll) allows for JS concurrency concerns to be limited to a small number of places, i.e no pre-emptive. This makes concurrency in JS easier to reason about.
  - zbjornson 9y ago
    Are you sure the thread pool isn't used for IO? I thought that was it's primary use. This explanation is a bit old but I think still correct: http://stackoverflow.com/a/20346545 http://stackoverflow.com/a/20346545 and http://dailyjs.com/post/libuv http://dailyjs.com/post/libuv
    - elmigranto 9y ago
      That is true only for file system calls. Everything else is using non-blocking OS-provided mechanism. http://docs.libuv.org/en/v1.x/design.html http://docs.libuv.org/en/v1.x/design.html
jordache 9y ago
js is single threaded. parallel number crunching is not possible. You can fake it by kicking off multiple executions on a data set and chunk the work load across the data set, so they progress incrementally and equivalently. The data might be small enough (or browser fast enough) to give the impression of things being done in parallel.
- munchor 9y ago
  OP referred to Web Workers[1] which do allow real parallel number crunching (at minimum on a different thread, might even be in a different core). It's a somewhat "new" feature of JavaScript but most modern browsers already implement it[2]. [1] https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers... [2] http://caniuse.com/#search=webworkers http://caniuse.com/#search=webworkers
  - jordache 9y ago
    do you read? OP asked for an example sans web workers. Plus web workers do not change the fact that JS is single threaded.
tyingq 9y ago
>Would be nice with an example showing parallel number-crunching without WebWorkers. Is that possible? Generators and yield for some types of crunching. Not parallel, but at least not a tight spinning loop.
Pherdnut 9y ago
Not without making calls to c-bound processes that plug into the event loop or two independent JS run-times talking to each other which is also possible on the web through iframes or in Node via spawning child processes. Single-threaded is desired behavior in the case of web ui and of Node, which is multi-process c/++ under the hood and JS responding to message queues. It's pretty simple. Functions block. The messaging queue that triggers function calls in event-driven, setTimeout, XHR, talking to another JS runtime cases doesn't. So yes, you have to be careful to not lock up your entire server by processing images with JS function calls in the same runtime. But the huge benefit you get out of this tradeoff is that you don't ever have to worry about two things trying to change something at the same time. Everybody is locked until your functions are finished and with async that doesn't typically take very long. The synchronous lib calls in Node are mostly for convenience. I use them all the time when doing simple file I/O stuff where I just want to read something from a file, do something to it and stick it in a var or a new file. In a server or more complex application, however, you always want to use async callbacks.

tejohnso 9y ago

> A very interesting property of the event loop model is that JavaScript, unlike a lot of other languages, never blocks. That seems like a dangerous, misleading statement.

kyberias 9y ago
Indeed, nothing in Javascript _language_ indicates that is has to be single-threaded. Then the author continues by comparing this with C and talks about threads. Again, nothing in C programming language indicates that it's multi-threaded.
- tyingq 9y ago
  C is low level enough that you can mutex around non thread safe stuff, without big changes to the underlying implementation. I suppose the JS language spec itself doesn't rule out multi-threaded, but I imagine that would be a fairly big endeavor to re-do the guts of any implementation. The current limitations of web workers seems to confirm that.
  - saurik 9y ago
    With a concurrency-safe garbage collector (which isn't even hard to come by) I would argue the exact same is true of JavaScript. It really isn't a language detail and I would argue it would be a weekend project to get real multithreading working for JavaScript (as in, to the point where it works as well as in C, which is to say it might require tons of mutex locks everywhere).
    - pcwalton 9y ago
      It's already been implemented, in fact, in the venerable Rhino JS engine.
    - aardvark179 9y ago
      It can be done, and has been in Rhino, Nashorn, and graal.js. The problem isn't in the language per se but in the libraries provided as standard (mutexes, thread safe hashes/property access etc.) and in the implementation. It's much easier to build a JIT when you don't have to worry about concurrent threads of execution, and converting a single threaded JIT to a robust multithreaded one can take a lot of work.

Lerc 9y ago

In my time I have encountered quite a few descriptions of how the event loop works in JavaScript. What I would like to find is _why_ do they work like this. What is so important that this is the way it must be done. We live with browsers that lock up for a while until they figure out that some code has inadvertantly done a while(true). Promises and similar callback amelioration techniques took ages to turn up, when without the event model they might never have been required at all. Is there a reason for this? Is it a good reason?

pavlov 9y ago
The reason is that JavaScript runs within the same event loop as the browser GUI itself. The history of JavaScript is that it started as a sort of "quick and dirty" scripting language at Netscape. Creating a completely separate event model wasn't possible in those circumstances, so JavaScript ended up being shaped by the single-threaded lowest common denominator of Win32 and other platforms that Netscape ran on.
skybrian 9y ago
The tradeoff is that JavaScript programmers don't have to understand concurrency or deal with locking. Threads in Java (which became popular around the same time) are trickier both to implement and to use. Early multithreading was buggy, even at the OS level. You also don't gain a lot from multithreading in typical UI event handling. Most event handlers have to finish quickly anyway or the user will notice lag in updating the display. A UI that responds to events slowly and out of order would confuse users, even if it had no concurrency bugs (which is unlikely).
jayflux 9y ago
Because it's a great (quick and easy) way of making sure JavaScript and the browser's internal rendering functions don't run at the same time and cause data races. Asynchronous callbacks share the same queue as gui tasks, and those tasks can run whilst JS is waiting on an action or an I/O task to finish. As the UI uses the same thread as JS, if JS blocks, this would lock up the whole window (clicking on things would do nothing), so it's important functions don't take too long or suspend the thread. There was an experiment within servo to see if rendering and JS could run in separate threads entirely, but I'm not sure how that went? https://github.com/servo/servo/wiki/Design https://github.com/servo/servo/wiki/Design

twic 9y ago

One thing this doesn't touch on is the different handling of microtasks and macrotasks: https://jakearchibald.com/2015/tasks-microtasks-queues-and-schedules/ https://jakearchibald.com/2015/tasks-microtasks-queues-and-s... http://www.c-sharpcorner.com/article/overview-of-micro-tasks-in-knockoutjs/ http://www.c-sharpcorner.com/article/overview-of-micro-tasks...

nevir 9y ago
It's very important that JS devs be aware of the distinction there too - especially as Promises gain traction
eriknstr 9y ago
>In what order should the logs appear? Firefox 53 produced a log that matched what I expected but then I ran it a few more times and then it happened in another order which I then read was the order they were supposed to appear. >Microsoft Edge, Firefox 40, iOS Safari and desktop Safari 8.0.8 log setTimeout before promise1 and promise2 - although it appears to be a race condition. This is really weird, as Firefox 39 and Safari 8.0.7 get it consistently right. So Firefox 39 may have gotten it consistently right on his machine, Firefox 40 did not on his machine and Firefox 53 still does not on my machine.
Touche 9y ago
And within macro and micro tasks there are several different queues that have different priorities.

bsaul 9y ago

Side topic : The event loop architecture is also heavily used in iOS / Cocoa, although it is often not well understood by developper. Each thread has an event loop, including the main UI thread, and many weird behaviors can be understood better once you know a bit about them. Which made me wonder if a simple implementation of agent based concurrency in swift server couldn't simply be one agent - one event loop, plus a way to prevent direct calls across agent boundaries. Server is not iOS, but i suppose some language facilities should already be there and make it easier to implement. /side topic

pacaro 9y ago
Also win32/16 and X both have event loops at the core
- toast0 9y ago
  Xlib has an event loop at the core, but if you write your client directly to the socket protocol, you can arrange it however you like.
  - pacaro 9y ago
    You could probably do the same with PeekMessage in win32, maybe call it periodically in a green threading library, or similar. But the most common mechanism is an event loop. I'm sure people have developed X applications that talk the protocol directly, but it isn't the typical way to do it.
- Matthias247 9y ago
  The interesting question is more or less which GUI frameworks are not build around a singlethreaded event loop. QT and GTK/glib are too. Adobe AIR too. I think the Java frameworks (AWT, Swing) also.
bschwindHN 9y ago
Android also has event loops at its core but the Android API seems to do a pretty crappy job at exposing it in a nice way.

jlward4th 9y ago

Misleading title. There was actually nothing about concurrency in the article.

thereIsCon 9y ago
+1 If the article had used I/O (http request) as an example of event handling (message queuing), it would have added some concurrency context.
- abhikandoi2000 9y ago
  I agree.
- Pherdnut 9y ago
  setTimeout uses the same message queuing system as XHR, DOM events, or communicating with stuff that has its own stack like iframes or workers. That's why a function with a timeout of 0 will not fire until after it's enclosing function has popped. Functions block. Waiting for messages doesn't. That's the model for JS concurrency.

jordache 9y ago

practically speaking, I think for the most part, when folks talk about blocking execution, they are referring to the browser locking up while the long running process is running. To gracefully establish expectation in the user - A processing indicator will buy the application some time before the user gets impatient. The setTimeout 0 pattern is useful to delay the long running process enough for the browser to redraw the UI to throw up the processing indicator. One can further achieve another state in the processing indicator by leveraging time delayed CSS animation which runs in a thread that's parallel to JS. So you could use CSS to augment the content of the processing dialog while the blocking JS is running.

migstopheles 9y ago

This reminded me of Phil Roberts' excellent talk a couple of years ago at ScotlandJS: "Help, I'm stuck in an event loop". Well worth the 20 minute watch. https://vimeo.com/96425312 https://vimeo.com/96425312