4 ms·

> Many are forgetting the initial reason node.js became popular. Consider the popular server-side landscape before node.js. It was dominated by Java, Python, et

> Many are forgetting the initial reason node.js became popular. Consider the popular server-side landscape before node.js. It was dominated by Java, Python, etc. The primary way these ecosystems handle concurrency is OS-level threading. There was nothing else in the popular languages. What? I am sorry but this is not true. Coroutines have been a concept for...well a long time: https://en.wikipedia.org/wiki/Coroutine https://en.wikipedia.org/wiki/Coroutine Green threads also have been around for a while. > but these all had a critical flaw, which is the rest of the ecosystem didn't support it. Basically every library, almost all existing code used the threading model. And when you mix threaded libraries with a non-blocking I/O server, it completely falls apart because the threaded code blocks. I am not sure how you can say this while using Python as an example. Python has the GIL which, typically viewed as a limitation, actually prevents the situation your assertion suggests. In fact, one of the largest barriers to removing the GIL is the wealth of libraries and C-extensions to Python that implicitly rely on the thread safety guarantees/limitations of the GIL (as covered by Larry Hastings as part of his first Gilectomy talk). To be clear, there are reasons Node became popular, but let's not pretend that it brought some sort of concurrency revolution to the "server-side landscape" in 2009. Edit: Heck, NGINX was an asynchronous, event driven web engine released in 2004 and written in C: https://en.wikipedia.org/wiki/Nginx https://en.wikipedia.org/wiki/Nginx

AgentME 9y ago
>Coroutines have been a concept for...well a long time: ... Green threads also have been around for a while. What web frameworks were built around using them? Was there a good ecosystem of libraries that were easy to use with the web framework without introducing blocking I/O? >I am not sure how you can say this while using Python as an example. Python has the GIL which, typically viewed as a limitation, actually prevents the situation your assertion suggests. In fact, one of the largest barriers to removing the GIL is the wealth of libraries and C-extensions to Python that implicitly rely on the thread safety guarantees/limitations of the GIL (as covered by Larry Hastings as part of his first Gilectomy talk). Aren't all python threads backed by OS threads? And pointing out that threading has been used by too many Python libraries to change supports the previous comment's assertion that non-blocking I/O was hard to use exclusively in languages like Python because too many libraries used blocking I/O. >Edit: Heck, NGINX was an asynchronous, event driven web engine released in 2004 and written in C: Nginx is pretty specialized. Not really sure it's useful to compare it to a programming language / web framework with an ecosystem of compatible libraries like PHP, Ruby on Rails, django, nodejs, etc.
- btown 9y ago
  I often wonder why Python's gevent didn't become more popular. It would monkey patch the socket libraries to yield to an event loop, allowing you to magically use blocking code and libraries with practically zero code changes, and zero callback hell. I imagine it had to do with marketing - when Node was released, everyone thought the spaghetti horror that was Twisted was the only other event-loop-based game in town. Ironically, now that we have async/await to replace nested callbacks, Node finally has the code readability that gevent users took for granted all this time.
  - bastawhiz 9y ago
    > I often wonder why Python's gevent didn't become more popular. Quite honestly, it wasn't very good. It worked magically until the magic ran out. Mysterious breakages caused by stuff deep in gevent interacting badly with other libraries hampered my ability to use it effectively. For instance, you couldn't use it with python-requests directly, you need to use another library that makes requests compatible, unless you roll your own magic, which was often very complicated. And, at least in my experience, gevent stuff tends not to work with Python 3. Now that asyncio is a thing, I'm glad I never have to debug gevent-related problems anymore.
cstrahan 9y ago
It should also be noted that there's another statically typed language that provides multiplexed M:N threading (aka "green threads") and a programming style that looks synchronous while actually using asynchronous file and network IO behind the scenes (so all of the perf benefits of JavaScript without the awkward, nested callbacks and/or promises). The language I'm referring to is none other than Haskell (which has been around in some form since 1990). Here's the earliest paper I know of detailing the internals of the Glasgow Haskell Compiler's IO event handling system (from 2010, I believe): https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36841.pdf https://static.googleusercontent.com/media/research.google.c... And here's a more recent update: http://haskell.cs.yale.edu/wp-content/uploads/2013/08/hask035-voellmy.pdf http://haskell.cs.yale.edu/wp-content/uploads/2013/08/hask03... (Just in case of link-rot, the titles are respectively: "Scalable I/O Event Handling for GHC", "Mio: A High-Performance Multicore IO Manager for GHC") Here's a short example: main = do putStrLn "Hello, what's your name?" name <- getLine putStrLn ("Hey " ++ name ++ ", you rock!") Unfamiliar syntax aside, there's not a callback in sight -- it looks just like standard, imperative, synchronous code. It drives me crazy when people tout Node's approach to concurrency as being ideal, as if callbacks are a necessity. They're not. Haskell, among other languages, is an existence proof that you can have lightweight threads and asynchronous IO without callback spaghetti. And I haven't even gone into Haskell's threaded parallelism, Software Transactional Memory, etc.
- Tobani 9y ago
  Your still missing the point. It's not that it was possible to do so. It's that you could quickly write web services idomatically that did so by default. Your example shows a simple hello world and doesn't speak to the point being made. I am sure you can get a more relevant example going in Scotty, but it appears that it didn't show up until 2012.
  - cstrahan 9y ago
    > Your still missing the point. No, I'm not missing the point. > Your example shows a simple hello world and doesn't speak to the point being made. Hmm... Let's do an analysis: * My example demonstrates reading from a given file descriptor (FD 0 -- stdin -- in this case), and that reading happens asynchronously (as I had pointed out, and even provided white papers describing how this works). Again, this is despite the syntax looking like a naive, synchronous program. The GHC Run Time System handles asynchronously reading from the FD and scheduling threads as appropriate such that OS threads aren't tied up waiting on what would otherwise be a synchronous read(2). * As a bonus, it also demonstrates writing to standard out. Now, one could take exception to the fact that my example doesn't give any evidence that similar asynchrony should be expected in a web stack. One would be wrong: as anyone with, say, as little as a year of experience writing web applications on POSIX systems should know, the Berkeley sockets API exposes sockets as file descriptors, so the same generic async event handling system in GHC's RTS applies equally well to networked IO. GHC and Node (via libuv) uses similar primitive kernel APIs at the end of the day, the only difference being that Haskell doesn't require that you reify the callbacks in the source language. Idiomatic? Check. By default? Check. So no, I think I do get the point, but if you still disagree, I'm all ears. If my tone sounds harsh, it's because I spent the time to provide links to two papers and even gave a small example of what the syntax would look like for a minimal program that demonstrates IO. In turn, you're telling me that I'm missing the point. I would argue that, if you read either of those two papers, and you knew about the implications with respect to networked IO (if you didn't, you could easily research the Berkeley sockets API, kqueue(2), poll(2), etc -- all mentioned in the papers), you would see what I'm getting at with my example code and the papers referenced. I can only posit that you haven't read the papers, or if you did, you must have not done so in earnest. Hopefully you can see how I would be frustrated when I've put in time to research this space, and when I share something, I'm met with what (in appearance) amounts to a dissimilar level of diligence and, ultimately, an empty dismissal. If something else is going on, I would be more than happy to issue an apology. > I am sure you can get a more relevant example going in Scotty, but it appears that it didn't show up until 2012. Open source web frameworks for Haskell go back to (at least as far as) 2009. Happstack: (2009): https://hackage.haskell.org/package/happstack-server-0.1 https://hackage.haskell.org/package/happstack-server-0.1 Yesod (2010): https://hackage.haskell.org/package/yesod-0.0.0 https://hackage.haskell.org/package/yesod-0.0.0 Snap (2010): https://hackage.haskell.org/package/snap-0.3.0 https://hackage.haskell.org/package/snap-0.3.0 I didn't provide a "hello world"-like example of a web app because my command-line example only requires a trivial inference on the part of the reader to realize that the same asynchronous treatment of FDs there applies equally well to sockets, and thus web app programming in general; I deemed a full fledged web app example as more complicated than necessary to get my point across. For what it's worth, here's an example in Happstack, straight from the documentation: module Main where import Happstack.Server (nullConf, simpleHTTP, toResponse, ok) main :: IO () main = simpleHTTP nullConf $ ok "Hello, World!" And ... $ curl http://localhost:8000/ Hello, World! Also, a note on tense: > It's that you could quickly write web services idomatically that did so by default. I wasn't questioning the rationale for using Node back in 2009. My point is this: eight years have passed since Node was released, and alternative solutions have since been released. Somehow the Node community (and the greater web dev community in general) are oblivious to these alternative approaches. I suspect this is because most web devs see the callbacks as evidence that IO is asynchronous in Node, and at the same time lack the requisite experience to intuit how another language might implement asynchronous IO while providing a syntax that looks like the usual synchronous model. Now, they could actually read the research papers put out, but they don't, and so everyone keeps going on about the technological revolution that is Node and its approach to IO, oblivious to alternative solutions that (arguably) provide a more convenient programming model (and is, if we throw in support for multiple threads, more computationally powerful, too). Such laziness and/or lack of curiosity is pervasive in the industry, and that's frustrating.
esfandia 9y ago
> Edit: Heck, NGINX was an asynchronous, event driven web engine released in 2004 and written in C: In the interview, he mentions himself that Node was inspired by Nginx.
ricardobeat 9y ago
Remember the "C10K challenge"? Show me what else could achieve those ten thousand concurrent connections back then (hint: no python).
- ungamed 9y ago
  Erlang -easily- acheived the C10k challenge back then.

AgentME 9y ago

>Coroutines have been a concept for...well a long time: ... Green threads also have been around for a while. What web frameworks were built around using them? Was there a good ecosystem of libraries that were easy to use with the web framework without introducing blocking I/O? >I am not sure how you can say this while using Python as an example. Python has the GIL which, typically viewed as a limitation, actually prevents the situation your assertion suggests. In fact, one of the largest barriers to removing the GIL is the wealth of libraries and C-extensions to Python that implicitly rely on the thread safety guarantees/limitations of the GIL (as covered by Larry Hastings as part of his first Gilectomy talk). Aren't all python threads backed by OS threads? And pointing out that threading has been used by too many Python libraries to change supports the previous comment's assertion that non-blocking I/O was hard to use exclusively in languages like Python because too many libraries used blocking I/O. >Edit: Heck, NGINX was an asynchronous, event driven web engine released in 2004 and written in C: Nginx is pretty specialized. Not really sure it's useful to compare it to a programming language / web framework with an ecosystem of compatible libraries like PHP, Ruby on Rails, django, nodejs, etc.

btown 9y ago
I often wonder why Python's gevent didn't become more popular. It would monkey patch the socket libraries to yield to an event loop, allowing you to magically use blocking code and libraries with practically zero code changes, and zero callback hell. I imagine it had to do with marketing - when Node was released, everyone thought the spaghetti horror that was Twisted was the only other event-loop-based game in town. Ironically, now that we have async/await to replace nested callbacks, Node finally has the code readability that gevent users took for granted all this time.
- bastawhiz 9y ago
  > I often wonder why Python's gevent didn't become more popular. Quite honestly, it wasn't very good. It worked magically until the magic ran out. Mysterious breakages caused by stuff deep in gevent interacting badly with other libraries hampered my ability to use it effectively. For instance, you couldn't use it with python-requests directly, you need to use another library that makes requests compatible, unless you roll your own magic, which was often very complicated. And, at least in my experience, gevent stuff tends not to work with Python 3. Now that asyncio is a thing, I'm glad I never have to debug gevent-related problems anymore.

cstrahan 9y ago

It should also be noted that there's another statically typed language that provides multiplexed M:N threading (aka "green threads") and a programming style that looks synchronous while actually using asynchronous file and network IO behind the scenes (so all of the perf benefits of JavaScript without the awkward, nested callbacks and/or promises). The language I'm referring to is none other than Haskell (which has been around in some form since 1990). Here's the earliest paper I know of detailing the internals of the Glasgow Haskell Compiler's IO event handling system (from 2010, I believe): https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36841.pdf https://static.googleusercontent.com/media/research.google.c... And here's a more recent update: http://haskell.cs.yale.edu/wp-content/uploads/2013/08/hask035-voellmy.pdf http://haskell.cs.yale.edu/wp-content/uploads/2013/08/hask03... (Just in case of link-rot, the titles are respectively: "Scalable I/O Event Handling for GHC", "Mio: A High-Performance Multicore IO Manager for GHC") Here's a short example: main = do putStrLn "Hello, what's your name?" name <- getLine putStrLn ("Hey " ++ name ++ ", you rock!") Unfamiliar syntax aside, there's not a callback in sight -- it looks just like standard, imperative, synchronous code. It drives me crazy when people tout Node's approach to concurrency as being ideal, as if callbacks are a necessity. They're not. Haskell, among other languages, is an existence proof that you can have lightweight threads and asynchronous IO without callback spaghetti. And I haven't even gone into Haskell's threaded parallelism, Software Transactional Memory, etc.

Tobani 9y ago
Your still missing the point. It's not that it was possible to do so. It's that you could quickly write web services idomatically that did so by default. Your example shows a simple hello world and doesn't speak to the point being made. I am sure you can get a more relevant example going in Scotty, but it appears that it didn't show up until 2012.
- cstrahan 9y ago
  > Your still missing the point. No, I'm not missing the point. > Your example shows a simple hello world and doesn't speak to the point being made. Hmm... Let's do an analysis: * My example demonstrates reading from a given file descriptor (FD 0 -- stdin -- in this case), and that reading happens asynchronously (as I had pointed out, and even provided white papers describing how this works). Again, this is despite the syntax looking like a naive, synchronous program. The GHC Run Time System handles asynchronously reading from the FD and scheduling threads as appropriate such that OS threads aren't tied up waiting on what would otherwise be a synchronous read(2). * As a bonus, it also demonstrates writing to standard out. Now, one could take exception to the fact that my example doesn't give any evidence that similar asynchrony should be expected in a web stack. One would be wrong: as anyone with, say, as little as a year of experience writing web applications on POSIX systems should know, the Berkeley sockets API exposes sockets as file descriptors, so the same generic async event handling system in GHC's RTS applies equally well to networked IO. GHC and Node (via libuv) uses similar primitive kernel APIs at the end of the day, the only difference being that Haskell doesn't require that you reify the callbacks in the source language. Idiomatic? Check. By default? Check. So no, I think I do get the point, but if you still disagree, I'm all ears. If my tone sounds harsh, it's because I spent the time to provide links to two papers and even gave a small example of what the syntax would look like for a minimal program that demonstrates IO. In turn, you're telling me that I'm missing the point. I would argue that, if you read either of those two papers, and you knew about the implications with respect to networked IO (if you didn't, you could easily research the Berkeley sockets API, kqueue(2), poll(2), etc -- all mentioned in the papers), you would see what I'm getting at with my example code and the papers referenced. I can only posit that you haven't read the papers, or if you did, you must have not done so in earnest. Hopefully you can see how I would be frustrated when I've put in time to research this space, and when I share something, I'm met with what (in appearance) amounts to a dissimilar level of diligence and, ultimately, an empty dismissal. If something else is going on, I would be more than happy to issue an apology. > I am sure you can get a more relevant example going in Scotty, but it appears that it didn't show up until 2012. Open source web frameworks for Haskell go back to (at least as far as) 2009. Happstack: (2009): https://hackage.haskell.org/package/happstack-server-0.1 https://hackage.haskell.org/package/happstack-server-0.1 Yesod (2010): https://hackage.haskell.org/package/yesod-0.0.0 https://hackage.haskell.org/package/yesod-0.0.0 Snap (2010): https://hackage.haskell.org/package/snap-0.3.0 https://hackage.haskell.org/package/snap-0.3.0 I didn't provide a "hello world"-like example of a web app because my command-line example only requires a trivial inference on the part of the reader to realize that the same asynchronous treatment of FDs there applies equally well to sockets, and thus web app programming in general; I deemed a full fledged web app example as more complicated than necessary to get my point across. For what it's worth, here's an example in Happstack, straight from the documentation: module Main where import Happstack.Server (nullConf, simpleHTTP, toResponse, ok) main :: IO () main = simpleHTTP nullConf $ ok "Hello, World!" And ... $ curl http://localhost:8000/ Hello, World! Also, a note on tense: > It's that you could quickly write web services idomatically that did so by default. I wasn't questioning the rationale for using Node back in 2009. My point is this: eight years have passed since Node was released, and alternative solutions have since been released. Somehow the Node community (and the greater web dev community in general) are oblivious to these alternative approaches. I suspect this is because most web devs see the callbacks as evidence that IO is asynchronous in Node, and at the same time lack the requisite experience to intuit how another language might implement asynchronous IO while providing a syntax that looks like the usual synchronous model. Now, they could actually read the research papers put out, but they don't, and so everyone keeps going on about the technological revolution that is Node and its approach to IO, oblivious to alternative solutions that (arguably) provide a more convenient programming model (and is, if we throw in support for multiple threads, more computationally powerful, too). Such laziness and/or lack of curiosity is pervasive in the industry, and that's frustrating.

esfandia 9y ago

> Edit: Heck, NGINX was an asynchronous, event driven web engine released in 2004 and written in C: In the interview, he mentions himself that Node was inspired by Nginx.

ricardobeat 9y ago

Remember the "C10K challenge"? Show me what else could achieve those ten thousand concurrent connections back then (hint: no python).

ungamed 9y ago
Erlang -easily- acheived the C10k challenge back then.