4 ms·

Given any long-running task, there are two natural things to do: 1. Do something while the long-running task is running 2. Do something after the long-runn

by jaekwon 16y ago

Given any long-running task, there are two natural things to do: 1. Do something while the long-running task is running 2. Do something after the long-running task has completed Node.js' callback function convention simply makes it stupendously easy for you to write code for both cases, and leans towards making case (1) as natural as possible. Case (2) is naturally easy in Javascript (and even easier in Coffeescript) because of the way Javascript supports closures. application = function() { // do stuff database_call( options, function(err, callback) { // javascript closure // case (2) logic } ); // case (1) logic } How does Tora fare on this regard? Node.js is about parallelism through forking closures, not any solution to long running CPU intensive tasks. The weakpoints of Node.js, in my opinion (and please correct me if I'm wrong, that would make me happy), is having to write a lot of boilerplate code for error and exception handling. Exception handling is particularly onerous because depending on the function call you'll never know which stack the exception will travel up. In the example above, you (probably) can't catch an exception from case (2) by wrapping the database_call in a try-catch block. Does anyone have a good solution to this? All-in-all, I like that Node.js tries to stay pure by enforcing the above convention. Noders who stay in "userland" can't shoot themselves in the foot by making an asynchronous procedure synchronous, which may lead to faster code overall.

silentbicycle 16y ago
To me, it seems the real weakness of node.js is having to manage asynchronous control flow yourself, at all. It grows exponentially more ugly as the system increases in size. I really like continuation-passing-style as a technique (especially as a compiler IR), but doing that stuff by hand is so 70s. I got pretty far along writing a similar system in Lua (because, hey, event-driven systems are pretty nice), and unlike Javascript, Lua has well-implemented coroutines - they singlehandedly eliminate a LOT of the ugly control flow management. Still, most libraries are blocking* . I eventually decided that, for small systems, an event-loop framework wasn't necessary in Lua (it's easy enough to just do it from scratch!), and for larger systems, I was better off using Erlang, which has the ideal foundation for that sort of system. In particular, I'm really not clear how much error-handling node.js does; that was what convinced me that Erlang's approach is refreshingly sane. When I started writing process-supervision and hot update code in Lua, I realized I was just re-implementing what Erlang does best. * Which is a funny objection, like being mad that most libraries default to using decimal numbers rather than octal. It must be a conspiracy! See also: http://news.ycombinator.com/item?id=2150800 http://news.ycombinator.com/item?id=2150800 and http://news.ycombinator.com/item?id=1304599 http://news.ycombinator.com/item?id=1304599
- jaekwon 16y ago
  Thanks for this, you've nailed the problem. I'm looking for a language that can do everything that Node.js does but also solves this control flow problem, but is also fast and has support for OpenCL and beefy math libraries. I don't find Erlang to be performant in this regard, so I'm looking at Scala but I'm constantly getting pissed.at.java.packages.and.conventions. I think it also suffers from the same problems as Node.js and doesn't support the async/control-flow magic that Haskell supposedly has. Perhaps I should consider Haskell after all. It's that or C++.
  - deleted 16y ago
    [deleted]
  - malkia 16y ago
    Take a look at LuaJIT. I have some early FFI bindings for OpenCL (and OpenGL, glfw) with some simple demos. Not updated recently, but it's done by using stock luajit (take latest), and not writing a binding code at all (just FFI definitions which LuaJIT understands). You can't do callbacks (yet). That is - you can't rely on "C" function to call you back at lua land. That's why I chose glfw instead of glut or others. There is way to setup your main loop without callbacks. http://github.com/malkia/luajit-opencl http://github.com/malkia/luajit-opencl
    - justincormack 16y ago
      There is also a coroutine based embedding of Lua/Luajit in nginx if you want to embed into a web server environment (without callbacks). https://github.com/chaoslawful/lua-nginx-module https://github.com/chaoslawful/lua-nginx-module
  - sayhello 16y ago
    Perhaps you could try python, numpy, twisted, inline callbacks and one of the python opencl packages. Inline callbacks are syntactic sugar included recent versions of twisted that make your code look synchronous while still running on the event loop. With python, you get to use a wealth of libraries, but you need to still be mindful of what is blocking. Wnen something you need to call is blocking, you can try deferToThread. Don't worry, you won't need to think too much about race conditions and other concurrency issues with threading since there is the GIL in python. Also, Twisted makes it a cinch. Good luck!

by jaekwon 16y ago

silentbicycle 16y ago

To me, it seems the real weakness of node.js is having to manage asynchronous control flow yourself, at all. It grows exponentially more ugly as the system increases in size. I really like continuation-passing-style as a technique (especially as a compiler IR), but doing that stuff by hand is so 70s. I got pretty far along writing a similar system in Lua (because, hey, event-driven systems are pretty nice), and unlike Javascript, Lua has well-implemented coroutines - they singlehandedly eliminate a LOT of the ugly control flow management. Still, most libraries are blocking* . I eventually decided that, for small systems, an event-loop framework wasn't necessary in Lua (it's easy enough to just do it from scratch!), and for larger systems, I was better off using Erlang, which has the ideal foundation for that sort of system. In particular, I'm really not clear how much error-handling node.js does; that was what convinced me that Erlang's approach is refreshingly sane. When I started writing process-supervision and hot update code in Lua, I realized I was just re-implementing what Erlang does best. * Which is a funny objection, like being mad that most libraries default to using decimal numbers rather than octal. It must be a conspiracy! See also: http://news.ycombinator.com/item?id=2150800 http://news.ycombinator.com/item?id=2150800 and http://news.ycombinator.com/item?id=1304599 http://news.ycombinator.com/item?id=1304599

jaekwon 16y ago
Thanks for this, you've nailed the problem. I'm looking for a language that can do everything that Node.js does but also solves this control flow problem, but is also fast and has support for OpenCL and beefy math libraries. I don't find Erlang to be performant in this regard, so I'm looking at Scala but I'm constantly getting pissed.at.java.packages.and.conventions. I think it also suffers from the same problems as Node.js and doesn't support the async/control-flow magic that Haskell supposedly has. Perhaps I should consider Haskell after all. It's that or C++.
- deleted 16y ago
  [deleted]
- malkia 16y ago
  Take a look at LuaJIT. I have some early FFI bindings for OpenCL (and OpenGL, glfw) with some simple demos. Not updated recently, but it's done by using stock luajit (take latest), and not writing a binding code at all (just FFI definitions which LuaJIT understands). You can't do callbacks (yet). That is - you can't rely on "C" function to call you back at lua land. That's why I chose glfw instead of glut or others. There is way to setup your main loop without callbacks. http://github.com/malkia/luajit-opencl http://github.com/malkia/luajit-opencl
  - justincormack 16y ago
    There is also a coroutine based embedding of Lua/Luajit in nginx if you want to embed into a web server environment (without callbacks). https://github.com/chaoslawful/lua-nginx-module https://github.com/chaoslawful/lua-nginx-module
- sayhello 16y ago
  Perhaps you could try python, numpy, twisted, inline callbacks and one of the python opencl packages. Inline callbacks are syntactic sugar included recent versions of twisted that make your code look synchronous while still running on the event loop. With python, you get to use a wealth of libraries, but you need to still be mindful of what is blocking. Wnen something you need to call is blocking, you can try deferToThread. Don't worry, you won't need to think too much about race conditions and other concurrency issues with threading since there is the GIL in python. Also, Twisted makes it a cinch. Good luck!