4 ms·

Nothing against react or reactive, but it's yet another reaction to going down the wrong path originally -- it seems. I really don't understand why so many peo

by MCRed 12y ago

Nothing against react or reactive, but it's yet another reaction to going down the wrong path originally -- it seems. I really don't understand why so many people jump to event driven systems when genuinely parallel computing is so easy. Specifically Erlang/elixir make it easy. EASY I tell you! Yeah, yeah, Erlang has a weird syntax. Ok, so lose two weeks learning it. But you don't even need to do that-- just learn Elixir which is pretty comprehensible right off the bat. Where's the downside? There isn't one. Yet people are going out of their way, doing all these different projects and attempts whose sole purpose is to try and fake your way around a problem that was genuinely solved 20 years ago! To be honest- and maybe I'm wrong and missing something here-- it seems like so many programmers are following what's hip and popular. V8 was a neat javascript engine, and then node cam along, so lets jump into callback hell! Why put up with that when erlang was around and well known? It's like people choose technologies based on fashion. But that seems too... pessimistic so I don't want to believe that.

mstromb 12y ago
As someone familiar (and generally quite happy with) reactive programming, what does Erlang/Elixir do that makes it a more general, better solution? And to what types of problems, specifically? I think the reason that I picked up Rx is that it was a natural extension of LINQ, both literally and figuratively. What made Rx really click for me was thinking of it in terms of IEnumerable that explicitly acknowledged of the time domain. Since then I've come to understand some of the concepts that underlay both of them (monads, mostly), but I wouldn't have made the leap to Rx if it wasn't so easy to get there from existing knowledge. I haven't encountered anything where the next step would be "learn Erlang." You can make the argument that we should all learn about things that are unfamiliar, but there are lot of things that are unfamiliar and I haven't seen a lot of reasons why Erlang should rise to the top.
- pron 12y ago
  There is a confusion of terms here. Rx is reactive programming (aka dataflow). What this article talks about -- isn't (and indeed is better served by Erlang or other similar solutions).
  - TuringTest 12y ago
    I followed the Coursera course linked from the article and the similar "Programming Paradigms" course from Standford[1], and from what I recall, dataflow and actors where formally equivalent - i.e. you could write any one using the primitives of the other. Dataflows can be implemented through message calls to independent actors, but I also saw a stateful actor implemented as a purely functional dataflow in the shape of a sequence of its successive states. I've seen recently a lot of people from both styles saying "but that's not really what 'reactive' is" at the other, and I think it may be a misunderstanding caused by that equivalence. [1] https://www.edx.org/course/paradigms-computer-programming-louvainx-louv1-1x#.VRWGi_mWmZc https://www.edx.org/course/paradigms-computer-programming-lo...
ihaveajob 12y ago
Alan Kay argued [https://queue.acm.org/detail.cfm?id=1039523 https://queue.acm.org/detail.cfm?id=1039523] that programming is a sort of pop culture. I think this just illustrates that point.
- collyw 12y ago
  As an experienced developer it is getting really grating. Why so many jobs requiring NoSQL "skills" when a relational database will solve 90% of the perceived problems. We have good mature back end frameworks. So unless you are writing a chat application, why is everyone looking to Nodejs?
  - jpravetz 12y ago
    Easier, or at least easy, fun, less compatibility issues when you live in a pure js and json world on the front and backends, productive, good libraries, good built-in libraries, easy to deploy, productive.
    - collyw 12y ago
      Converting to JSON is pretty easy in almost any language. And the framework I currently use is productive, has good libraries, has good built in libraries. Has a good security record and is extremely productive and stable. Deployment is one area where it doesn't shine. If that's the best reason for choosing a language I would be using PHP. And it doesn't involve JavaScript, which is in my opinion a benefit.
jamii 12y ago
I spent a total of 11 months consulting with a company that built a large (50kloc) financial system in erlang. They had terrible performance problems that were caused entirely by erlang. Imagine you have a large amount of data (order books, accounts, etc). You could put it all in on erlang process, but the gc does not cope well with large heaps (multi-second pauses). You could store the data outside the heap (eg ets) but then you pay the cost of copying on every access and have to tradeoff ease of use (more data per key) vs performance (less data per key). You could split the data up into many processes and then all your simple calculations become asynchronous protocols. Have fun debugging the math or rolling back changes on errors. I went into that contract with a fondness for erlang. Now I wouldn't touch it ever again. A naive single-threaded blocking server achieved 10x less code, 40x better throughput and 100x better latency. I used clojure, but any sane platform would have worked just as well with that design.
- Toine 12y ago
  50kloc is considered "large financial system" ? Just curious.
  - sz4kerto 12y ago
    It depends. I worked on a trading platform that >1M LOC. The overall complexity was lower than another quantitative modelling engine that only had couple of tens of thousands LOC. Complexity, work and LOC are correlated, but the correlation is not 1. In the case of the former system, 95% of the code was connectivity, GUI, persistence, etc., the actual business logic was maybe 5%. And I'm not even talking about stuff written in kdb/q, where I produced maybe 500 LOC in a month. :)
- polskibus 12y ago
  I'm curious whether you brought this problem up to the attention of erlang maintainers, what where their reactions and whether it will be addressed in the future. GCing large objects on the heap was (perhaps still is despite improvements) a significant problem also on .NET platform. Not sure whether this is the case, but have you considered patching the GC yourself or perhaps moving the problem to BIFs and managing that memory manually (without copying to ETS) ?
mamcx 12y ago
Ok, I know a bit about Erlang and think elixir is cool. And for do a professional work, go for it is the right thing. But here the problem of "just do erlang, is better" and where the JS folks are doing BETTER is that they try to show how do things, not just say this or that is better. And look: I read that post, and because I see some functional stuff (map, filters, blah) I get the impression that was a good explain! I know... a lot of JS code around is a terrible hack, and kinda cool/sad/funny how in JS are re-inventing (and a lot of times, poorly) what is better in X.... but as marketing, they are winning. So: Where is something like this that show how build reactive properly? And I mean, build, not just use.. And hopefully in a digestible manner. I read a ton of stuff related to languages (because I wanna do one, for fun) and is like this: http://fsharpforfunandprofit.com/fppatterns/ivorytower.jpg http://fsharpforfunandprofit.com/fppatterns/ivorytower.jpg So I think some examples in how do this well and why are needed. P.D: For example, http://fsharpforfunandprofit.com/fppatterns http://fsharpforfunandprofit.com/fppatterns is probably the best advocacy that I have found about practical functional thinking. Is the reason I have trying F#!
noelwelsh 12y ago
Erlang has two things that are relevant to this discussion: user level threads (aka green threads), and asynchronous message passing. User level threads reduce the memory footprint of threads, and possibly the context switch time (depending on implementation) but they are not without drawbacks. Heck, Java 1.1 had green threads but they dropped them for kernel threads in later releases (http://en.wikipedia.org/wiki/Green_threads#Green_threads_in_the_Java_virtual_machine http://en.wikipedia.org/wiki/Green_threads#Green_threads_in_...). The main issue is balancing work on multicore machines. With kernel threads the OS will automatically balance them. With green threads you are reliant on the user level scheduler to do this work. It is more difficult to get right without the level of information the OS has. If you want to give the programmer control over scheduling you need to provide some access to kernel level threads. Other issues include: not all the OS IO primitives have non-blocking equivalents you can use (typically you have to use a pool of kernel threads for these operations; the runtime may hide this.) and you still pay some context switch time with you don't actually have at all in a simple epoll / kqueue based system. I'm running out of time, so I'll just note that asynchronous message passing is hard to reason about.

by MCRed 12y ago

mstromb 12y ago

As someone familiar (and generally quite happy with) reactive programming, what does Erlang/Elixir do that makes it a more general, better solution? And to what types of problems, specifically? I think the reason that I picked up Rx is that it was a natural extension of LINQ, both literally and figuratively. What made Rx really click for me was thinking of it in terms of IEnumerable that explicitly acknowledged of the time domain. Since then I've come to understand some of the concepts that underlay both of them (monads, mostly), but I wouldn't have made the leap to Rx if it wasn't so easy to get there from existing knowledge. I haven't encountered anything where the next step would be "learn Erlang." You can make the argument that we should all learn about things that are unfamiliar, but there are lot of things that are unfamiliar and I haven't seen a lot of reasons why Erlang should rise to the top.

pron 12y ago
There is a confusion of terms here. Rx is reactive programming (aka dataflow). What this article talks about -- isn't (and indeed is better served by Erlang or other similar solutions).
- TuringTest 12y ago
  I followed the Coursera course linked from the article and the similar "Programming Paradigms" course from Standford[1], and from what I recall, dataflow and actors where formally equivalent - i.e. you could write any one using the primitives of the other. Dataflows can be implemented through message calls to independent actors, but I also saw a stateful actor implemented as a purely functional dataflow in the shape of a sequence of its successive states. I've seen recently a lot of people from both styles saying "but that's not really what 'reactive' is" at the other, and I think it may be a misunderstanding caused by that equivalence. [1] https://www.edx.org/course/paradigms-computer-programming-louvainx-louv1-1x#.VRWGi_mWmZc https://www.edx.org/course/paradigms-computer-programming-lo...

ihaveajob 12y ago

Alan Kay argued [https://queue.acm.org/detail.cfm?id=1039523 https://queue.acm.org/detail.cfm?id=1039523] that programming is a sort of pop culture. I think this just illustrates that point.

collyw 12y ago
As an experienced developer it is getting really grating. Why so many jobs requiring NoSQL "skills" when a relational database will solve 90% of the perceived problems. We have good mature back end frameworks. So unless you are writing a chat application, why is everyone looking to Nodejs?
- jpravetz 12y ago
  Easier, or at least easy, fun, less compatibility issues when you live in a pure js and json world on the front and backends, productive, good libraries, good built-in libraries, easy to deploy, productive.
  - collyw 12y ago
    Converting to JSON is pretty easy in almost any language. And the framework I currently use is productive, has good libraries, has good built in libraries. Has a good security record and is extremely productive and stable. Deployment is one area where it doesn't shine. If that's the best reason for choosing a language I would be using PHP. And it doesn't involve JavaScript, which is in my opinion a benefit.

jamii 12y ago

I spent a total of 11 months consulting with a company that built a large (50kloc) financial system in erlang. They had terrible performance problems that were caused entirely by erlang. Imagine you have a large amount of data (order books, accounts, etc). You could put it all in on erlang process, but the gc does not cope well with large heaps (multi-second pauses). You could store the data outside the heap (eg ets) but then you pay the cost of copying on every access and have to tradeoff ease of use (more data per key) vs performance (less data per key). You could split the data up into many processes and then all your simple calculations become asynchronous protocols. Have fun debugging the math or rolling back changes on errors. I went into that contract with a fondness for erlang. Now I wouldn't touch it ever again. A naive single-threaded blocking server achieved 10x less code, 40x better throughput and 100x better latency. I used clojure, but any sane platform would have worked just as well with that design.

Toine 12y ago
50kloc is considered "large financial system" ? Just curious.
- sz4kerto 12y ago
  It depends. I worked on a trading platform that >1M LOC. The overall complexity was lower than another quantitative modelling engine that only had couple of tens of thousands LOC. Complexity, work and LOC are correlated, but the correlation is not 1. In the case of the former system, 95% of the code was connectivity, GUI, persistence, etc., the actual business logic was maybe 5%. And I'm not even talking about stuff written in kdb/q, where I produced maybe 500 LOC in a month. :)
polskibus 12y ago
I'm curious whether you brought this problem up to the attention of erlang maintainers, what where their reactions and whether it will be addressed in the future. GCing large objects on the heap was (perhaps still is despite improvements) a significant problem also on .NET platform. Not sure whether this is the case, but have you considered patching the GC yourself or perhaps moving the problem to BIFs and managing that memory manually (without copying to ETS) ?

mamcx 12y ago

Ok, I know a bit about Erlang and think elixir is cool. And for do a professional work, go for it is the right thing. But here the problem of "just do erlang, is better" and where the JS folks are doing BETTER is that they try to show how do things, not just say this or that is better. And look: I read that post, and because I see some functional stuff (map, filters, blah) I get the impression that was a good explain! I know... a lot of JS code around is a terrible hack, and kinda cool/sad/funny how in JS are re-inventing (and a lot of times, poorly) what is better in X.... but as marketing, they are winning. So: Where is something like this that show how build reactive properly? And I mean, build, not just use.. And hopefully in a digestible manner. I read a ton of stuff related to languages (because I wanna do one, for fun) and is like this: http://fsharpforfunandprofit.com/fppatterns/ivorytower.jpg http://fsharpforfunandprofit.com/fppatterns/ivorytower.jpg So I think some examples in how do this well and why are needed. P.D: For example, http://fsharpforfunandprofit.com/fppatterns http://fsharpforfunandprofit.com/fppatterns is probably the best advocacy that I have found about practical functional thinking. Is the reason I have trying F#!

noelwelsh 12y ago

Erlang has two things that are relevant to this discussion: user level threads (aka green threads), and asynchronous message passing. User level threads reduce the memory footprint of threads, and possibly the context switch time (depending on implementation) but they are not without drawbacks. Heck, Java 1.1 had green threads but they dropped them for kernel threads in later releases (http://en.wikipedia.org/wiki/Green_threads#Green_threads_in_the_Java_virtual_machine http://en.wikipedia.org/wiki/Green_threads#Green_threads_in_...). The main issue is balancing work on multicore machines. With kernel threads the OS will automatically balance them. With green threads you are reliant on the user level scheduler to do this work. It is more difficult to get right without the level of information the OS has. If you want to give the programmer control over scheduling you need to provide some access to kernel level threads. Other issues include: not all the OS IO primitives have non-blocking equivalents you can use (typically you have to use a pool of kernel threads for these operations; the runtime may hide this.) and you still pay some context switch time with you don't actually have at all in a simple epoll / kqueue based system. I'm running out of time, so I'll just note that asynchronous message passing is hard to reason about.