8 ms·
That's why I like Erlang and Elixir. They were built to handle concurrency down to the core. Currently only languages on BEAM VM provide a set of mature, built-
by rdtsc 10y ago
That's why I like Erlang and Elixir. They were built to handle concurrency down to the core. Currently only languages on BEAM VM provide a set of mature, built-in fault tolerance features (code reloading, isolated concurrency units -- processes, immutable data, sending messages between processes instead of acquiring lock).
I often see some frameworks which claim implement Erlang in "language $x" by adding a queue to a thread. But that is still very much behind what Erlang does, because it is missing these other components, namely fault tolerance.
Sure you can spawn OS processes, or spin up multiple machines/containers. But that is not built-in, so have to manage the additional stack for that. Java has code reloading, but it is not quite the same and so on.
And this is not just talk in generalities, these properties translate directly to benefits and profits -- faster development time, less ops overhead (some parts of the backend can crash and restart, without having to wake everyone up), less code to maintain and dependencies to manage.
Just the other day had a typical distributed systems problem -- didn't add backpressure and so messages were piling up in receiver mailbox. The simplest solution I tested was just switch a gen_server:cast to a gen_server:call. It was a 2-3 line change. Hotpatched on a test cluster and problem was fixed in a few minutes. Ultimately I did something else, but the point is using a language which was built in for concurrency is that it was just a couple of lines of code. Had that been a custom RPC solution, with some serialization and some socket code, it would have taken a lot more to write, test and deploy. All that adds up quickly and can make or break the project.
- heavenlyhash 10y agoThe biggest question I have about this: Is it really a good idea to make local actors and supervisor trees indistinguishable from remote ones? Reusability is good. Abstractions that are so "good" that they obscure potentially order-of-mag performance differences -- I'm not sold. Whether an actor is in-memory with me or across a TCP socket is order-of-mag, and may make the difference between whether my program satisfies is service goals, or fails them abjectly. Especially if that randomized latency happens a number of times, or in unpredictable places. Error handling paths that have to account for the Two-Generals problem being indistinguishable from local errors I can be confident I'll hear about -- I'm not sold on that, either. I'd be interested to hear rebuttals to those from folks who have worked deeply with OTP. Is it just... not that bad, in practice? Or are there parts of the abstraction that specifically help counter these concerns?
- pyotrgalois 10y agoIn some way this article by one of Erlang creators is related to your comment: https://armstrongonsoftware.blogspot.com.ar/2008/05/road-we-didnt-go-down.html https://armstrongonsoftware.blogspot.com.ar/2008/05/road-we-... > The fundamental problem with taking a remote operation and wrapping it up so that it looks like a local operation is that the failure modes of local and remote operations are completely different.
- qznc 10y agoArmstrong clearly sees the problem. I understand his solution as: Erlang makes local and remote actors indistinguishable, but you should build your own custom wrapper for remote calls. This seems fishy to me. You build low-level boilerplate over a high-level foundation? To clarify, I consider implementing the timeout as "low-level" and a mechanism like "link(remote_process)" as "high-level".
- pyotrgalois 10y agoCould you please explain why do you think a timeout would be low level?
- derefr 10y agoI think of Erlang less as making remote operations masquerade as local ones, and more as forcing you to assume that all local code has the potential failure-modes of remote code. Which, often (for weird and roundabout reasons) it does.
- jeffdavis 10y agoVery insightful!
- im_down_w_otp 10y agoThis.
- abecedarius 10y ago
- catnaroek 10y ago> They were built to handle concurrency down to the core. And they weren't built to handle data parallelism at all.
- rdtsc 10y agoNot sure I know what you mean by "data parallelism". I wrote code not too long ago which had multiple processes reference the same binary blob. I have enough CPUs to ensure some processes were actually accessing it in parallel. In other case, used an ETS table to have multiple processes read and write data to and from it. Or maybe to put it another way, if you think I am not already doing data parallelism, sell me on why I need data parallelism.
- catnaroek 10y ago> reference the same binary blob Is there some way to share data whose structure is actually understood and checked (statically or dynamically) by the language? > to have multiple processes read and write data to and from it. How did you coordinate those reads and writes? A writer not having exclusive access to what it's writing to, at the time of the writing... sounds... very bad. > sell me on why I need data parallelism. Just to name one example: machine learning. If you're feeding real-time data to a ML algorithm, chances are you'll have to perform multiple independent calculations that use the same data as input. To speed up those calculations, you need to perform them in parallel.
- di4na 10y agoNothing is. There was an interesting discussion between Simon Peyton Jones and Joe Armstrong on that. https://www.infoq.com/interviews/armstrong-peyton-jones-erlang-haskell https://www.infoq.com/interviews/armstrong-peyton-jones-erla... In general, the thing is... parallelism is hard. If not impossible to do right from automated tools.
- catnaroek 10y agoFirst we have to make it safe: make it inexpressible to do parallelism wrong. Only then can we discuss automating it. There is no value whatsoever in automating the wrong thing.