5 ms·

As someone who has mostly been an outside observer to Erlang, you have succinctly described why I have not felt the need to use it. It's amazing at what it doe

As someone who has mostly been an outside observer to Erlang, you have succinctly described why I have not felt the need to use it. It's amazing at what it does - If you had lots of physical nodes and needed to build a robust, fixed-size system at any moment in time, it's an incredible option. But with container orchestrators, you get the ability to design a polyglot with more fluidity in your design. The admission that different languages are best suited to different tasks throws a monkey wrench into the Erlang ecosystem, because now I all of a sudden can't use Erlang for everything. It does sound like an excellent choice for a control plane in many cases, however. I think it's phenomenal at what it does, but I think you have provided excellent context regarding its ideal use case.

enraged_camel 5y ago
>> But with container orchestrators The other commenter said it best: "But at the end of the day, Erlang is one technology that is good enough for most and does not require large operational experience to run. Your entire stack is in your own code, understandable by your entire team." So the difference is doing your scaling and distribution natively in code, versus hiring and maintaining devops expertise to manage your containers. On top of that you now have one more "node" in your team (whether a single devops engineer or a group of them) that needs to collaborate with everyone.
hnedeotes 5y ago
Does it really? A wrench? Maybe a banana. With the jungle and the gorilla holding it. And anecdotally, for me who never had the need to use distributed erlang in practice, the benefits lay completely elsewhere. Just looking at programs written out there in the wild, some ubiquitous, and knowing the mess that is software development makes one wonder.
mamcx 5y ago
After read the explanation of parent, I see it as certainly one of the most sensible models to build scalable and efficient software. This a great way to make things: - You have N Cores on machine - You create N Threads to match, you don't "create/destroy/create/destroy", no, you already have the size of the problem set! - You assign roles, maybe just split between "heavy" and "light" - Your context switching is for computing small things on light - You do sequential code, tight loops and heavy I/O on heavy I truly wish I could do something like this on most languages (ie: is part of the paradigm/semantics). I use Rust now, and async is great but the mental model is HARD. - You don't have a easy way to split between workloads, anything could suddenly stall - When is easy to launch async/goroutines you launch them like is no tomorrow. - So, this mean I don't know what are the size of my threads or task are(could) be running without manually inspecting or assume. - I don't know, after look at async function, to know which workload is for (without inspect the code and the flow of it) (ok, in rust you can maybe infer if you are clever in the use of types to be markers... but still is disconnected to the run of the program) "container orchestrators" is like put a hammer to handle another hammer ("heavy OS + VM/DbEngines") under the pretense of being "light" and efficient, when actually this is how most software stack are: - A "heavy" DB engine - A "light" logic server that connect to it - MAYBE out-of-band processing for send emails and stuff So, the thing is, having Kubernetes or anything like put the solution too far away. I need, as developer, to MODEL that relationships and KNOW in what workloads this or that must operate. And the most fixed/predictable is it, the better.
rapsey 5y ago
>But with container orchestrators, you get the ability to design a polyglot with more fluidity in your design. You are then adding some redis/memcached, kafka or something similar, etcd perhaps and kubernetes. A bunch of technologies your developers need to use properly and be running to develop on. As opposed to having an easy to use single technology named Erlang which is almost certainly good enough for your needs. And your developers can easily be running a distributed system of erlang nodes locally without any issues or complex setups.
derefr 5y ago
> The admission that different languages are best suited to different tasks throws a monkey wrench into the Erlang ecosystem, because now I all of a sudden can't use Erlang for everything. I would note that Erlang is pretty good at absorbing other things into the Erlang abstraction. Depending on your needs, you can use: • NIFs (library code loaded into the Erlang VM — good for performance, but not good for fault-tolerance) • Port programs: arbitrary POSIX processes (as long as they speak a specified protocol on their stdio) spawned by the Erlang VM and communicated with + managed through a fancy socket abstraction (Erlang ports). • Erlang "C nodes": arbitrary software that links in Erlang's erl_interface distribution-protocol library (https://erlang.org/doc/apps/erl_interface/ei_users_guide.html https://erlang.org/doc/apps/erl_interface/ei_users_guide.htm...) as C code, and so can then join an Erlang distribution-set as a full peer. That last one is especially interesting if you're building a polyglot system where Erlang is just one component: it allows you to build an Erlang "node" entirely in C or Go or Rust or Java or Haskell or whatever, that — from Erlang's perspective — can occupy one of the node-roles in your Erlang system architecture; while from the other software's side, can treat the Erlang abstract-machine as a guest within some other abstracted system. Through the erl_interface library, the C node gets access to all the interesting distribution and fault-tolerance tools of Erlang (e.g. it can interact with other nodes' ports; it can hold monitor refs; etc.) So you still can "use Erlang for everything" (related to distribution and fault-tolerance, at least) even in your other software.

enraged_camel 5y ago

>> But with container orchestrators The other commenter said it best: "But at the end of the day, Erlang is one technology that is good enough for most and does not require large operational experience to run. Your entire stack is in your own code, understandable by your entire team." So the difference is doing your scaling and distribution natively in code, versus hiring and maintaining devops expertise to manage your containers. On top of that you now have one more "node" in your team (whether a single devops engineer or a group of them) that needs to collaborate with everyone.

hnedeotes 5y ago

Does it really? A wrench? Maybe a banana. With the jungle and the gorilla holding it. And anecdotally, for me who never had the need to use distributed erlang in practice, the benefits lay completely elsewhere. Just looking at programs written out there in the wild, some ubiquitous, and knowing the mess that is software development makes one wonder.

mamcx 5y ago

After read the explanation of parent, I see it as certainly one of the most sensible models to build scalable and efficient software. This a great way to make things: - You have N Cores on machine - You create N Threads to match, you don't "create/destroy/create/destroy", no, you already have the size of the problem set! - You assign roles, maybe just split between "heavy" and "light" - Your context switching is for computing small things on light - You do sequential code, tight loops and heavy I/O on heavy I truly wish I could do something like this on most languages (ie: is part of the paradigm/semantics). I use Rust now, and async is great but the mental model is HARD. - You don't have a easy way to split between workloads, anything could suddenly stall - When is easy to launch async/goroutines you launch them like is no tomorrow. - So, this mean I don't know what are the size of my threads or task are(could) be running without manually inspecting or assume. - I don't know, after look at async function, to know which workload is for (without inspect the code and the flow of it) (ok, in rust you can maybe infer if you are clever in the use of types to be markers... but still is disconnected to the run of the program) "container orchestrators" is like put a hammer to handle another hammer ("heavy OS + VM/DbEngines") under the pretense of being "light" and efficient, when actually this is how most software stack are: - A "heavy" DB engine - A "light" logic server that connect to it - MAYBE out-of-band processing for send emails and stuff So, the thing is, having Kubernetes or anything like put the solution too far away. I need, as developer, to MODEL that relationships and KNOW in what workloads this or that must operate. And the most fixed/predictable is it, the better.

rapsey 5y ago

>But with container orchestrators, you get the ability to design a polyglot with more fluidity in your design. You are then adding some redis/memcached, kafka or something similar, etcd perhaps and kubernetes. A bunch of technologies your developers need to use properly and be running to develop on. As opposed to having an easy to use single technology named Erlang which is almost certainly good enough for your needs. And your developers can easily be running a distributed system of erlang nodes locally without any issues or complex setups.

derefr 5y ago

> The admission that different languages are best suited to different tasks throws a monkey wrench into the Erlang ecosystem, because now I all of a sudden can't use Erlang for everything. I would note that Erlang is pretty good at absorbing other things into the Erlang abstraction. Depending on your needs, you can use: • NIFs (library code loaded into the Erlang VM — good for performance, but not good for fault-tolerance) • Port programs: arbitrary POSIX processes (as long as they speak a specified protocol on their stdio) spawned by the Erlang VM and communicated with + managed through a fancy socket abstraction (Erlang ports). • Erlang "C nodes": arbitrary software that links in Erlang's erl_interface distribution-protocol library (https://erlang.org/doc/apps/erl_interface/ei_users_guide.html https://erlang.org/doc/apps/erl_interface/ei_users_guide.htm...) as C code, and so can then join an Erlang distribution-set as a full peer. That last one is especially interesting if you're building a polyglot system where Erlang is just one component: it allows you to build an Erlang "node" entirely in C or Go or Rust or Java or Haskell or whatever, that — from Erlang's perspective — can occupy one of the node-roles in your Erlang system architecture; while from the other software's side, can treat the Erlang abstract-machine as a guest within some other abstracted system. Through the erl_interface library, the C node gets access to all the interesting distribution and fault-tolerance tools of Erlang (e.g. it can interact with other nodes' ports; it can hold monitor refs; etc.) So you still can "use Erlang for everything" (related to distribution and fault-tolerance, at least) even in your other software.