5 ms·

I'm more interested in a language like gleam, but the BEAM VM does sound appealing. But one thing I have been wondering is what is the hosting story like for BE

I'm more interested in a language like gleam, but the BEAM VM does sound appealing. But one thing I have been wondering is what is the hosting story like for BEAM languages? Is there easy tooling for deploying to something like AWS ECS and having nodes communicate with each other?

rob 3y ago
Maybe something like fly.io? I know they support Elixir/Phoenix. https://fly.io/docs/elixir/the-basics/clustering/ https://fly.io/docs/elixir/the-basics/clustering/
mehphp 3y ago
In my experience, deploying BEAM has been just as easy as anything else I can containerize. WRT getting them to communicate? I ended up giving up on that altogether… I’m usually good at figuring that stuff out but this one ended up not being worth it (in my case).
- worthless-trash 3y ago
  What ended up being the problem, maybe is could make this a little clearer (I used https://www.erlang.org/doc/apps/ssl/ssl_distribution https://www.erlang.org/doc/apps/ssl/ssl_distribution as a guide). Was your requirement to have ssl as the distribution communication mechanism ?
  - mehphp 3y ago
    I appreciate it, but it's been a couple of years now, so I can't remember specifically.
lkurusa 3y ago
We deploy Erlang VMs to EC2 and they communicate just fine. Deploying via a Dockerfile and `mix release` has been a breeze. As a bonus point, no real need for container orchestration as the BEAM just figures it out.
sph 3y ago
The answer for any language is just to stick it in a container. Elixir fits inside a container, and the Phoenix generator creates a Dockerfile for you out of the box.
melx 3y ago
You can host Elixir apps on PaaS like Gigalixir[0] or Fly.io[1] (no associated with these). I personally use mix release[2] that assembles a tarball of itself (together with BEAM), then rsync that to Hetzner, restart the remote process and viola. I like simplicity. Re cluster of nodes, the easiest would be to use a library[3] to automate the formation of said cluster. [0] https://www.gigalixir.com/ https://www.gigalixir.com/ [1] https://fly.io/docs/elixir/getting-started/ https://fly.io/docs/elixir/getting-started/ [2] https://elixir-lang.org/getting-started/mix-otp/config-and-releases.html#releases https://elixir-lang.org/getting-started/mix-otp/config-and-r... [3] https://github.com/bitwalker/libcluster https://github.com/bitwalker/libcluster
- parthdesai 3y ago
  A caveat with libcluster + k8s though - it doesn't play nicely with sudden burst of traffic.
  - weatherlight 3y ago
    Can you talk about this more? :) Why is that the case?
    - parthdesai 3y ago
      I'll give you a bit of background to explain it better. Our service is primarily a graphql API. The mobile apps talk to our service, and then it internally talks to other services. We use Absinthe for graphql, and a lot of it is subscription based. Since these nodes (pods in k8s terms) are clustered, each pod is aware of the other pod. Sometimes we get a huge spike in traffic and the clusters can't start fast enough (we're increasing the scale up time in k8s) and a pod can get OOM killed due to number of graphql subscription and then just go in crashloop. This leads to libcluster thinking the node is still good but when infact it's in crashloop, and thus not allowing any new pods coming up to start up. We're experimenting with few things, but yeah clustering is not without it's pain
      - 3y ago
emerongi 3y ago
> Is there easy tooling for deploying to something like AWS ECS and having nodes communicate with each other? libcluster[0] has a bunch of strategies to form clusters. It seems that ECS supports service discovery through DNS, so the DNSPoll[1] strategy should work. [0] https://github.com/bitwalker/libcluster https://github.com/bitwalker/libcluster [1] https://hexdocs.pm/libcluster/Cluster.Strategy.DNSPoll.html https://hexdocs.pm/libcluster/Cluster.Strategy.DNSPoll.html
pdimitar 3y ago
You already got good answers, I'm only going to add that you can also integrate the self-contained `mix release` final build production artifact with systemd just fine. I've done it.
H12 3y ago
As an aside, Elixir is doing some interesting, novel stuff exploring its own type system. José Valim's keynote at ElixirConf last week went into detail on the topic, so I'd keep an eye out for it on YouTube in the coming weeks if that sort of thing interests you.
- turdprincess 3y ago
  I couldn’t imagine working on a large scale project without strong types. What happens when you have to refactor a large elixir feature on a code base which many teams touch? In such cases you can’t test manually all code paths for runtime errors, so are you just relying (hoping for) perfect test coverage to catch any type related errors you may have caused?
  - weatherlight 3y ago
    It's not that hard, also Elixir is basically gradually typed with dialyzer. It's not like working on a dynamic and weakly typed lang like vanilla JavaScript or Perl at all. I do want to clear something up. Elixir is strongly typed, but its not statically typed. More here: https://www.educative.io/answers/statically-v-dynamically-v-strongly-v-weakly-typed-languages https://www.educative.io/answers/statically-v-dynamically-v-... Even for langs that are statically/ and strongly typed on the BEAM (like Gleam, whose type system is similar to that of Standard ML or OCaml) still subscribe to the "Let is Crash" philosophy, especially when it comes to messages sent and received between processes. The only thing that is guaranteed is your system will fail at some point, how should the system protect its self form that? https://www.educative.io/courses/concurrent-data-processing-elixir/understand-the-let-it-crash-philosophy https://www.educative.io/courses/concurrent-data-processing-... In short, its not like working with Ruby or JavaScript or Python, etc..
    - turdprincess 3y ago
      The “let it crash” philosophy is great, but at the same time a proper static type system protects you from a whole class of crashes before they ever reach deployment. If you change the shape of a model object and break its type, it’s great that BEAM will gracefully handle that mistake, but no matter how many times that service restarts, it will still be broken at runtime. I think type systems have gotten so good in modern languages (Kotlin for example) that it’s a disservice to your org to not use a statically typed language. I haven’t looked at Dialyzer lately, but is it as robust and easy to use as a first class type system? Having experienced partial typing with annotations with python I must say it’s not nearly as smooth as a proper statically typed language. I do wish something like Gleam was the front runner language on BEAM - I bet it would get taken a lot more readily.

rob 3y ago

Maybe something like fly.io? I know they support Elixir/Phoenix. https://fly.io/docs/elixir/the-basics/clustering/ https://fly.io/docs/elixir/the-basics/clustering/

mehphp 3y ago

In my experience, deploying BEAM has been just as easy as anything else I can containerize. WRT getting them to communicate? I ended up giving up on that altogether… I’m usually good at figuring that stuff out but this one ended up not being worth it (in my case).

worthless-trash 3y ago
What ended up being the problem, maybe is could make this a little clearer (I used https://www.erlang.org/doc/apps/ssl/ssl_distribution https://www.erlang.org/doc/apps/ssl/ssl_distribution as a guide). Was your requirement to have ssl as the distribution communication mechanism ?
- mehphp 3y ago
  I appreciate it, but it's been a couple of years now, so I can't remember specifically.

lkurusa 3y ago

We deploy Erlang VMs to EC2 and they communicate just fine. Deploying via a Dockerfile and `mix release` has been a breeze. As a bonus point, no real need for container orchestration as the BEAM just figures it out.

sph 3y ago

The answer for any language is just to stick it in a container. Elixir fits inside a container, and the Phoenix generator creates a Dockerfile for you out of the box.

melx 3y ago

You can host Elixir apps on PaaS like Gigalixir[0] or Fly.io[1] (no associated with these). I personally use mix release[2] that assembles a tarball of itself (together with BEAM), then rsync that to Hetzner, restart the remote process and viola. I like simplicity. Re cluster of nodes, the easiest would be to use a library[3] to automate the formation of said cluster. [0] https://www.gigalixir.com/ https://www.gigalixir.com/ [1] https://fly.io/docs/elixir/getting-started/ https://fly.io/docs/elixir/getting-started/ [2] https://elixir-lang.org/getting-started/mix-otp/config-and-releases.html#releases https://elixir-lang.org/getting-started/mix-otp/config-and-r... [3] https://github.com/bitwalker/libcluster https://github.com/bitwalker/libcluster

parthdesai 3y ago
A caveat with libcluster + k8s though - it doesn't play nicely with sudden burst of traffic.
- weatherlight 3y ago
  Can you talk about this more? :) Why is that the case?
  - parthdesai 3y ago
    I'll give you a bit of background to explain it better. Our service is primarily a graphql API. The mobile apps talk to our service, and then it internally talks to other services. We use Absinthe for graphql, and a lot of it is subscription based. Since these nodes (pods in k8s terms) are clustered, each pod is aware of the other pod. Sometimes we get a huge spike in traffic and the clusters can't start fast enough (we're increasing the scale up time in k8s) and a pod can get OOM killed due to number of graphql subscription and then just go in crashloop. This leads to libcluster thinking the node is still good but when infact it's in crashloop, and thus not allowing any new pods coming up to start up. We're experimenting with few things, but yeah clustering is not without it's pain
    - 3y ago

emerongi 3y ago

> Is there easy tooling for deploying to something like AWS ECS and having nodes communicate with each other? libcluster[0] has a bunch of strategies to form clusters. It seems that ECS supports service discovery through DNS, so the DNSPoll[1] strategy should work. [0] https://github.com/bitwalker/libcluster https://github.com/bitwalker/libcluster [1] https://hexdocs.pm/libcluster/Cluster.Strategy.DNSPoll.html https://hexdocs.pm/libcluster/Cluster.Strategy.DNSPoll.html

pdimitar 3y ago

You already got good answers, I'm only going to add that you can also integrate the self-contained `mix release` final build production artifact with systemd just fine. I've done it.

H12 3y ago

As an aside, Elixir is doing some interesting, novel stuff exploring its own type system. José Valim's keynote at ElixirConf last week went into detail on the topic, so I'd keep an eye out for it on YouTube in the coming weeks if that sort of thing interests you.

turdprincess 3y ago
I couldn’t imagine working on a large scale project without strong types. What happens when you have to refactor a large elixir feature on a code base which many teams touch? In such cases you can’t test manually all code paths for runtime errors, so are you just relying (hoping for) perfect test coverage to catch any type related errors you may have caused?
- weatherlight 3y ago
  It's not that hard, also Elixir is basically gradually typed with dialyzer. It's not like working on a dynamic and weakly typed lang like vanilla JavaScript or Perl at all. I do want to clear something up. Elixir is strongly typed, but its not statically typed. More here: https://www.educative.io/answers/statically-v-dynamically-v-strongly-v-weakly-typed-languages https://www.educative.io/answers/statically-v-dynamically-v-... Even for langs that are statically/ and strongly typed on the BEAM (like Gleam, whose type system is similar to that of Standard ML or OCaml) still subscribe to the "Let is Crash" philosophy, especially when it comes to messages sent and received between processes. The only thing that is guaranteed is your system will fail at some point, how should the system protect its self form that? https://www.educative.io/courses/concurrent-data-processing-elixir/understand-the-let-it-crash-philosophy https://www.educative.io/courses/concurrent-data-processing-... In short, its not like working with Ruby or JavaScript or Python, etc..
  - turdprincess 3y ago
    The “let it crash” philosophy is great, but at the same time a proper static type system protects you from a whole class of crashes before they ever reach deployment. If you change the shape of a model object and break its type, it’s great that BEAM will gracefully handle that mistake, but no matter how many times that service restarts, it will still be broken at runtime. I think type systems have gotten so good in modern languages (Kotlin for example) that it’s a disservice to your org to not use a statically typed language. I haven’t looked at Dialyzer lately, but is it as robust and easy to use as a first class type system? Having experienced partial typing with annotations with python I must say it’s not nearly as smooth as a proper statically typed language. I do wish something like Gleam was the front runner language on BEAM - I bet it would get taken a lot more readily.