7 ms·

Caramel: An OCaml for the Erlang VM

201 points by alokrai 6y ago

cultofmetatron 6y ago
2 years ago, I went all in with elixir/phoenix with my startup. Happy to see a language like this which appears to handle integration with the beam ecosystem as a first class consideration. I can see this being useful for some parts of the app where I need to integrate with external services.
- praveenperera 6y ago
  If you’re looking for a typed BEAM language with great interop with Elixir and other BEAM languages, have you seen Gleam? https://github.com/gleam-lang/gleam https://github.com/gleam-lang/gleam
- sagolikasoppor 6y ago
  How did it work out for you?
  - cultofmetatron 6y ago
    there's been a few challenges related to lack of libraries and the occasional dependency hell but overall it has been our secret weapon. We can knock out features very fast and the api server has NEVER been the bottleneck. performance is a dream. Elixir has been refreshingly low on hype and strong on delivery. We were able to scale using Oban to run background workers. The BEAM makes runnign background workers so easy we can add background processes without needing to setup any external orchestration. This lets me focus on building features instead of puttng together micro-serverices or orchestration tooling Channels let us consider realtime from the get go. Since push functionality over web-sockets is so cheap resoruces wise, we use it everywhere. Our scaling strategy when the database is he bottleneck is to insert data as a job in oban job and send the result out over a channel. This would be an orchestration nightmare in production with any other system. WE have a few other challenges (which I can't go into) where elixir makes it possible for a small team to do things that would be impossible in other languages without vastly more resources. Suffice to say, the ability to create independent processes under a supervision tree both via configuration and dynamically via the registry allow us to create scalable topologies that would be a crushing level of complexity to manage without the solid shoulders that the beam stands on. Basically, if realtime is a component of your startup, phoenix does not dissapoint.
may4m 6y ago
I was looking at elixir and F# for my next project. I liked F# for the type system and elixir for OTP/Beam. This looks like something I wanted
- bmitc 6y ago
  You might also be interested in Gleam. https://gleam.run/ https://gleam.run/ Is it possible to share any details about your project? I'm just curious since I also use F# and am learning Elixir.
  - devmunchies 6y ago
    Does gleam let me destructure a list so that I can bind the head and body each to a variable? e.g. in f# let splitHeadAndTail = function | h :: [] -> printfn "got head and EMPTY tail" | h :: t -> printfn "got head and tail" | [] -> printfn "got empty list" The example in their docs does a 1:1 map and doesn't show if you can bind the whole tail[1] [1]: https://gleam.run/book/tour/case-expressions.html#destructuring https://gleam.run/book/tour/case-expressions.html#destructur...
    - derefr 6y ago
      Yes; see https://hexdocs.pm/gleam_stdlib/gleam/list/ https://hexdocs.pm/gleam_stdlib/gleam/list/. The syntax is [head, ..tail] It'd be kind of silly to create a greenfield BEAM language that doesn't expose one of the most fundamental operations you can do in BEAM bytecode, and that every list-processing function ultimately relies upon :)
      - devmunchies 6y ago
        > It'd be kind of silly to have a BEAM language that doesn't expose one of the most primitive operations you can do in the BEAM, that every list-processing function ultimately relies upon :) Agree, which is why i was confused it wasn't in the tour I linked to in the Lists or Destructuring sections
    - 6y ago
toolz 6y ago
being unable to do dynamic dispatch seems like a big show-stopper for distributed/meshed apps, which seems to be the only reason for targeting the BEAM to begin with, no? If you don't pass functions around as MFAs in a distributed app you run into trouble with upgrading your cluster as they'll have different versions of a function during a rolling deploy
- jolux 6y ago
  Different organizations use different amounts of the tools the BEAM provides in building their distributed applications. We containerize everything and don't really use the hot upgrade features.
  - toolz 6y ago
    sure, but I'm not talking about hot upgrades, I'm talking about upgrading entire nodes that exist in a cluster. The nodes will not be able to pass around functions as an upgraded node in a cluster will have a different version of that function. So this is an issue without considering hot code reloading, as I don't use that feature either.
    - jolux 6y ago
      I don't quite follow how static types change this situation?
    - yawaramin 6y ago
      So we're not talking about hot code reloading here, we're talking exactly about upgrading nodes in a cluster. Many--I'd say most--Erlang/Elixir deployments today don't do that. They use Kubernetes.
      - toolz 6y ago
        Kubernetes does not solve the issue, it merely has strategies for how to upgrade nodes. The issue is that when you compile new code, even if the function didn't change at all, it will be a new version of the function. If the upgraded node then sends a function to another node to be executed (as is a very common practice in meshed OTP apps) the older node will not be able to execute the function. Now, I understand that most erlang apps still don't utilize clustering and instead are stateless apps and that's fine, but ocaml can already do stateless apps just fine, right? So my assumption here is that they're putting ocaml on the BEAM to utilize OTP and clustered apps, correct? If that's the case I don't see how you could have a real-world app that you can quickly scale up/down (which is something you often want in a clustered app) - instead it seems like you'd be forced to spin up entirely new clusters and then direct traffic to the new cluster to bypass this problem. This of course isn't a viable tradeoff for very large clusters.
jaegerpicker 6y ago
I loved working with the Beam VM and Elixir. I just wish it had a stronger presence in the Data/ML/AI fields. As scalable and trivially parallizable as Elixir is with a fast SciPy/NumPy/Pandas type of eco-system and you'd have a fantastic environment for data heavy apps IMO.
- a_bored_husky 6y ago
  Have you seen project Nx? https://github.com/elixir-nx/nx https://github.com/elixir-nx/nx
- jbott 6y ago
  Have you seen nx[1]? It looks like the first step towards this! 1. https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-available https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-...
- joisig 6y ago
  Take a look at https://github.com/elixir-nx/nx https://github.com/elixir-nx/nx
- dsiegel2275 6y ago
  Have you seen the other three replies in this thread?
- machiaweliczny 6y ago
  There's also Broadway for ETL
- rmetzler 6y ago
  Lots of people mention Nx and of course I want to do the same but with a link to a youtube video, in case you wonder what's possible currently. https://www.youtube.com/watch?v=fPKMmJpAGWc https://www.youtube.com/watch?v=fPKMmJpAGWc
macintux 6y ago
Obligatory list of alternative Erlang VM languages: https://gist.github.com/macintux/6349828#alternative-languages-targeting-the-erlang-vm https://gist.github.com/macintux/6349828#alternative-languag...
pharmakom 6y ago
The creator of Erlang said that static typing was not appropriate for the applications Erlang is designed for. Worth keeping this in mind!
- jasone 6y ago
  Erlang APIs are designed around the availability of dynamic pattern matching on messages, but static message typing is definitely feasible. The only feature of Erlang that stands out me as fundamentally incompatible with static typing is "live upgrade". And although this feature might be critical in the context for which it was designed (Ericcson telecom switches), it appears that very few Erlang-based deployments use live upgrade; the most common practice is to restart services.
  - toast0 6y ago
    I'm not that up on static typing, but it would seem to me that static typing would be difficult with different versioned nodes on dist as well as hot upgrade. While a lot of users avoid hot load and restart/replace nodes instead, I don't think many restart/replace the whole cluster.
    - dnautics 6y ago
      > different versioned nodes Exactly. Blue-green deploys could get seriously messed up if the static typing system doesn't strongly think about this.
      - carlmr 6y ago
        I'm not too familiar with Erlang. Doesn't dynamic typing just mean you have to strongly think about it?
        macintux 6y ago
        You're absolutely right: with Erlang you have to spend a great deal of effort planning for, implementing, and testing code that can work correctly across nodes with different versions of the code and/or versions of Erlang itself. I think the open question is whether strong, _static_ type system can handle it at all.
jimbokun 6y ago
Knowing nothing else about this project, it falls in the category of things about which I thought "Wouldn't it be cool if somebody..."
vihangd 6y ago
Now only if we could have a decent OCaml/F# for the JVM too.. at least for GraalVM
- darksaints 6y ago
  I know it's not OCaml, but you should really give Scala a try. Martin Odersky has never been shy about his love for SML and OCaml, and it really shines through in the semantics found in Scala.
- lpedrosa 6y ago
  Java as a language is becoming more and more like OCaml. Once sealed types are introduced, we'll get closer and closer since you already have records and pattern matching. Obviously, it'll never be the same. Java will always have it's ways of doing things, the community, etc. I don't know who said it, but it was probably here in HN, where "if you squint very hard, you'll see what Brian et al are trying to accomplish". This was regarding the MLfication of Java.
- programLyrique 6y ago
  There is ocamljava: http://www.ocamljava.org/ http://www.ocamljava.org/ But it does not seem to be maintained.
namelosw 6y ago
This is cool. It seems that it even supports Reason syntax. There are really many languages start targeting BEAM lately. Quite a pleasant trend since Erlang was always underrated IMO.
leostera 6y ago
Hola folks, I'm making Caramel. AMA. There's usually updates on Twitter too here: https://twitter.com/CaramelLang https://twitter.com/CaramelLang
- akavel 6y ago
  In the installation guides, please mention at least in passing that it needs Erlang to be installed as a prerequisite. Being a casual reader, I wasn't sure about that until I found https://caramel.run/manual/getting-started/first-steps.html#prerequisites https://caramel.run/manual/getting-started/first-steps.html#.... Notably, the "it's just a single binary" might be highly misleading in this context for people not coming from the BEAM users community. Also, it would be super cool if I could easily find a list of limitations/differences vs. conventional OCaml language & runtime. And a hello-world snippet on the front page (or possibly some other tiny one showing some cool BEAM feature), it took me a while to find the "examples" folder in the github repo. Please note those are just some tiny complaints rooted in the fact that I absolutely love what you did here! :)
  - leostera 6y ago
    Good points! I've made 3 issues to make sure we get this done. https://github.com/AbstractMachinesLab/caramel/issues/73 https://github.com/AbstractMachinesLab/caramel/issues/73 https://github.com/AbstractMachinesLab/caramel/issues/74 https://github.com/AbstractMachinesLab/caramel/issues/74 https://github.com/AbstractMachinesLab/caramel/issues/75 https://github.com/AbstractMachinesLab/caramel/issues/75 Thanks a lot <3 - you're welcome in our Discord: http://discord.caramel.run http://discord.caramel.run
chromatin 6y ago
This looks very cool, and I have been casually looking into OCaml lately. Please excuse this naïve question, but what is the main value proposition here over plain OCaml: Erlang/BEAM library ecosystem interop, or more something relating to high concurrency? It wasn’t really clear to me from the website/docs.

201 points by alokrai 6y ago

2 years ago, I went all in with elixir/phoenix with my startup. Happy to see a language like this which appears to handle integration with the beam ecosystem as a first class consideration. I can see this being useful for some parts of the app where I need to integrate with external services.

praveenperera 6y ago
If you’re looking for a typed BEAM language with great interop with Elixir and other BEAM languages, have you seen Gleam? https://github.com/gleam-lang/gleam https://github.com/gleam-lang/gleam
sagolikasoppor 6y ago
How did it work out for you?
- cultofmetatron 6y ago
  there's been a few challenges related to lack of libraries and the occasional dependency hell but overall it has been our secret weapon. We can knock out features very fast and the api server has NEVER been the bottleneck. performance is a dream. Elixir has been refreshingly low on hype and strong on delivery. We were able to scale using Oban to run background workers. The BEAM makes runnign background workers so easy we can add background processes without needing to setup any external orchestration. This lets me focus on building features instead of puttng together micro-serverices or orchestration tooling Channels let us consider realtime from the get go. Since push functionality over web-sockets is so cheap resoruces wise, we use it everywhere. Our scaling strategy when the database is he bottleneck is to insert data as a job in oban job and send the result out over a channel. This would be an orchestration nightmare in production with any other system. WE have a few other challenges (which I can't go into) where elixir makes it possible for a small team to do things that would be impossible in other languages without vastly more resources. Suffice to say, the ability to create independent processes under a supervision tree both via configuration and dynamically via the registry allow us to create scalable topologies that would be a crushing level of complexity to manage without the solid shoulders that the beam stands on. Basically, if realtime is a component of your startup, phoenix does not dissapoint.

may4m 6y ago

I was looking at elixir and F# for my next project. I liked F# for the type system and elixir for OTP/Beam. This looks like something I wanted

bmitc 6y ago
You might also be interested in Gleam. https://gleam.run/ https://gleam.run/ Is it possible to share any details about your project? I'm just curious since I also use F# and am learning Elixir.
- devmunchies 6y ago
  Does gleam let me destructure a list so that I can bind the head and body each to a variable? e.g. in f# let splitHeadAndTail = function | h :: [] -> printfn "got head and EMPTY tail" | h :: t -> printfn "got head and tail" | [] -> printfn "got empty list" The example in their docs does a 1:1 map and doesn't show if you can bind the whole tail[1] [1]: https://gleam.run/book/tour/case-expressions.html#destructuring https://gleam.run/book/tour/case-expressions.html#destructur...
  - derefr 6y ago
    Yes; see https://hexdocs.pm/gleam_stdlib/gleam/list/ https://hexdocs.pm/gleam_stdlib/gleam/list/. The syntax is [head, ..tail] It'd be kind of silly to create a greenfield BEAM language that doesn't expose one of the most fundamental operations you can do in BEAM bytecode, and that every list-processing function ultimately relies upon :)
    - devmunchies 6y ago
      > It'd be kind of silly to have a BEAM language that doesn't expose one of the most primitive operations you can do in the BEAM, that every list-processing function ultimately relies upon :) Agree, which is why i was confused it wasn't in the tour I linked to in the Lists or Destructuring sections
  - 6y ago

toolz 6y ago

being unable to do dynamic dispatch seems like a big show-stopper for distributed/meshed apps, which seems to be the only reason for targeting the BEAM to begin with, no? If you don't pass functions around as MFAs in a distributed app you run into trouble with upgrading your cluster as they'll have different versions of a function during a rolling deploy

jolux 6y ago
Different organizations use different amounts of the tools the BEAM provides in building their distributed applications. We containerize everything and don't really use the hot upgrade features.
- toolz 6y ago
  sure, but I'm not talking about hot upgrades, I'm talking about upgrading entire nodes that exist in a cluster. The nodes will not be able to pass around functions as an upgraded node in a cluster will have a different version of that function. So this is an issue without considering hot code reloading, as I don't use that feature either.
  - jolux 6y ago
    I don't quite follow how static types change this situation?
  - yawaramin 6y ago
    So we're not talking about hot code reloading here, we're talking exactly about upgrading nodes in a cluster. Many--I'd say most--Erlang/Elixir deployments today don't do that. They use Kubernetes.
    - toolz 6y ago
      Kubernetes does not solve the issue, it merely has strategies for how to upgrade nodes. The issue is that when you compile new code, even if the function didn't change at all, it will be a new version of the function. If the upgraded node then sends a function to another node to be executed (as is a very common practice in meshed OTP apps) the older node will not be able to execute the function. Now, I understand that most erlang apps still don't utilize clustering and instead are stateless apps and that's fine, but ocaml can already do stateless apps just fine, right? So my assumption here is that they're putting ocaml on the BEAM to utilize OTP and clustered apps, correct? If that's the case I don't see how you could have a real-world app that you can quickly scale up/down (which is something you often want in a clustered app) - instead it seems like you'd be forced to spin up entirely new clusters and then direct traffic to the new cluster to bypass this problem. This of course isn't a viable tradeoff for very large clusters.

jaegerpicker 6y ago

I loved working with the Beam VM and Elixir. I just wish it had a stronger presence in the Data/ML/AI fields. As scalable and trivially parallizable as Elixir is with a fast SciPy/NumPy/Pandas type of eco-system and you'd have a fantastic environment for data heavy apps IMO.

a_bored_husky 6y ago
Have you seen project Nx? https://github.com/elixir-nx/nx https://github.com/elixir-nx/nx
jbott 6y ago
Have you seen nx[1]? It looks like the first step towards this! 1. https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-available https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-...
joisig 6y ago
Take a look at https://github.com/elixir-nx/nx https://github.com/elixir-nx/nx
dsiegel2275 6y ago
Have you seen the other three replies in this thread?
machiaweliczny 6y ago
There's also Broadway for ETL
rmetzler 6y ago
Lots of people mention Nx and of course I want to do the same but with a link to a youtube video, in case you wonder what's possible currently. https://www.youtube.com/watch?v=fPKMmJpAGWc https://www.youtube.com/watch?v=fPKMmJpAGWc

macintux 6y ago

Obligatory list of alternative Erlang VM languages: https://gist.github.com/macintux/6349828#alternative-languages-targeting-the-erlang-vm https://gist.github.com/macintux/6349828#alternative-languag...

pharmakom 6y ago

The creator of Erlang said that static typing was not appropriate for the applications Erlang is designed for. Worth keeping this in mind!

jasone 6y ago
Erlang APIs are designed around the availability of dynamic pattern matching on messages, but static message typing is definitely feasible. The only feature of Erlang that stands out me as fundamentally incompatible with static typing is "live upgrade". And although this feature might be critical in the context for which it was designed (Ericcson telecom switches), it appears that very few Erlang-based deployments use live upgrade; the most common practice is to restart services.
- toast0 6y ago
  I'm not that up on static typing, but it would seem to me that static typing would be difficult with different versioned nodes on dist as well as hot upgrade. While a lot of users avoid hot load and restart/replace nodes instead, I don't think many restart/replace the whole cluster.
  - dnautics 6y ago
    > different versioned nodes Exactly. Blue-green deploys could get seriously messed up if the static typing system doesn't strongly think about this.
    - carlmr 6y ago
      I'm not too familiar with Erlang. Doesn't dynamic typing just mean you have to strongly think about it?
      - macintux 6y ago
        You're absolutely right: with Erlang you have to spend a great deal of effort planning for, implementing, and testing code that can work correctly across nodes with different versions of the code and/or versions of Erlang itself. I think the open question is whether strong, _static_ type system can handle it at all.

jimbokun 6y ago

Knowing nothing else about this project, it falls in the category of things about which I thought "Wouldn't it be cool if somebody..."

vihangd 6y ago

Now only if we could have a decent OCaml/F# for the JVM too.. at least for GraalVM

darksaints 6y ago
I know it's not OCaml, but you should really give Scala a try. Martin Odersky has never been shy about his love for SML and OCaml, and it really shines through in the semantics found in Scala.
lpedrosa 6y ago
Java as a language is becoming more and more like OCaml. Once sealed types are introduced, we'll get closer and closer since you already have records and pattern matching. Obviously, it'll never be the same. Java will always have it's ways of doing things, the community, etc. I don't know who said it, but it was probably here in HN, where "if you squint very hard, you'll see what Brian et al are trying to accomplish". This was regarding the MLfication of Java.
programLyrique 6y ago
There is ocamljava: http://www.ocamljava.org/ http://www.ocamljava.org/ But it does not seem to be maintained.

namelosw 6y ago

This is cool. It seems that it even supports Reason syntax. There are really many languages start targeting BEAM lately. Quite a pleasant trend since Erlang was always underrated IMO.

leostera 6y ago

Hola folks, I'm making Caramel. AMA. There's usually updates on Twitter too here: https://twitter.com/CaramelLang https://twitter.com/CaramelLang

akavel 6y ago
In the installation guides, please mention at least in passing that it needs Erlang to be installed as a prerequisite. Being a casual reader, I wasn't sure about that until I found https://caramel.run/manual/getting-started/first-steps.html#prerequisites https://caramel.run/manual/getting-started/first-steps.html#.... Notably, the "it's just a single binary" might be highly misleading in this context for people not coming from the BEAM users community. Also, it would be super cool if I could easily find a list of limitations/differences vs. conventional OCaml language & runtime. And a hello-world snippet on the front page (or possibly some other tiny one showing some cool BEAM feature), it took me a while to find the "examples" folder in the github repo. Please note those are just some tiny complaints rooted in the fact that I absolutely love what you did here! :)
- leostera 6y ago
  Good points! I've made 3 issues to make sure we get this done. https://github.com/AbstractMachinesLab/caramel/issues/73 https://github.com/AbstractMachinesLab/caramel/issues/73 https://github.com/AbstractMachinesLab/caramel/issues/74 https://github.com/AbstractMachinesLab/caramel/issues/74 https://github.com/AbstractMachinesLab/caramel/issues/75 https://github.com/AbstractMachinesLab/caramel/issues/75 Thanks a lot <3 - you're welcome in our Discord: http://discord.caramel.run http://discord.caramel.run

chromatin 6y ago

This looks very cool, and I have been casually looking into OCaml lately. Please excuse this naïve question, but what is the main value proposition here over plain OCaml: Erlang/BEAM library ecosystem interop, or more something relating to high concurrency? It wasn’t really clear to me from the website/docs.