6 ms·

Leveraging Elixir's hot code loading capabilities to modularize a monolithic app

128 points by ronxjansen 1y ago

chantepierre 1y ago
Author here, thanks for sharing ! Happy to answer any questions. I think the article outlines it, but I'm at very low scale, with custom development for every client. I mostly build mini-figmas, collaborative or not, that automate specific document pipelines on top of my software, backed by elixir+liveview (or elixir+vue+channels).
- ralphc 1y ago
  Why Vue over the other JS frameworks?
  - chantepierre 1y ago
    Personal habit, honestly. If I had to re-start I'd probably pick React today, but the Vue switch to a composition API make me like it enough when I was thinking of switching. What I'm truly rooting for is a non-limited Elm.
ipnon 1y ago
Elixir is the best general purpose programming language for distributed systems.
- isodev 1y ago
  It's also the best ecosystem for indie/small team development. With a handful of well-established libraries, one can go a long way without having to reach for separate/dedicated queue/messaging systems, job or workflow orchestrations, even spinning up an ephemeral machine just to run heavy workloads is not complex. Frameworks like Ash make an entire category of server applications a matter of a few declarative modules.
- pclowes 1y ago
  What makes you say that? Honestly asking. I know a team using it to replace ancient massive mainframe based systems with modern distributed systems and the gist is that the language is fine, but mostly ideal for use cases that leverage the ErlangVM or BEAM stack. The downside they run into is the ecosystem isnt there, at least a couple guys wish they had just used Kotlin/Java for library interoperability with so much existing code already built and battle tested for specific purposes.
  - darkmarmot 1y ago
    I think that's a good point. Our largest pain point with Elixir is definitely the size of the community and the associated dearth of niche libraries. The technology behind it, though, is solid enough that once those libraries exist, things really take off. My team wrote several open source medical libraries for Elixir and we've seen it really expand into the healthcare market.
    - ralphc 1y ago
      I'd like to have a look at those, have a github link?
      - darkmarmot 1y ago
        Yes! Thanks for the interest, hope they're helpful! for HL7: https://hexdocs.pm/elixir_hl7/main.html https://hexdocs.pm/elixir_hl7/main.html for MLLP: https://hexdocs.pm/mllp/readme.html https://hexdocs.pm/mllp/readme.html
  - ipnon 1y ago
    To put it simply, the BEAM lets you swallow all of your dependent services into a consistent API, no matter network distance or machine dependability. In Python it feels like my main thread, DB, job queue, and OS are all speaking different languages. With Elixir I don't spend much time at all getting different services to work together, at least an order of magnitude less. Elixir is not perfect, but for me working alone dependency hell was the bottleneck with Python. Now the bottleneck is adding features, which is right where it should be.
- lenkite 1y ago
  Out of curiosity, what does one use for distributed storage in Elixir ?
fud101 1y ago
I thought elixir devs have cooled on the whole hot reload update or is this different?
- conradfr 1y ago
  That seems more about loading dynamic code.
- elitepleb 1y ago
  Elixir removed a jankier https://www.erlang.org/doc/apps/sasl/appup.html https://www.erlang.org/doc/apps/sasl/appup.html mechanism that defined how state is upgraded or downgraded, while watching a directory and recompiling a module automatically or manually from the repl is still common
  - diggan 1y ago
    > while watching a directory and recompiling a module automatically or manually from the repl is still common That makes it sound like the "hot" part has been removed then, and it's just basically a "live reload" rather than "hot code loading", is that right? There is no persistent state and you get a fresh one after the automatic compilation happened?
    - elitepleb 1y ago
      queued messages stay around in the mailboxes, so no state is lost, but don't get migrated/transformed/versioned via the appup mechanism, unless you opt back into it via libraries for it like https://github.com/ausimian/castle https://github.com/ausimian/castle
    - toast0 1y ago
      I've used utility functions in Erlang where I make changes, then compile and load all modified modules... It's absolutely hot loading, there's persistent state, any fully qualified function calls run in the newest module. The gen_server pattern always calls into your behavior module with fully qualified calls, so those are pretty easy to get into new code at a reasonable time. If you write your own service loop, it's pretty common to call ?MODULE:loop() to enable hotloading there too. There's footguns; hotloading is a way to make rapid changes to your system, but sometimes rapid changes isn't what's needed and sometimes the rapid change you make is to go from a partially broken system to a fully broken system. But, there's a virtuous circle when you can make production changes quickly, because you can make a small change and observe and make many follow ups in a single day. With push processes that take a long time, you end up encouraged to make a bigger change one time, otherwise you spend all day waiting for traffic to move between old and new versions, or waiting for instances to deploy, etc.
svkids 1y ago
I wonder how the elixir will do in different temp circumstances

128 points by ronxjansen 1y ago

Author here, thanks for sharing ! Happy to answer any questions. I think the article outlines it, but I'm at very low scale, with custom development for every client. I mostly build mini-figmas, collaborative or not, that automate specific document pipelines on top of my software, backed by elixir+liveview (or elixir+vue+channels).

ralphc 1y ago
Why Vue over the other JS frameworks?
- chantepierre 1y ago
  Personal habit, honestly. If I had to re-start I'd probably pick React today, but the Vue switch to a composition API make me like it enough when I was thinking of switching. What I'm truly rooting for is a non-limited Elm.

ipnon 1y ago

Elixir is the best general purpose programming language for distributed systems.

isodev 1y ago
It's also the best ecosystem for indie/small team development. With a handful of well-established libraries, one can go a long way without having to reach for separate/dedicated queue/messaging systems, job or workflow orchestrations, even spinning up an ephemeral machine just to run heavy workloads is not complex. Frameworks like Ash make an entire category of server applications a matter of a few declarative modules.
pclowes 1y ago
What makes you say that? Honestly asking. I know a team using it to replace ancient massive mainframe based systems with modern distributed systems and the gist is that the language is fine, but mostly ideal for use cases that leverage the ErlangVM or BEAM stack. The downside they run into is the ecosystem isnt there, at least a couple guys wish they had just used Kotlin/Java for library interoperability with so much existing code already built and battle tested for specific purposes.
- darkmarmot 1y ago
  I think that's a good point. Our largest pain point with Elixir is definitely the size of the community and the associated dearth of niche libraries. The technology behind it, though, is solid enough that once those libraries exist, things really take off. My team wrote several open source medical libraries for Elixir and we've seen it really expand into the healthcare market.
  - ralphc 1y ago
    I'd like to have a look at those, have a github link?
    - darkmarmot 1y ago
      Yes! Thanks for the interest, hope they're helpful! for HL7: https://hexdocs.pm/elixir_hl7/main.html https://hexdocs.pm/elixir_hl7/main.html for MLLP: https://hexdocs.pm/mllp/readme.html https://hexdocs.pm/mllp/readme.html
- ipnon 1y ago
  To put it simply, the BEAM lets you swallow all of your dependent services into a consistent API, no matter network distance or machine dependability. In Python it feels like my main thread, DB, job queue, and OS are all speaking different languages. With Elixir I don't spend much time at all getting different services to work together, at least an order of magnitude less. Elixir is not perfect, but for me working alone dependency hell was the bottleneck with Python. Now the bottleneck is adding features, which is right where it should be.
lenkite 1y ago
Out of curiosity, what does one use for distributed storage in Elixir ?

fud101 1y ago

I thought elixir devs have cooled on the whole hot reload update or is this different?

conradfr 1y ago
That seems more about loading dynamic code.
elitepleb 1y ago
Elixir removed a jankier https://www.erlang.org/doc/apps/sasl/appup.html https://www.erlang.org/doc/apps/sasl/appup.html mechanism that defined how state is upgraded or downgraded, while watching a directory and recompiling a module automatically or manually from the repl is still common
- diggan 1y ago
  > while watching a directory and recompiling a module automatically or manually from the repl is still common That makes it sound like the "hot" part has been removed then, and it's just basically a "live reload" rather than "hot code loading", is that right? There is no persistent state and you get a fresh one after the automatic compilation happened?
  - elitepleb 1y ago
    queued messages stay around in the mailboxes, so no state is lost, but don't get migrated/transformed/versioned via the appup mechanism, unless you opt back into it via libraries for it like https://github.com/ausimian/castle https://github.com/ausimian/castle
  - toast0 1y ago
    I've used utility functions in Erlang where I make changes, then compile and load all modified modules... It's absolutely hot loading, there's persistent state, any fully qualified function calls run in the newest module. The gen_server pattern always calls into your behavior module with fully qualified calls, so those are pretty easy to get into new code at a reasonable time. If you write your own service loop, it's pretty common to call ?MODULE:loop() to enable hotloading there too. There's footguns; hotloading is a way to make rapid changes to your system, but sometimes rapid changes isn't what's needed and sometimes the rapid change you make is to go from a partially broken system to a fully broken system. But, there's a virtuous circle when you can make production changes quickly, because you can make a small change and observe and make many follow ups in a single day. With push processes that take a long time, you end up encouraged to make a bigger change one time, otherwise you spend all day waiting for traffic to move between old and new versions, or waiting for instances to deploy, etc.

svkids 1y ago

I wonder how the elixir will do in different temp circumstances