5 ms·

> Elixir is a very nice language to write distributed systems in. Functional in all the right places plus it has OTP. This is the biggest blocker for introduci

by ctvo 4y ago

> Elixir is a very nice language to write distributed systems in. Functional in all the right places plus it has OTP. This is the biggest blocker for introducing Elixir to any company I work at. I don't want to become / hire experts in the OTP and the Erlang VM. I'm ignorant about it in general, but my feeling is it's not only a new language, it's built on abstractions that I'm not sure I'm comfortable owning or operating. Is that wrong and how did you handle the tradeoffs for your company?

bongobingo1 4y ago
You can kinda ignore OTP outside of the initial "run these (erlang) processes when the vm boots", which is a few lines - often scaffolded out automatically. But then when you suddenly need some async task x service x parallelism its there. I guess if you're building a company it does behoove you to at least read a bit about it but you don't need every team member to be an OTP expert. If you can understand how an OS runs processes at the most basic level and how javascripts async/await works you're already most of the way in a practical day-to-day sense.
throwawaymaths 4y ago
The abstractions you need to critically know are map and reduce. It's not functional like Haskell, the limit at which you need to "think about it being functional" is that a value inside a variable can't change from underneath you when you pass it to a function. It pretty quickly changes from "I have think about values not chamging" to "I don't have to worry about values changing"
- ctvo 4y ago
  The abstractions I’m talking about are the Erlang VMs and the primitives like actors that are provided by the OTP. I have cold sweats thinking about needing to debug a non-obvious performance issue and diving into that layer.
  - throwawaymaths 4y ago
    Ok. Well there are a lot of other options for performance and no matter what system you're in (python, ruby, rust, jvm, c++) those kind of performance debugging is going to be a slog, and GenServers are relatively easy to work with and the VM gives you a lot of tools to figure it out. Most people at scale seem to be doing okay with elixir. I will say the one thing that I do see coming up over and over again is OOM errors, but I personally feel that's because there are a few gotchas that juniors don't always know about
eclark 4y ago
Ignore OTP for the most part. Just write good code that works at one level at a time. Then when you hit something that needs OTP I send them towards Designing Elixir Systems with OTP https://pragprog.com/titles/jgotp/designing-elixir-systems-with-otp/ https://pragprog.com/titles/jgotp/designing-elixir-systems-w... This is all about the onboarding experience of engineers coming onto a team. So you have to be careful with the projects that engineers newer to elxir get. But that's part of the cost of a more niche language.
Fire-Dragon-DoL 4y ago
My answer to this is the following: do your software have by any chance, SQL writes and HTTP request to any third party APIs within the same endpoint? Or maybe it uses some background job processing that's not SQL based and you have at least one endpoint where a job is enqueued AND something is written with SQL? If the answer is yes, you already have a distributed systems with all the downsides and none of the mitigations. If that's the case, you can also use Elixir without still knowing anything and have the same level of penalties. It's really hard nowdays to avoid distributed systems, it just happen that people don't realize they have one. The classic Rails + Postgres + Sidekiq is a distributed system
- ctvo 4y ago
  It's not the distributed system bits. It's the heavy abstractions over the distributed system bits that make me question if operating it requires specialized knowledge. I have a performance issue or an edge case with actors from the OTP. How do I debug and resolve it? Say network partitions aren't behaving like I expect and there seems to be data inconsistency or lost. I think you're doing a disservice when you pretend this is the same class of issue as a three tier app you can Google and StackOverflow for. This dependency should raise questions for anyone looking to adopt this, and I want to know how people have attempted to mitigate this vs. handwaving.
  - toast0 4y ago
    The central abstraction isn't IMHO that heavy. It's just this: A process has an isolated execution state, a message queue, and an id. Messages can be added to the queue using the process id (from local or remote processes), and messages can be pulled from the queue by the process (with pattern matching, first matching message is removed from the queue) If you have a performance issue, introspect with erlang:process_info to see what your processes are doing, and language indepedent tools to see what the VM is doing. process_info gives tons of information, IMHO, much more operable than other systems I'm familiar with. Ex: if your process has a large message queue, either it's not processing it's messages fast enough, it's selectively processing messages and leaving junk in the queue, or some combination; you can get inspect the queue with process_info and maybe figure that out. Lots of possibilities for why too slow, maybe it's waiting, maybe it's slow numerically, etc; process_info can tell you what function is running, which is usually helpful, etc. Network partitions and their effects are highly application dependant, but if you have multiple nodes in your existing system, you have to think about it already, or choose to ignore it, which you can also do here, although if you use mnesia, you shouldn't ignore it: the default is for partitioned and then rejoined nodes to stay apart, waiting for an intervention, which may be the only reasonable default (because there's no right answer), but it's not what people usually want.

by ctvo 4y ago

bongobingo1 4y ago

You can kinda ignore OTP outside of the initial "run these (erlang) processes when the vm boots", which is a few lines - often scaffolded out automatically. But then when you suddenly need some async task x service x parallelism its there. I guess if you're building a company it does behoove you to at least read a bit about it but you don't need every team member to be an OTP expert. If you can understand how an OS runs processes at the most basic level and how javascripts async/await works you're already most of the way in a practical day-to-day sense.

throwawaymaths 4y ago

The abstractions you need to critically know are map and reduce. It's not functional like Haskell, the limit at which you need to "think about it being functional" is that a value inside a variable can't change from underneath you when you pass it to a function. It pretty quickly changes from "I have think about values not chamging" to "I don't have to worry about values changing"

ctvo 4y ago
The abstractions I’m talking about are the Erlang VMs and the primitives like actors that are provided by the OTP. I have cold sweats thinking about needing to debug a non-obvious performance issue and diving into that layer.
- throwawaymaths 4y ago
  Ok. Well there are a lot of other options for performance and no matter what system you're in (python, ruby, rust, jvm, c++) those kind of performance debugging is going to be a slog, and GenServers are relatively easy to work with and the VM gives you a lot of tools to figure it out. Most people at scale seem to be doing okay with elixir. I will say the one thing that I do see coming up over and over again is OOM errors, but I personally feel that's because there are a few gotchas that juniors don't always know about

eclark 4y ago

Ignore OTP for the most part. Just write good code that works at one level at a time. Then when you hit something that needs OTP I send them towards Designing Elixir Systems with OTP https://pragprog.com/titles/jgotp/designing-elixir-systems-with-otp/ https://pragprog.com/titles/jgotp/designing-elixir-systems-w... This is all about the onboarding experience of engineers coming onto a team. So you have to be careful with the projects that engineers newer to elxir get. But that's part of the cost of a more niche language.

Fire-Dragon-DoL 4y ago

My answer to this is the following: do your software have by any chance, SQL writes and HTTP request to any third party APIs within the same endpoint? Or maybe it uses some background job processing that's not SQL based and you have at least one endpoint where a job is enqueued AND something is written with SQL? If the answer is yes, you already have a distributed systems with all the downsides and none of the mitigations. If that's the case, you can also use Elixir without still knowing anything and have the same level of penalties. It's really hard nowdays to avoid distributed systems, it just happen that people don't realize they have one. The classic Rails + Postgres + Sidekiq is a distributed system

ctvo 4y ago
It's not the distributed system bits. It's the heavy abstractions over the distributed system bits that make me question if operating it requires specialized knowledge. I have a performance issue or an edge case with actors from the OTP. How do I debug and resolve it? Say network partitions aren't behaving like I expect and there seems to be data inconsistency or lost. I think you're doing a disservice when you pretend this is the same class of issue as a three tier app you can Google and StackOverflow for. This dependency should raise questions for anyone looking to adopt this, and I want to know how people have attempted to mitigate this vs. handwaving.
- toast0 4y ago
  The central abstraction isn't IMHO that heavy. It's just this: A process has an isolated execution state, a message queue, and an id. Messages can be added to the queue using the process id (from local or remote processes), and messages can be pulled from the queue by the process (with pattern matching, first matching message is removed from the queue) If you have a performance issue, introspect with erlang:process_info to see what your processes are doing, and language indepedent tools to see what the VM is doing. process_info gives tons of information, IMHO, much more operable than other systems I'm familiar with. Ex: if your process has a large message queue, either it's not processing it's messages fast enough, it's selectively processing messages and leaving junk in the queue, or some combination; you can get inspect the queue with process_info and maybe figure that out. Lots of possibilities for why too slow, maybe it's waiting, maybe it's slow numerically, etc; process_info can tell you what function is running, which is usually helpful, etc. Network partitions and their effects are highly application dependant, but if you have multiple nodes in your existing system, you have to think about it already, or choose to ignore it, which you can also do here, although if you use mnesia, you shouldn't ignore it: the default is for partitioned and then rejoined nodes to stay apart, waiting for an intervention, which may be the only reasonable default (because there's no right answer), but it's not what people usually want.