5 ms·

I think it is misleading to frame this as "behavior". I think what we really have are battle-tested concurrency patterns, which then uses the language feature c

by hosh 4y ago

I think it is misleading to frame this as "behavior". I think what we really have are battle-tested concurrency patterns, which then uses the language feature called "behavior" to implement. This collection of battle-tested concurrency patterns that has been in use for a long time, in a variety of settings, is called OTP. It's why you'll hear long-time Erlang folks talk about how, it's really about OTP. You can implement those same patterns, if you also have lightweight processes, messages, and message queues. There's a Rust library that aims to do exactly that. The new WASM runtime (Firefly, formerly known as Lumen) aims to bring those patterns into the WASM ecosystem, potentially using this kind of concurrency pattern client-side. What's potentially novel is adding in newer concurrency patterns that are in use in the Kubernetes community -- use of selectors, services, scaling with replicasets -- so that, instead of a static supervision tree, it can be a dynamic supervision tree, spread out across the entire pool, even as that pool expands or shrinks.

imachine1980_ 4y ago
Please, can you share the rust library??
- linkdd 4y ago
  Not sure this is what GP is talking about but to implement the actor model in https://letlang.dev https://letlang.dev I use tokio.
- hosh 4y ago
  I don’t remember the name and I don’t remember if it is tokio or something else. However, I did find this: https://docs.rs/genserver/latest/genserver/ https://docs.rs/genserver/latest/genserver/ I will keep looking. Keep in mind, BEAM uses a preemptive actor rather than async, so it avoids the “storms” that can happen when an async reactor runs out of resources. EDIT: but I guess Tokio is capable of preemption through work-stealing too?
  - hosh 4y ago
    This looks interesting too, though no OTP out-of-the-box: https://docs.rs/axiom/latest/axiom/ https://docs.rs/axiom/latest/axiom/
  - verdagon 4y ago
    Can you say more about these storms? Sounds pretty interesting. How does preempting solve it?
    - hosh 4y ago
      I don't know Python or Rust well, but I think this is the problem: https://betterprogramming.pub/the-dangers-of-async-in-python-and-how-to-avoid-them-6e6f98f19f0e https://betterprogramming.pub/the-dangers-of-async-in-python... And this is how some people are solving it: https://async.rs/blog/stop-worrying-about-blocking-the-new-async-std-runtime/ https://async.rs/blog/stop-worrying-about-blocking-the-new-a... This is how it works in BEAM: https://github.com/happi/theBeamBook/blob/master/chapters/scheduling.asciidoc#preemptive-multitasking-in-erts-cooperating-in-c https://github.com/happi/theBeamBook/blob/master/chapters/sc... The commonality is to limit execution time and yield (preempt) so something else can run too.
- bkolobara 4y ago
  I'm the author of something related https://github.com/lunatic-solutions/lunatic https://github.com/lunatic-solutions/lunatic
  - deleted 4y ago
    [deleted]
dangoor 4y ago
Here's an attempt at implementing OTP in Go: https://github.com/ergo-services/ergo https://github.com/ergo-services/ergo
- metadat 4y ago
  Why do you say "attempt"? Does it not work sufficiently well [yet]? Edit: Thanks for the additional context, dangoor. Cheers.
  - dangoor 4y ago
    I haven't tried it, it's just on my radar. It may be great! But OTP has been around a long time, so I'm not going to assume that they've nailed it unless I've either tried it myself or seen folks talking about using it successfully.
- rad_gruchalski 4y ago
  A used this a couple of times in production: https://github.com/asynkron/protoactor-go https://github.com/asynkron/protoactor-go. No problem launching a 100k actors on a laptop.
  - dangoor 4y ago
    That looks cool. Thanks for the link!
hinkley 4y ago
I started using worker threads for some batch processing and development tools recently. We have a couple of tricky bits that pull data from different environments to compare them, so the code has to run 'as' two or more environments and forcing a state change was bug ridden as hell, and spawning a child process was slow enough that we limited the scope of how much we scanned (running n worker threads for n environments dropped a lookup from 1.5s to 350ms). Armed with those successes I turned to thinking about applying something similar to a production Node app, and every time I think about it I realize what I really want is OTP. Each type of processing we do has its own caches, and its own CPU uses, which could dovetail well with worker threads, but I have a chicken and egg problem with trying to move any of that work out of process because I don't get any benefit until 1) none of the workers can silently crash and leave the application nonfunctional and 2) before I'm handling half a dozen concurrent requests per Node process instead of one or two. and 3) before message passing overhead is cheaper It's either too late or too early to rewrite this whole thing in Elixir.
- orf 4y ago
  Perhaps Node is the problem, rather than Elixir being the solution?
  - toolz 4y ago
    If node is the issue here that would make elixir a solution, no? Changing languages (i.e. node is the issue) to get the features described would put you looking at elixir (among other languages that can fill the described properties). Maybe what you mean is that elixir isn't the only solution?
    - orf 4y ago
      Yeah, that’s what I meant: “elixir being the (only) solution”. Guess it was worded ambiguously.
  - hinkley 4y ago
    Sometimes your problem is that you're using a reciprocating saw when you should be using an angle grinder, sometimes your problem is that you bought Ryobi instead of Milwaukee or Makita.
mcphage 4y ago
> This collection of battle-tested concurrency patterns that has been in use for a long time, in a variety of settings, is called OTP. OTP seems like a pretty overloaded acronym—what does it mean here?
- jaggederest 4y ago
  OTP in an Erlang context refers to the Open Telecom Platform - a bit of a misnomer since you can do a lot of things with it that do not inherently relate to telecoms, but that's how it evolved.
  - chc 4y ago
    For context, Erlang itself was originally created at the telecom company Ericsson — the name is short for "Ericsson Language". But as concurrency and networking became bigger parts of software in general, the qualities that made Erlang good for telecom software turned out to be generally useful.
    - thaumasiotes 4y ago
      > the name is short for "Ericsson Language" But just for fun, it's also the name of a Chinese Buddhist god: https://en.wikipedia.org/wiki/Erlang_Shen https://en.wikipedia.org/wiki/Erlang_Shen
    - YouWhy 4y ago
      I think the name was a play on both "Ericsson Language" and the name of Agner Krarup Erlang, who pioneered queuing theory (before it was known as such) to address the fundamental question of inter-exchange capacity planning. The unit "erlang", for "1 busy circuit/resource" is named in honor of that research. https://en.m.wikipedia.org/wiki/Agner_Krarup_Erlang https://en.m.wikipedia.org/wiki/Agner_Krarup_Erlang
- gilbetron 4y ago
  Open Telecom Platform, I think?
- 4y ago
kqr 4y ago
> It's why you'll hear long-time Erlang folks talk about how, it's really about OTP. Indeed, this didn't come as a surprise because I've heard Erlang folks rave about OTP. What was really nice about this article (complemented by some of the comments here) was that it gave me a really nice intuition for what OTP really is, beyond "supervisor trees, y'know".

by hosh 4y ago

imachine1980_ 4y ago

Please, can you share the rust library??

linkdd 4y ago
Not sure this is what GP is talking about but to implement the actor model in https://letlang.dev https://letlang.dev I use tokio.
hosh 4y ago
I don’t remember the name and I don’t remember if it is tokio or something else. However, I did find this: https://docs.rs/genserver/latest/genserver/ https://docs.rs/genserver/latest/genserver/ I will keep looking. Keep in mind, BEAM uses a preemptive actor rather than async, so it avoids the “storms” that can happen when an async reactor runs out of resources. EDIT: but I guess Tokio is capable of preemption through work-stealing too?
- hosh 4y ago
  This looks interesting too, though no OTP out-of-the-box: https://docs.rs/axiom/latest/axiom/ https://docs.rs/axiom/latest/axiom/
- verdagon 4y ago
  Can you say more about these storms? Sounds pretty interesting. How does preempting solve it?
  - hosh 4y ago
    I don't know Python or Rust well, but I think this is the problem: https://betterprogramming.pub/the-dangers-of-async-in-python-and-how-to-avoid-them-6e6f98f19f0e https://betterprogramming.pub/the-dangers-of-async-in-python... And this is how some people are solving it: https://async.rs/blog/stop-worrying-about-blocking-the-new-async-std-runtime/ https://async.rs/blog/stop-worrying-about-blocking-the-new-a... This is how it works in BEAM: https://github.com/happi/theBeamBook/blob/master/chapters/scheduling.asciidoc#preemptive-multitasking-in-erts-cooperating-in-c https://github.com/happi/theBeamBook/blob/master/chapters/sc... The commonality is to limit execution time and yield (preempt) so something else can run too.
bkolobara 4y ago
I'm the author of something related https://github.com/lunatic-solutions/lunatic https://github.com/lunatic-solutions/lunatic
- deleted 4y ago
  [deleted]

dangoor 4y ago

Here's an attempt at implementing OTP in Go: https://github.com/ergo-services/ergo https://github.com/ergo-services/ergo

metadat 4y ago
Why do you say "attempt"? Does it not work sufficiently well [yet]? Edit: Thanks for the additional context, dangoor. Cheers.
- dangoor 4y ago
  I haven't tried it, it's just on my radar. It may be great! But OTP has been around a long time, so I'm not going to assume that they've nailed it unless I've either tried it myself or seen folks talking about using it successfully.
rad_gruchalski 4y ago
A used this a couple of times in production: https://github.com/asynkron/protoactor-go https://github.com/asynkron/protoactor-go. No problem launching a 100k actors on a laptop.
- dangoor 4y ago
  That looks cool. Thanks for the link!

hinkley 4y ago

I started using worker threads for some batch processing and development tools recently. We have a couple of tricky bits that pull data from different environments to compare them, so the code has to run 'as' two or more environments and forcing a state change was bug ridden as hell, and spawning a child process was slow enough that we limited the scope of how much we scanned (running n worker threads for n environments dropped a lookup from 1.5s to 350ms). Armed with those successes I turned to thinking about applying something similar to a production Node app, and every time I think about it I realize what I really want is OTP. Each type of processing we do has its own caches, and its own CPU uses, which could dovetail well with worker threads, but I have a chicken and egg problem with trying to move any of that work out of process because I don't get any benefit until 1) none of the workers can silently crash and leave the application nonfunctional and 2) before I'm handling half a dozen concurrent requests per Node process instead of one or two. and 3) before message passing overhead is cheaper It's either too late or too early to rewrite this whole thing in Elixir.

orf 4y ago
Perhaps Node is the problem, rather than Elixir being the solution?
- toolz 4y ago
  If node is the issue here that would make elixir a solution, no? Changing languages (i.e. node is the issue) to get the features described would put you looking at elixir (among other languages that can fill the described properties). Maybe what you mean is that elixir isn't the only solution?
  - orf 4y ago
    Yeah, that’s what I meant: “elixir being the (only) solution”. Guess it was worded ambiguously.
- hinkley 4y ago
  Sometimes your problem is that you're using a reciprocating saw when you should be using an angle grinder, sometimes your problem is that you bought Ryobi instead of Milwaukee or Makita.

mcphage 4y ago

> This collection of battle-tested concurrency patterns that has been in use for a long time, in a variety of settings, is called OTP. OTP seems like a pretty overloaded acronym—what does it mean here?

jaggederest 4y ago
OTP in an Erlang context refers to the Open Telecom Platform - a bit of a misnomer since you can do a lot of things with it that do not inherently relate to telecoms, but that's how it evolved.
- chc 4y ago
  For context, Erlang itself was originally created at the telecom company Ericsson — the name is short for "Ericsson Language". But as concurrency and networking became bigger parts of software in general, the qualities that made Erlang good for telecom software turned out to be generally useful.
  - thaumasiotes 4y ago
    > the name is short for "Ericsson Language" But just for fun, it's also the name of a Chinese Buddhist god: https://en.wikipedia.org/wiki/Erlang_Shen https://en.wikipedia.org/wiki/Erlang_Shen
  - YouWhy 4y ago
    I think the name was a play on both "Ericsson Language" and the name of Agner Krarup Erlang, who pioneered queuing theory (before it was known as such) to address the fundamental question of inter-exchange capacity planning. The unit "erlang", for "1 busy circuit/resource" is named in honor of that research. https://en.m.wikipedia.org/wiki/Agner_Krarup_Erlang https://en.m.wikipedia.org/wiki/Agner_Krarup_Erlang
gilbetron 4y ago
Open Telecom Platform, I think?
4y ago

kqr 4y ago

> It's why you'll hear long-time Erlang folks talk about how, it's really about OTP. Indeed, this didn't come as a surprise because I've heard Erlang folks rave about OTP. What was really nice about this article (complemented by some of the comments here) was that it gave me a really nice intuition for what OTP really is, beyond "supervisor trees, y'know".