6 ms·

> Node-RED is a amazing[*] tool for creating flows that describe concurrent processing, it is just a shame the NodeJS is single threaded. So why not use somethi

> Node-RED is a amazing[*] tool for creating flows that describe concurrent processing, it is just a shame the NodeJS is single threaded. So why not use something that is multi-process from the ground up? That's great! But now that we are doing this, it kind of makes me wish that it was not multi-processing and Erlang, but a more mainstream language with a better library ecosystem and focused on multi-threading instead, Rust comes to mind but there could be a better one. Is there a visual flow programming language based on Rust that is relatively mature?

bheadmaster 1y ago
> it kind of makes me wish that it was not multi-processing and Erlang, but a more mainstream language with a better library ecosystem and focused on multi-threading instead, Rust comes to mind but there could be a better one Go sounds like a good candidate.
- Intermernet 1y ago
  There are 3 or 4 Go FBP packages. It seems to be completely capable, but FBP itself doesn't seem to have reached enough critical mass for them to be popular. https://github.com/flowbase/flowbase https://github.com/flowbase/flowbase is one, and it has links to to some others.
Towaway69 1y ago
> wish that it was not multi-processing and Erlang Hehe - honestly I chose Erlang because I love the syntax and secondly because it is so niche. Flow based programming is niche, visual FBP is nicher, Node-RED is even nicher and Erlang is niche - it's niche all the way down ;) Have a look at Elixir which is Erlang with Ruby syntax and also has a large community. Also the intention is to be using the flow editor and not the lanuage underneath, so the end user won't even know that it's Erlang or NodeJS - I aim t stay compatible with Node-RED so that flows are actually interchangeable. So to an certain extent it doesn't matter which language is used. I did chose Erlang also because it is well suited to flow based programming being message based and flow based programming is all about passing immutable objects amongst independent processes .... that's Erlang in a nutshell.
notpushkin 1y ago
Note that this uses (I hope!) Erlang processes, which are not to be confused with OS processes.
- Towaway69 1y ago
  Yes Erlang processes all the way down!
- macintux 1y ago
  Yep. I researched JVM and .NET thread sizes once, to compare/contrast with Erlang processes, and the difference is ludicrous. Erlang: ~1k 64-bit .NET thread: 4MB
  - notpushkin 1y ago
    Whoa!
  - SigmundA 1y ago
    .Net uses real OS threads and that 4mb is the stack not the heap which must be allocated on thread creation. The heap is shared in the process amongst all the threads whereas Erlang is separate lightweight green thread/processes with dynamic heap and stack per process. .Net originates in the Windows world where new processes are expensive but threads are much cheaper however they still use native stacks for iterop and hardware protection, stack must be contiguous so it is fixed at thread creation this is typical for all native os threads. Erlang is a VM and runs on top of of processes and threads but exposes its own process model to the language that does not map 1:1 and therefore does not have a fixed stack size allowing many light weight processes. It pays an indirection penalty in performance for this but allows some nice stuff. Java started with green threads and abandoned them, both .Net and Java use asynchronous concepts instead to get high concurrency now days cooperatively releasing threads and rentering them. There was talk of trying green threads out in .Net and bringing them back in Java for ergonomics compared to async, these green threads would not have such large fixed stacks and many more could be made similar to Erlang processes.
    - 1y ago
fnord77 1y ago
seems like a jvm-based one would also fit that requirement. None that I can find
rdtsc 1y ago
> makes me wish that it was not multi-processing and Erlang, but a more mainstream language with a better library ecosystem and focused on multi-threading instead, Rust comes to mind but there could be a better one. I am not quite following, why would we drop multi-processing with isolated tiny heaps with a few KBs each and move to multi-threading with a large multi-megabyte stacks per thread and all sharing and writing to the same heap? It seems like a step backward. > Rust comes to mind but there could be a better one. I can see the excitement, but at the same time this is becoming a meme. I mean the drive-by comments of "why don't your rewrite it in Rust" on every repo or "show HN" post. It's kind of like nodejs from years past: "why don't your write in nodejs, it's webscale!". The problem is, after a while it starts to have an inverse dose response: it makes people run away from Rust just based on that kind of stuff.
- SigmundA 1y ago
  >I am not quite following, why would we drop multi-processing with isolated tiny heaps with a few KBs each and move to multi-threading with a large multi-megabyte stacks per thread and all sharing and writing to the same heap? It seems like a step backward. Performance typically, hard to work in parallel on large amount of data performantly without multiple thread sharing a heap and you typically don't need large amount of threads because you don't actually have that many real cores to run them on. Lots of little share nothing process are great conceptually but that does create significant overhead.
  - rdtsc 1y ago
    > Lots of little share nothing process are great conceptually but that does create significant overhead. It doesn't, really? I have clusters running 1M+ Erlang processes comfortably per node. > you typically don't need large amount of threads Exactly, that's why Erlang only spawns just the right amount of threads. One scheduler thread per CPU, then a bunch of long running CPU task threads (same number as CPUs as well), plus some to do IO (10-20) and that's it.
    - SigmundA 1y ago
creata 1y ago
> a better library ecosystem For what it's worth, Erlang's standard library is decently "batteries-included". The C interop also isn't terrible.[1] [0]: https://www.erlang.org/doc/apps/stdlib/api-reference.html https://www.erlang.org/doc/apps/stdlib/api-reference.html [1]: https://www.erlang.org/doc/system/tutorial.html https://www.erlang.org/doc/system/tutorial.html

bheadmaster 1y ago

> it kind of makes me wish that it was not multi-processing and Erlang, but a more mainstream language with a better library ecosystem and focused on multi-threading instead, Rust comes to mind but there could be a better one Go sounds like a good candidate.

Intermernet 1y ago
There are 3 or 4 Go FBP packages. It seems to be completely capable, but FBP itself doesn't seem to have reached enough critical mass for them to be popular. https://github.com/flowbase/flowbase https://github.com/flowbase/flowbase is one, and it has links to to some others.

Towaway69 1y ago

> wish that it was not multi-processing and Erlang Hehe - honestly I chose Erlang because I love the syntax and secondly because it is so niche. Flow based programming is niche, visual FBP is nicher, Node-RED is even nicher and Erlang is niche - it's niche all the way down ;) Have a look at Elixir which is Erlang with Ruby syntax and also has a large community. Also the intention is to be using the flow editor and not the lanuage underneath, so the end user won't even know that it's Erlang or NodeJS - I aim t stay compatible with Node-RED so that flows are actually interchangeable. So to an certain extent it doesn't matter which language is used. I did chose Erlang also because it is well suited to flow based programming being message based and flow based programming is all about passing immutable objects amongst independent processes .... that's Erlang in a nutshell.

notpushkin 1y ago

Note that this uses (I hope!) Erlang processes, which are not to be confused with OS processes.

Towaway69 1y ago
Yes Erlang processes all the way down!
macintux 1y ago
Yep. I researched JVM and .NET thread sizes once, to compare/contrast with Erlang processes, and the difference is ludicrous. Erlang: ~1k 64-bit .NET thread: 4MB
- notpushkin 1y ago
  Whoa!
- SigmundA 1y ago
  .Net uses real OS threads and that 4mb is the stack not the heap which must be allocated on thread creation. The heap is shared in the process amongst all the threads whereas Erlang is separate lightweight green thread/processes with dynamic heap and stack per process. .Net originates in the Windows world where new processes are expensive but threads are much cheaper however they still use native stacks for iterop and hardware protection, stack must be contiguous so it is fixed at thread creation this is typical for all native os threads. Erlang is a VM and runs on top of of processes and threads but exposes its own process model to the language that does not map 1:1 and therefore does not have a fixed stack size allowing many light weight processes. It pays an indirection penalty in performance for this but allows some nice stuff. Java started with green threads and abandoned them, both .Net and Java use asynchronous concepts instead to get high concurrency now days cooperatively releasing threads and rentering them. There was talk of trying green threads out in .Net and bringing them back in Java for ergonomics compared to async, these green threads would not have such large fixed stacks and many more could be made similar to Erlang processes.
  - 1y ago

fnord77 1y ago

seems like a jvm-based one would also fit that requirement. None that I can find

rdtsc 1y ago

> makes me wish that it was not multi-processing and Erlang, but a more mainstream language with a better library ecosystem and focused on multi-threading instead, Rust comes to mind but there could be a better one. I am not quite following, why would we drop multi-processing with isolated tiny heaps with a few KBs each and move to multi-threading with a large multi-megabyte stacks per thread and all sharing and writing to the same heap? It seems like a step backward. > Rust comes to mind but there could be a better one. I can see the excitement, but at the same time this is becoming a meme. I mean the drive-by comments of "why don't your rewrite it in Rust" on every repo or "show HN" post. It's kind of like nodejs from years past: "why don't your write in nodejs, it's webscale!". The problem is, after a while it starts to have an inverse dose response: it makes people run away from Rust just based on that kind of stuff.

SigmundA 1y ago
>I am not quite following, why would we drop multi-processing with isolated tiny heaps with a few KBs each and move to multi-threading with a large multi-megabyte stacks per thread and all sharing and writing to the same heap? It seems like a step backward. Performance typically, hard to work in parallel on large amount of data performantly without multiple thread sharing a heap and you typically don't need large amount of threads because you don't actually have that many real cores to run them on. Lots of little share nothing process are great conceptually but that does create significant overhead.
- rdtsc 1y ago
  > Lots of little share nothing process are great conceptually but that does create significant overhead. It doesn't, really? I have clusters running 1M+ Erlang processes comfortably per node. > you typically don't need large amount of threads Exactly, that's why Erlang only spawns just the right amount of threads. One scheduler thread per CPU, then a bunch of long running CPU task threads (same number as CPUs as well), plus some to do IO (10-20) and that's it.
  - SigmundA 1y ago

creata 1y ago

> a better library ecosystem For what it's worth, Erlang's standard library is decently "batteries-included". The C interop also isn't terrible.[1] [0]: https://www.erlang.org/doc/apps/stdlib/api-reference.html https://www.erlang.org/doc/apps/stdlib/api-reference.html [1]: https://www.erlang.org/doc/system/tutorial.html https://www.erlang.org/doc/system/tutorial.html