5 ms·

Show HN: Visual flow-based programming for Erlang, inspired by Node-RED

243 points by Towaway69 1y ago

Hi There, Erlang-RED has been my project for the last couple of months and I would love to get some feedback from the HN community. The idea is to take advantage of Erlangs message passing and low overhead processes to have true concurrency in Node-RED flows. Plus also to bring low-code visual flow-based programming to Erlang.

deleted 1y ago
[deleted]
tzarko 1y ago
This is neat - maybe move the example section higher up; it looks great and gives a much better idea of what the system looks like visually, which to me is key in node-based envs.
- Towaway69 1y ago
  One example would be the website I've created using Erlang-RED[1] But it's early days so I've yet come up with good examples. I would really like to get some examples from the Erlang community and implement those, so if anyone has any ideas please contact me! [1] https://red-erik.org https://red-erik.org
js4ever 1y ago
Congratulations for the launch! Also I never heard of that license before: "LICENSE - DON'T DO EVIL" https://github.com/gorenje/erlang-red?tab=License-1-ov-file#readme https://github.com/gorenje/erlang-red?tab=License-1-ov-file#... Also I recommend you to put screenshots higher in the readme and also provide real world use case instead of fully abstract examples
- Towaway69 1y ago
  Thanks for the suggestion, I’ll move the examples up and add a link to a running instance. The license is a reminder that open source software isn’t free, in this case, I would like folks to think about what is evil. That’s the price tag. The license isn’t enforceable and won’t be enforced.
  - tough 1y ago
    Hopefully karma takes care of it heh
  - pvg 1y ago
    You should absolutely use whatever license you want but the actual price tag a nonstandard license adds is 'far fewer people will be able to try or contribute to your project'. Which might be totally ok for your purposes, just pointing it out since many people who dabble in 'license as messaging' for fun aren't aware of the impact/implications.
    - Towaway69 1y ago
      This has become clear since I started using the license, fortunately what I do is so niche that few contribute :) I understand the implications but I also don't like big-tech to stealing my code and leaving me out in the cold. In sense, this license is a snub of the wall-gardens that big-tech has become. This software is something that they can't simple integrate into their product and sell on for profit. It stays out of the wall gardens.
      - pvg 1y ago
        Oh I'm definitely not arguing with you over what license you ought to have on your own project for your own reasons. You want a zany license, let your freak flag fly!
  - adastra22 1y ago
    Licenses are enforceable and will prevent many people from collaborating with you, due to the burden of legal review.
- deleted 1y ago
  [deleted]
  - deleted 1y ago
    [deleted]
- 90s_dev 1y ago
  About 15 years ago I made a custom MIT or BSD license that added: "You agree to think carefully and always reflect about what you do and why you do it" or something to that effect. I thought I was so clever at the time.
  - 90s_dev 1y ago
    Now I'm on the fence about adding philosophy to software licenses, even if only in a joking way. It can be fun and even thought provoking, but it could get in the way of genuine business adoption for basically no return (you never know if it helps influence anyone for the better, and most likely it doesn't).
    - SoftTalker 1y ago
      Any license that isn't a word-for-word match to one of the "approved" licenses can potentially trigger a need for legal review, depending on the enterprise. Lawyers are expensive so quirky licenses can be a deal-killer for any customers who take licenses seriously. If you want to be funny, put an easter egg in your code, don't mess with your license.
      - Towaway69 1y ago
        Easter eggs in code that break things are evil! I don't write code for corporates, so my license is purely fictive. I cannot enforce my license but I can prevent corporates from taking my code and wrapping it into a product and selling it on for a profit. While not passing on a cent to me. If a corporate wishes to use my code, then they are welcome to pay me a license fee or a one-off payment for a non-distributable license.
    - Towaway69 1y ago
      My inspiration was came from Douglas Crockford and the JSONlint[1] license. Why not have a message? I mean if big-tech won't use my software because they legally think they might do evil with it, so be it. Do I really want big-tech to wrap my software into a product and sell it for profit while not giving me a cent because what I did was share my code without strings attached? I don't know. I would like to make this place just that little bit better and if if that's a license that makes folks think about what is evil, heck why not! [1] https://gist.github.com/kemitchell/fdc179d60dc88f0c9b76e5d38fe47076 https://gist.github.com/kemitchell/fdc179d60dc88f0c9b76e5d38...
- deleted 1y ago
  [deleted]
- pydry 1y ago
  It's a cool license in that it triggered Google enough to bar the use of jslint and led IBM's lawyers to seek out a special exemption to allow them and their customers to use JSON for evil. No company's PR will ever tell you what the soul of a company is like but their lawyers will indirectly tell you everything.
  - adastra22 1y ago
    That’s a ridiculous take. The fact is “evil” is so ill defined that it would have been an unlimited liability. That’s a reflection of the world, not IBM.
    - pydry 1y ago
      Plenty (in not most) legal cases hinge upon non specifically defined terms hence lot of lawyering is not "this is definitely legal/illegal" but "what's the risk of this going wrong if we face a jury?" Companies take tens of thousands of legal risks every day and they single out particular risks over others to try and indemnify themselves because they think that risk is serious. Theyre not going to admit that theyre worried that the company is evil enough to qualify under a reasonable person's interpretation but thats what theyre thinking. It's the same with your employment contract. The level of nasty bullshit they put in there ("lawyers made us!") is probably the most accurate meter of how horrible (or not) the company will be towards you as an employer.
concerndc1tizen 1y ago
I've been interested in this for years. Are there other similar projects that you're familiar with? Perhaps targeting other languages? What are the major problems with this approach to programming? Are large programs hard to work with?
- Towaway69 1y ago
  I know of Py-Red[1] which is an attempt to do the same but with Python. Using Node-RED as a frontend and something else as a backend, no I haven't heard of anything similar. Main problems with visual flow based programming is the tooling: there aren't any good tools for doing visual comparison nor visual version control. Things like GitHub aren't able to render flow code as seen in the editor. For Node-RED that means that the Json that defines a flow can be compared but that soon becomes meaningless because semantic changes get mixed with visual changes, i.e., that the x,y coordinates of a node have changed - is meaningless for the logic of the code. This means that code sharing and collaborative development is difficult purely visually but only because tooling is missing. Compared to textual programming before things like SourceForge or Git and we had the same issues as we do with visual coding now. Node-RED has a bunch of features to make large programs maintainable. Things like link nodes that allow jumping between flows also allow code reuse. There also subflows which encapsulate repeated code and can be referenced from anywhere. That's also why I chose Node-RED for this because it seems to be the most mature and best maintained low-code visual flow-based programming tool. There are others such as n8n (which seems to be the other biggie) but they lack the generalness of Node-RED. I can create web sites with Node-RED while and the same time controlling my LEDs via Raspberry installation. Node-RED is extremely flexible. [1]: https://github.com/mdkrieg/py-red https://github.com/mdkrieg/py-red
  - regularfry 1y ago
    Speaking of controlling LEDs, there's also https://www.atomvm.net/doc/main/index.html https://www.atomvm.net/doc/main/index.html. From a very quick scan it looks like it would need some fiddling (especially around the filesystem) to get it to work, but being able to scale down to smaller devices than a full pi would be very, very interesting.
smartmic 1y ago
> No Artificial Intelligence was harmed in the creation of this codebase. This codebase is old skool search engine (ddg), stackoverflow, blog posts and RTFM technology. Good. To me, that also means that no human intelligence was degraded as a result. It sounds like a quality badge.
- deleted 1y ago
  [deleted]
thinkyfish 1y ago
I was just thinking of making something like this last night but using FLENG instead of Erlang ;) I'm really glad more people are taking inspiration from Node-RED, I will definitely be playing with this!
- Towaway69 1y ago
  FLENG[1] - Wow, "low level concurrent logic programming language descended from Prolog" - sounds like an easy swap out for Erlang since they have the same heritage. ;) [1] http://www.call-with-current-continuation.org/fleng/fleng.html http://www.call-with-current-continuation.org/fleng/fleng.ht...
arresin 1y ago
Can someone recommend an Erlang course / book?
- Towaway69 1y ago
  I keep running into learn you some erlang[1] which seems to be a good online resource. The author of which also wrote the "Erlang in Anger"[2] book. Then there is the BEAM book[3] which provides an deep dive into Erlang and the BEAM virtual machine. I'm not the best person to ask since I'm only starting out on Erlang! I much prefer to code visually :) [1] https://learnyousomeerlang.com https://learnyousomeerlang.com [2] https://www.heroku.com/blog/erlang-in-anger/ https://www.heroku.com/blog/erlang-in-anger/ [3] https://blog.stenmans.org/theBeamBook/ https://blog.stenmans.org/theBeamBook/
  - arresin 1y ago
    Thanks for the suggestions! I looked at "learn you some erlang" and I like the style and humor of it.
- randito 1y ago
  You may want to consider Elixir instead. It has an easier syntax and is (don't quote me on this) equivalent in function and purpose with Erlang. Plus you get lots of other goodies like LiveView (notebooks) and a good web stack (Phoenix).
  - Towaway69 1y ago
    I prefer Erlang because it makes the uniqueness of its paradigm clear. Tail recursion, function matching and, bang, everything is a message! Elixir makes everything seem like Ruby code and many of those Erlang concepts are hidden away, which is also ok but also takes much away. My aim with ErlangRED is that both are supported, there is a repository for Elixir code[1] that is included in ErlangRED - so I don't take sides, rather I would like to take advantage of both paradigms. [1] https://github.com/gorenje/erlang-red-elixir-helpers https://github.com/gorenje/erlang-red-elixir-helpers
paddy_m 1y ago
Nice work! A couple of comments 1. You don't explain what flow based programming is or when it should be used. It looks like it represents some type of data pipeline, but I'm not sure. 2. Who is this repo for? PL enthusiasts, tool builders of type $X? 3. What are operators? 4. I understand that this is supposed to execute on an Erlang cluster/VM... Could you build a JS implementation of the functions so that you could build interactive demos in a browser? The lack of multithreading in JS doesn't rally matter, you just want to be able to send 10 line examples from one function to another. 5. A fun challenge for documenting projects like this, explain it as you would to a non-technical family member. It forces you to define the other things in erlang in a very direct and simple way. 6. Make a video explaining the project.
- paddy_m 1y ago
  Ahh, you didn't create Node-RED editor. That's an external project. I didn't understand this at first, and probably explains a lot of my confusion https://nodered.org/ https://nodered.org/
  - Towaway69 1y ago
    No, that much talent I don't have :) Node-RED has a create frontend that just works and the APIs to the server are clearly defined. So it's relatively easy to swap out the backend - that's what I did. Erlang has many features that make flow based programming (as implemented by Node-RED) simpler - e.g., message passing builtin and cheap processes and cheap concurrency.
- adastra22 1y ago
  Why not just use Node-Red in that case?
  - Towaway69 1y ago
    NodeJS isn't truly concurrent, it is event-based single threaded - fast but not concurrent. I use Node-RED every day! This is an exercise in applying the concept of flow based programming to another programming language. For me, flow based programming is like object oriented programming or functional programming or procedural ... etc. So the idea is to take the paradigm and apply to Erlang. Clarification: I've been using everyday Node-RED for last three years and have built many things with it - including my mind map!
nisten 1y ago
the hivemind thinks alike
Arch-TK 1y ago
I saw Node-RED recently, and now this. While I think these are cool (and I would rather Erlang than NodeJS), is there some code based system for these kinds of things? I know I can just write code, but I mean a DSL or some library specifically oriented around solving exactly these kinds of event driven sequencing/action problems.
- Towaway69 1y ago
  Not to my knowledge, the closest thing to a DSL for Node-RED is its flows.json format for storing flow configurations. It's a very simple array-of-object format with the graph/flow definitions defined by node ids and wires. It is fairly simple to create a minimalist interpretor for that format. Also NodeRED is a very stable piece of software that has been around for over ten years. Yet I can take a flows.json file from today and have it run on the first releases of Node-RED, i.e., the format is backward compatible yet easily extendable. > event driven sequencing/action problems. Flow based programming[1] isn't strictly event driven, it's implemented that way in NodeJS because NodeJS does not have the concept of messages, only events. Erlang has message passing between independent processes and therefore, IMHO, better suited to FBP. [1] https://jpaulm.github.io/fbp/index.html https://jpaulm.github.io/fbp/index.html
  - Arch-TK 1y ago
    Firstly, thanks for the link and information on more of the roots of this kind of programming, I couldn't find this when I looked. Regarding NodeRED. I don't dispute that it's stable, but my experience actually running any nodejs based software is that it's all heavily designed for running out of a container and a massive PITA to maintain otherwise. I already have enough hassle running zigbee2mqtt on my FreeBSD home server which wouldn't work in a linux jail because I couldn't figure out how to expose the zigbee dongle to it. Secondly, the reason I wanted to run one of these was not because I wanted to use it to automate my house, but instead because I wanted to understand how it works so I could replicate it in my own weird way.
- NortySpock 1y ago
  I suggest one of Benthos' descendants, either Bento or RedPanda Connect. Configuration driven, with a transformation DSL, and good documentation (I prefer the Bento documentation over RedPanda Connect's documentation). And Benthos explicitly refused to ack a consumed message upstream unless it had been acknowledged downstream, so it wouldn't drop messages on the floor unless you explicitly told it to.
pjmlp 1y ago
I am a big fan of visual programming, and also think visual and digital circuit design tools overlap quite nicely with functional programming languages. The mess most people complain about in visual languages, also happens on text ones, the biggest difference is that visual ones make it quite clear the code lacks modularity. Get to package stuff into reusable ICs more often. All the best for the project.
- Towaway69 1y ago
  > visual ones make it quite clear the code lacks modularity. NodeRED has the concept of subflows to encapsulate reusable code. There are a number of features that NodeRED has that make it more usable for long term projects but it's a steep learning curve. I've been using Node-RED everyday for the last three years to find the edge, something that cannot be done but there has always been a visual solution. I'm starting to think Node-RED might be turing complete ;) But I have to admit, I did build a lot of tooling around NodeRED to make that happen - visual flow comparison[1] was something that was missing, introspection of message flow[2] and visual code management[3] - all these things I need to maintain an overview of what I am doing. [1] https://flows.nodered.org/node/@gregoriusrippenstein/node-red-contrib-flowcompare https://flows.nodered.org/node/@gregoriusrippenstein/node-re... [2] https://flows.nodered.org/node/@gregoriusrippenstein/node-red-contrib-introspection https://flows.nodered.org/node/@gregoriusrippenstein/node-re... [3] https://flowhub.org/ https://flowhub.org/ > All the best for the project. Thank you!
- tlodboy 1y ago
  Visual programming helps automate flows in circuit design and I feel adoption would be higher as well since most circuit designers for some reason prefer GUI
- deleted 1y ago
  [deleted]
mystraline 1y ago
I was an early adopter of NodeRed. Earlier, it worked exceptionally well. Now? Not so much. Well, that's not exactly true. Base NodeRed works as well as before. But the libraries of modules to interface with all sorts of websites, APIs, hardware, and other stuff is rotten to the core. Most plugins/js modules either just don't work, or 'successfully fail'. The easier fail case is where the module can't be installed due to ancient (6mo or older JS, sigh) modules. I abandoned NR because its basically a hyper-speed bitrot due to terrible library module versioning. And I didn't want to reinvent the wheel on every system I wanted to touch.
- wslh 1y ago
  I don't think this is easily to solve, in general. Similar orchestrators (e.g. n8n) have the same issues because there are a lot of components dependencies that change with time and there is no real cohesion between the core and all kind of plugins. Probably a future "contracts/manifests" linking orchestrators with components could help.
- Towaway69 1y ago
  I wrote an article about this[1] and you are definitely right: a lot of packages are rotting away a bit however, because NodeRED does a lot to stay backward compatible, older packages still work. The oldest ones I'm actively using are five or more years old, i.e., haven't been touched nor updated - still work. What I tried to say in the article is the same as you say: base NodeRED, the core works really well and has great features - no questions. And even if packages die, the core will still remain usable and that makes a difference. Its a bit like saying Linux is a pile of broken bits because the ls command has been updated in ten years: Linux will always work and those commands that are based on the core will continue to work becausae the Linux kernel will largely remain backward compatible. Packages fail when they have external dependencies that change but code that is solely based on the core will continue to work. [1] https://blog.openmindmap.org/blog/crunchy-numbers https://blog.openmindmap.org/blog/crunchy-numbers
- 01100011 1y ago
  Not surprising. Whenever you have a project like NR or HA you have a ton of barely working glue code written by people who just want to get it working and move on without any sort of commitment to maintenance. It allows these projects to rapidly expand their support but then that support quickly rots unless the core team assumes responsibility for maintenance. I really want to mess with home automation again but this sort of low software quality and resulting instability and maintenance hassles makes it not worth the effort.
oersted 1y ago
> Node-RED is a amazing[*] tool for creating flows that describe concurrent processing, it is just a shame the NodeJS is single threaded. So why not use something that is multi-process from the ground up? That's great! But now that we are doing this, it kind of makes me wish that it was not multi-processing and Erlang, but a more mainstream language with a better library ecosystem and focused on multi-threading instead, Rust comes to mind but there could be a better one. Is there a visual flow programming language based on Rust that is relatively mature?
- bheadmaster 1y ago
  > it kind of makes me wish that it was not multi-processing and Erlang, but a more mainstream language with a better library ecosystem and focused on multi-threading instead, Rust comes to mind but there could be a better one Go sounds like a good candidate.
  - Intermernet 1y ago
    There are 3 or 4 Go FBP packages. It seems to be completely capable, but FBP itself doesn't seem to have reached enough critical mass for them to be popular. https://github.com/flowbase/flowbase https://github.com/flowbase/flowbase is one, and it has links to to some others.
- Towaway69 1y ago
  > wish that it was not multi-processing and Erlang Hehe - honestly I chose Erlang because I love the syntax and secondly because it is so niche. Flow based programming is niche, visual FBP is nicher, Node-RED is even nicher and Erlang is niche - it's niche all the way down ;) Have a look at Elixir which is Erlang with Ruby syntax and also has a large community. Also the intention is to be using the flow editor and not the lanuage underneath, so the end user won't even know that it's Erlang or NodeJS - I aim t stay compatible with Node-RED so that flows are actually interchangeable. So to an certain extent it doesn't matter which language is used. I did chose Erlang also because it is well suited to flow based programming being message based and flow based programming is all about passing immutable objects amongst independent processes .... that's Erlang in a nutshell.
fnord77 1y ago
for a visual environment, I was expecting more images to show me what this tool is
- Towaway69 1y ago
  There is a live demo[1] if you want to try it out! For me, the focus is more on the Erlang part than the visual part since that's basically NodeRED. [1] https://ered.fly.dev/node-red https://ered.fly.dev/node-red
  - fnord77 1y ago
    thank you, very neat
lxe 1y ago
[flagged]
- nathanappere 1y ago
  It does if you have blocking calls or you cannot yield to the event loop in time consuming code.
- ummonk 1y ago
  Are you asserting that single threaded execution of concurrent code is just as performant as multi-process execution?
  - Towaway69 1y ago
    Concurrency isn’t possible if there is only a single thread is my interpretation.
    - hotpocket777 1y ago
      That is not true at all. Concurrency is a way of writing programs and is an orthogonal concern to how the program gets executed.
  - lxe 1y ago
    It certainly can be.
- deleted 1y ago
  [deleted]
aantix 1y ago
I would move your sample images to the top of the README. Don't bury the lede.
gitroom 1y ago
[dead]
curtisszmania 1y ago
[dead]
nilslindemann 1y ago
The thing which makes me be annoyed with Yours and the Node-RED site, is that you people do not define terms, before using them. Like "flow". What is this? A program? Please maintain a glossary.
- nilslindemann 1y ago
  Okay, It seems you do, https://flowfuse.com/node-red/terminology/ https://flowfuse.com/node-red/terminology/, https://nodered.org/docs/user-guide/concepts https://nodered.org/docs/user-guide/concepts, so please link to it.

243 points by Towaway69 1y ago

deleted 1y ago

[deleted]

tzarko 1y ago

This is neat - maybe move the example section higher up; it looks great and gives a much better idea of what the system looks like visually, which to me is key in node-based envs.

Towaway69 1y ago
One example would be the website I've created using Erlang-RED[1] But it's early days so I've yet come up with good examples. I would really like to get some examples from the Erlang community and implement those, so if anyone has any ideas please contact me! [1] https://red-erik.org https://red-erik.org

js4ever 1y ago

Congratulations for the launch! Also I never heard of that license before: "LICENSE - DON'T DO EVIL" https://github.com/gorenje/erlang-red?tab=License-1-ov-file#readme https://github.com/gorenje/erlang-red?tab=License-1-ov-file#... Also I recommend you to put screenshots higher in the readme and also provide real world use case instead of fully abstract examples

Towaway69 1y ago
Thanks for the suggestion, I’ll move the examples up and add a link to a running instance. The license is a reminder that open source software isn’t free, in this case, I would like folks to think about what is evil. That’s the price tag. The license isn’t enforceable and won’t be enforced.
- tough 1y ago
  Hopefully karma takes care of it heh
- pvg 1y ago
  You should absolutely use whatever license you want but the actual price tag a nonstandard license adds is 'far fewer people will be able to try or contribute to your project'. Which might be totally ok for your purposes, just pointing it out since many people who dabble in 'license as messaging' for fun aren't aware of the impact/implications.
  - Towaway69 1y ago
    This has become clear since I started using the license, fortunately what I do is so niche that few contribute :) I understand the implications but I also don't like big-tech to stealing my code and leaving me out in the cold. In sense, this license is a snub of the wall-gardens that big-tech has become. This software is something that they can't simple integrate into their product and sell on for profit. It stays out of the wall gardens.
    - pvg 1y ago
      Oh I'm definitely not arguing with you over what license you ought to have on your own project for your own reasons. You want a zany license, let your freak flag fly!
- adastra22 1y ago
  Licenses are enforceable and will prevent many people from collaborating with you, due to the burden of legal review.
deleted 1y ago
[deleted]
- deleted 1y ago
  [deleted]
90s_dev 1y ago
About 15 years ago I made a custom MIT or BSD license that added: "You agree to think carefully and always reflect about what you do and why you do it" or something to that effect. I thought I was so clever at the time.
- 90s_dev 1y ago
  Now I'm on the fence about adding philosophy to software licenses, even if only in a joking way. It can be fun and even thought provoking, but it could get in the way of genuine business adoption for basically no return (you never know if it helps influence anyone for the better, and most likely it doesn't).
  - SoftTalker 1y ago
    Any license that isn't a word-for-word match to one of the "approved" licenses can potentially trigger a need for legal review, depending on the enterprise. Lawyers are expensive so quirky licenses can be a deal-killer for any customers who take licenses seriously. If you want to be funny, put an easter egg in your code, don't mess with your license.
    - Towaway69 1y ago
      Easter eggs in code that break things are evil! I don't write code for corporates, so my license is purely fictive. I cannot enforce my license but I can prevent corporates from taking my code and wrapping it into a product and selling it on for a profit. While not passing on a cent to me. If a corporate wishes to use my code, then they are welcome to pay me a license fee or a one-off payment for a non-distributable license.
  - Towaway69 1y ago
    My inspiration was came from Douglas Crockford and the JSONlint[1] license. Why not have a message? I mean if big-tech won't use my software because they legally think they might do evil with it, so be it. Do I really want big-tech to wrap my software into a product and sell it for profit while not giving me a cent because what I did was share my code without strings attached? I don't know. I would like to make this place just that little bit better and if if that's a license that makes folks think about what is evil, heck why not! [1] https://gist.github.com/kemitchell/fdc179d60dc88f0c9b76e5d38fe47076 https://gist.github.com/kemitchell/fdc179d60dc88f0c9b76e5d38...
deleted 1y ago
[deleted]
pydry 1y ago
It's a cool license in that it triggered Google enough to bar the use of jslint and led IBM's lawyers to seek out a special exemption to allow them and their customers to use JSON for evil. No company's PR will ever tell you what the soul of a company is like but their lawyers will indirectly tell you everything.
- adastra22 1y ago
  That’s a ridiculous take. The fact is “evil” is so ill defined that it would have been an unlimited liability. That’s a reflection of the world, not IBM.
  - pydry 1y ago
    Plenty (in not most) legal cases hinge upon non specifically defined terms hence lot of lawyering is not "this is definitely legal/illegal" but "what's the risk of this going wrong if we face a jury?" Companies take tens of thousands of legal risks every day and they single out particular risks over others to try and indemnify themselves because they think that risk is serious. Theyre not going to admit that theyre worried that the company is evil enough to qualify under a reasonable person's interpretation but thats what theyre thinking. It's the same with your employment contract. The level of nasty bullshit they put in there ("lawyers made us!") is probably the most accurate meter of how horrible (or not) the company will be towards you as an employer.

concerndc1tizen 1y ago

I've been interested in this for years. Are there other similar projects that you're familiar with? Perhaps targeting other languages? What are the major problems with this approach to programming? Are large programs hard to work with?

Towaway69 1y ago
I know of Py-Red[1] which is an attempt to do the same but with Python. Using Node-RED as a frontend and something else as a backend, no I haven't heard of anything similar. Main problems with visual flow based programming is the tooling: there aren't any good tools for doing visual comparison nor visual version control. Things like GitHub aren't able to render flow code as seen in the editor. For Node-RED that means that the Json that defines a flow can be compared but that soon becomes meaningless because semantic changes get mixed with visual changes, i.e., that the x,y coordinates of a node have changed - is meaningless for the logic of the code. This means that code sharing and collaborative development is difficult purely visually but only because tooling is missing. Compared to textual programming before things like SourceForge or Git and we had the same issues as we do with visual coding now. Node-RED has a bunch of features to make large programs maintainable. Things like link nodes that allow jumping between flows also allow code reuse. There also subflows which encapsulate repeated code and can be referenced from anywhere. That's also why I chose Node-RED for this because it seems to be the most mature and best maintained low-code visual flow-based programming tool. There are others such as n8n (which seems to be the other biggie) but they lack the generalness of Node-RED. I can create web sites with Node-RED while and the same time controlling my LEDs via Raspberry installation. Node-RED is extremely flexible. [1]: https://github.com/mdkrieg/py-red https://github.com/mdkrieg/py-red
- regularfry 1y ago
  Speaking of controlling LEDs, there's also https://www.atomvm.net/doc/main/index.html https://www.atomvm.net/doc/main/index.html. From a very quick scan it looks like it would need some fiddling (especially around the filesystem) to get it to work, but being able to scale down to smaller devices than a full pi would be very, very interesting.

smartmic 1y ago

> No Artificial Intelligence was harmed in the creation of this codebase. This codebase is old skool search engine (ddg), stackoverflow, blog posts and RTFM technology. Good. To me, that also means that no human intelligence was degraded as a result. It sounds like a quality badge.

deleted 1y ago
[deleted]

thinkyfish 1y ago

I was just thinking of making something like this last night but using FLENG instead of Erlang ;) I'm really glad more people are taking inspiration from Node-RED, I will definitely be playing with this!

Towaway69 1y ago
FLENG[1] - Wow, "low level concurrent logic programming language descended from Prolog" - sounds like an easy swap out for Erlang since they have the same heritage. ;) [1] http://www.call-with-current-continuation.org/fleng/fleng.html http://www.call-with-current-continuation.org/fleng/fleng.ht...

arresin 1y ago

Can someone recommend an Erlang course / book?

Towaway69 1y ago
I keep running into learn you some erlang[1] which seems to be a good online resource. The author of which also wrote the "Erlang in Anger"[2] book. Then there is the BEAM book[3] which provides an deep dive into Erlang and the BEAM virtual machine. I'm not the best person to ask since I'm only starting out on Erlang! I much prefer to code visually :) [1] https://learnyousomeerlang.com https://learnyousomeerlang.com [2] https://www.heroku.com/blog/erlang-in-anger/ https://www.heroku.com/blog/erlang-in-anger/ [3] https://blog.stenmans.org/theBeamBook/ https://blog.stenmans.org/theBeamBook/
- arresin 1y ago
  Thanks for the suggestions! I looked at "learn you some erlang" and I like the style and humor of it.
randito 1y ago
You may want to consider Elixir instead. It has an easier syntax and is (don't quote me on this) equivalent in function and purpose with Erlang. Plus you get lots of other goodies like LiveView (notebooks) and a good web stack (Phoenix).
- Towaway69 1y ago
  I prefer Erlang because it makes the uniqueness of its paradigm clear. Tail recursion, function matching and, bang, everything is a message! Elixir makes everything seem like Ruby code and many of those Erlang concepts are hidden away, which is also ok but also takes much away. My aim with ErlangRED is that both are supported, there is a repository for Elixir code[1] that is included in ErlangRED - so I don't take sides, rather I would like to take advantage of both paradigms. [1] https://github.com/gorenje/erlang-red-elixir-helpers https://github.com/gorenje/erlang-red-elixir-helpers

paddy_m 1y ago

Nice work! A couple of comments 1. You don't explain what flow based programming is or when it should be used. It looks like it represents some type of data pipeline, but I'm not sure. 2. Who is this repo for? PL enthusiasts, tool builders of type $X? 3. What are operators? 4. I understand that this is supposed to execute on an Erlang cluster/VM... Could you build a JS implementation of the functions so that you could build interactive demos in a browser? The lack of multithreading in JS doesn't rally matter, you just want to be able to send 10 line examples from one function to another. 5. A fun challenge for documenting projects like this, explain it as you would to a non-technical family member. It forces you to define the other things in erlang in a very direct and simple way. 6. Make a video explaining the project.

paddy_m 1y ago
Ahh, you didn't create Node-RED editor. That's an external project. I didn't understand this at first, and probably explains a lot of my confusion https://nodered.org/ https://nodered.org/
- Towaway69 1y ago
  No, that much talent I don't have :) Node-RED has a create frontend that just works and the APIs to the server are clearly defined. So it's relatively easy to swap out the backend - that's what I did. Erlang has many features that make flow based programming (as implemented by Node-RED) simpler - e.g., message passing builtin and cheap processes and cheap concurrency.
adastra22 1y ago
Why not just use Node-Red in that case?
- Towaway69 1y ago
  NodeJS isn't truly concurrent, it is event-based single threaded - fast but not concurrent. I use Node-RED every day! This is an exercise in applying the concept of flow based programming to another programming language. For me, flow based programming is like object oriented programming or functional programming or procedural ... etc. So the idea is to take the paradigm and apply to Erlang. Clarification: I've been using everyday Node-RED for last three years and have built many things with it - including my mind map!

nisten 1y ago

the hivemind thinks alike

Arch-TK 1y ago

I saw Node-RED recently, and now this. While I think these are cool (and I would rather Erlang than NodeJS), is there some code based system for these kinds of things? I know I can just write code, but I mean a DSL or some library specifically oriented around solving exactly these kinds of event driven sequencing/action problems.

Towaway69 1y ago
Not to my knowledge, the closest thing to a DSL for Node-RED is its flows.json format for storing flow configurations. It's a very simple array-of-object format with the graph/flow definitions defined by node ids and wires. It is fairly simple to create a minimalist interpretor for that format. Also NodeRED is a very stable piece of software that has been around for over ten years. Yet I can take a flows.json file from today and have it run on the first releases of Node-RED, i.e., the format is backward compatible yet easily extendable. > event driven sequencing/action problems. Flow based programming[1] isn't strictly event driven, it's implemented that way in NodeJS because NodeJS does not have the concept of messages, only events. Erlang has message passing between independent processes and therefore, IMHO, better suited to FBP. [1] https://jpaulm.github.io/fbp/index.html https://jpaulm.github.io/fbp/index.html
- Arch-TK 1y ago
  Firstly, thanks for the link and information on more of the roots of this kind of programming, I couldn't find this when I looked. Regarding NodeRED. I don't dispute that it's stable, but my experience actually running any nodejs based software is that it's all heavily designed for running out of a container and a massive PITA to maintain otherwise. I already have enough hassle running zigbee2mqtt on my FreeBSD home server which wouldn't work in a linux jail because I couldn't figure out how to expose the zigbee dongle to it. Secondly, the reason I wanted to run one of these was not because I wanted to use it to automate my house, but instead because I wanted to understand how it works so I could replicate it in my own weird way.
NortySpock 1y ago
I suggest one of Benthos' descendants, either Bento or RedPanda Connect. Configuration driven, with a transformation DSL, and good documentation (I prefer the Bento documentation over RedPanda Connect's documentation). And Benthos explicitly refused to ack a consumed message upstream unless it had been acknowledged downstream, so it wouldn't drop messages on the floor unless you explicitly told it to.

pjmlp 1y ago

I am a big fan of visual programming, and also think visual and digital circuit design tools overlap quite nicely with functional programming languages. The mess most people complain about in visual languages, also happens on text ones, the biggest difference is that visual ones make it quite clear the code lacks modularity. Get to package stuff into reusable ICs more often. All the best for the project.

Towaway69 1y ago
> visual ones make it quite clear the code lacks modularity. NodeRED has the concept of subflows to encapsulate reusable code. There are a number of features that NodeRED has that make it more usable for long term projects but it's a steep learning curve. I've been using Node-RED everyday for the last three years to find the edge, something that cannot be done but there has always been a visual solution. I'm starting to think Node-RED might be turing complete ;) But I have to admit, I did build a lot of tooling around NodeRED to make that happen - visual flow comparison[1] was something that was missing, introspection of message flow[2] and visual code management[3] - all these things I need to maintain an overview of what I am doing. [1] https://flows.nodered.org/node/@gregoriusrippenstein/node-red-contrib-flowcompare https://flows.nodered.org/node/@gregoriusrippenstein/node-re... [2] https://flows.nodered.org/node/@gregoriusrippenstein/node-red-contrib-introspection https://flows.nodered.org/node/@gregoriusrippenstein/node-re... [3] https://flowhub.org/ https://flowhub.org/ > All the best for the project. Thank you!
tlodboy 1y ago
Visual programming helps automate flows in circuit design and I feel adoption would be higher as well since most circuit designers for some reason prefer GUI
deleted 1y ago
[deleted]

mystraline 1y ago

I was an early adopter of NodeRed. Earlier, it worked exceptionally well. Now? Not so much. Well, that's not exactly true. Base NodeRed works as well as before. But the libraries of modules to interface with all sorts of websites, APIs, hardware, and other stuff is rotten to the core. Most plugins/js modules either just don't work, or 'successfully fail'. The easier fail case is where the module can't be installed due to ancient (6mo or older JS, sigh) modules. I abandoned NR because its basically a hyper-speed bitrot due to terrible library module versioning. And I didn't want to reinvent the wheel on every system I wanted to touch.

wslh 1y ago
I don't think this is easily to solve, in general. Similar orchestrators (e.g. n8n) have the same issues because there are a lot of components dependencies that change with time and there is no real cohesion between the core and all kind of plugins. Probably a future "contracts/manifests" linking orchestrators with components could help.
Towaway69 1y ago
I wrote an article about this[1] and you are definitely right: a lot of packages are rotting away a bit however, because NodeRED does a lot to stay backward compatible, older packages still work. The oldest ones I'm actively using are five or more years old, i.e., haven't been touched nor updated - still work. What I tried to say in the article is the same as you say: base NodeRED, the core works really well and has great features - no questions. And even if packages die, the core will still remain usable and that makes a difference. Its a bit like saying Linux is a pile of broken bits because the ls command has been updated in ten years: Linux will always work and those commands that are based on the core will continue to work becausae the Linux kernel will largely remain backward compatible. Packages fail when they have external dependencies that change but code that is solely based on the core will continue to work. [1] https://blog.openmindmap.org/blog/crunchy-numbers https://blog.openmindmap.org/blog/crunchy-numbers
01100011 1y ago
Not surprising. Whenever you have a project like NR or HA you have a ton of barely working glue code written by people who just want to get it working and move on without any sort of commitment to maintenance. It allows these projects to rapidly expand their support but then that support quickly rots unless the core team assumes responsibility for maintenance. I really want to mess with home automation again but this sort of low software quality and resulting instability and maintenance hassles makes it not worth the effort.

oersted 1y ago

> Node-RED is a amazing[*] tool for creating flows that describe concurrent processing, it is just a shame the NodeJS is single threaded. So why not use something that is multi-process from the ground up? That's great! But now that we are doing this, it kind of makes me wish that it was not multi-processing and Erlang, but a more mainstream language with a better library ecosystem and focused on multi-threading instead, Rust comes to mind but there could be a better one. Is there a visual flow programming language based on Rust that is relatively mature?

bheadmaster 1y ago
> it kind of makes me wish that it was not multi-processing and Erlang, but a more mainstream language with a better library ecosystem and focused on multi-threading instead, Rust comes to mind but there could be a better one Go sounds like a good candidate.
- Intermernet 1y ago
  There are 3 or 4 Go FBP packages. It seems to be completely capable, but FBP itself doesn't seem to have reached enough critical mass for them to be popular. https://github.com/flowbase/flowbase https://github.com/flowbase/flowbase is one, and it has links to to some others.
Towaway69 1y ago
> wish that it was not multi-processing and Erlang Hehe - honestly I chose Erlang because I love the syntax and secondly because it is so niche. Flow based programming is niche, visual FBP is nicher, Node-RED is even nicher and Erlang is niche - it's niche all the way down ;) Have a look at Elixir which is Erlang with Ruby syntax and also has a large community. Also the intention is to be using the flow editor and not the lanuage underneath, so the end user won't even know that it's Erlang or NodeJS - I aim t stay compatible with Node-RED so that flows are actually interchangeable. So to an certain extent it doesn't matter which language is used. I did chose Erlang also because it is well suited to flow based programming being message based and flow based programming is all about passing immutable objects amongst independent processes .... that's Erlang in a nutshell.

fnord77 1y ago

for a visual environment, I was expecting more images to show me what this tool is

Towaway69 1y ago
There is a live demo[1] if you want to try it out! For me, the focus is more on the Erlang part than the visual part since that's basically NodeRED. [1] https://ered.fly.dev/node-red https://ered.fly.dev/node-red
- fnord77 1y ago
  thank you, very neat

lxe 1y ago

[flagged]

nathanappere 1y ago
It does if you have blocking calls or you cannot yield to the event loop in time consuming code.
ummonk 1y ago
Are you asserting that single threaded execution of concurrent code is just as performant as multi-process execution?
- Towaway69 1y ago
  Concurrency isn’t possible if there is only a single thread is my interpretation.
  - hotpocket777 1y ago
    That is not true at all. Concurrency is a way of writing programs and is an orthogonal concern to how the program gets executed.
- lxe 1y ago
  It certainly can be.
deleted 1y ago
[deleted]

aantix 1y ago

I would move your sample images to the top of the README. Don't bury the lede.

gitroom 1y ago

[dead]

curtisszmania 1y ago

[dead]

nilslindemann 1y ago

The thing which makes me be annoyed with Yours and the Node-RED site, is that you people do not define terms, before using them. Like "flow". What is this? A program? Please maintain a glossary.

nilslindemann 1y ago
Okay, It seems you do, https://flowfuse.com/node-red/terminology/ https://flowfuse.com/node-red/terminology/, https://nodered.org/docs/user-guide/concepts https://nodered.org/docs/user-guide/concepts, so please link to it.