murex author here. Hopefully I can answer some questions:
> I do scan alternative shells from time to time. I could have missed but I didn't see multiple dispatch in any of them. Which ones have it? Just to clarify: In which shell you can define several methods with the same name and when called, the method to invoke is selected based on the types of the arguments?
I've not heard of the term "multiple dispatch" but reading the thread it sounds like you're describing function overloading. Powershell does support this with classes[0].
murex does it's overloading at the API level[1]. The reason behind that decision is to keep the methods simple (eg a pipeline might contain JSON or CSV data) because you as a shell user don't want to run into situations where you've written a function that supports one data type but not another, you just want it to work first time. So murex automatically abstracts that part away for you. In addition to the point the other poster mentioned about consistent `jq`-like methods that are data type agnostic, murex allows for easy iteration through objects agnostic of their data type (eg `open somedata -> foreach { do stuff }` -- where you don't need to think about data types, file formats, etc, murex does the heavy lifting for you in a consistent and predictable way).
There are some specific builtins that support "overloading" of sorts but instead of overloading the function they have handlers that call user defined functions[2][3]. This removes the mystery behind function overloading because you can easily trace which handlers exist for which data types.
It's also worth adding that function overloading is supported in the same way that its also supported in Bash too where you might have a function but if there is a alias of the same name that will take priority. There are also private[4] functions (which are namespaced) so you have additional controls to avoid accidental overloading when writing modules too. And the interactive shell expands out any command (eg displays a hint text stating if a command is a function, alias, external command, etc) so there's a reduced risk of being unaware if `ls` is an alias, function, `/bin/ls`, or even a symlink to something else.
> I did not even see how to define named parameters - https://murex.rocks/docs/commands/function.html https://murex.rocks/docs/commands/function.html
Named parameters are optional because neither Windows nor POSIX have any understanding of named arguments in their processes. I did consider abstracting that into murex's functions regardless like Python et al would but I couldn't design a way that wasn't jarring nor cumbersome to write in a hurry nor worked transparently with Windows and POSIX ARGS[]. So I've come up with an optional builtin called `args`[5] which allows you to define flags, which are effectively the same thing as named parameters except they're supported natively by Windows and POSIX ARGS[], so you can write a murex script as a native shell script without leaving the user to write another abstraction layer themselves interpreting an indexed argument into an named arguments. Thus giving you the flexibility to write really quick Bash-like functions or more verbose scripting style functions depending on your need.
> Examples of things in NGS that I have not seen anywhere else: syntax for run-command-and-parse-output, proper handling of exit codes.
Both of these are baked into murex as well. The run-command-and-parse-output is the API stuff mentioned above. It's where the overloading happens.
As for error handling: any command is considered a failure if there is either a non-zero exit code, STDERR contains a failed message (eg "false", "failed", etc) or STDERR is > STDOUT[6]. This has covered every use case I've come across.
Additionally STDERR is highlighted red by default and when a process fails you're given a mini-stack trace showing where in the script the error happened. You have try/catch blocks, saner `if` syntax etc that all use the same API for detecting if a process has failed too. Keeping everything consistent.
The other thing murex has to help catch bugs and error is a testing framework baked into the shell language. The docs for test[7] need expanding but in essence:
- you can write proper unit tests
- you can intercept STDOUT (even if it's mid pipeline) and test that output. This works around instances where (1) you can't have full on unit tests due to side effects in a function that can't be easily mocked (2) you want to add your own debugging routines (rather than just printing values to STDOUT or running commands manually to see their output -- like one would normally have to do with shell scripting)
- you can add state watches. Except in murex the watches are Turing complete so you're not just adding noise to your terminal output but rather putting meaningful debug messages and scripts.
And all of the debugging stuff can be written straight into your normal shell routines and cause no additional execution overhead unless you've purposely enabled test mode.
So its fair to say I've spent a significant amount of time designing smarter ways of handling failures than your average shell scripting language. As I'm sure you have too.
> Regarding exit codes. Typical approach to exit codes varies. Python - "I don't care, the programmer should handle it". Some Python libraries and other places - "Non-zero is an error". That's simplistic and does not reflect the reality in which some utilities return 1 for "false". bash (and probably other shells too) is unable to handle in a straightforward manner situation where external command can return exit codes for "true", "false" and "error". It just doesn't fit in the "if" with two branches. NGS does handle it with "if" with two branches + possible exception thrown for "error" exit code.
If you're describing that as a "two branch" approach then technically murex could be argued as having "three branches" because it checks exit code, STDERR contents, and payload size too. Personally I just describe it as "error handling" because in my view this is how peoples expectations are rather than the reality of handling forked executables.
I guess where NGS and murex really differ is NGS likes to expose its smart features whereas in murex they're abstracted away a little (they can still be altered, customised, etc) to keep the daily mundane shell usage as KISS (keep it simple stupid) as possible. eg you can overload function calls if you really wanted but that can often cause unforeseen complications or other unexpected annoyances right when you least want it to. So murex keeps that stuff around for when you need it but finds ways to avoid people needing to rely on it and furthermore unwraps the covers of what a routine does in the interactive terminal. It's all about offering abstractions but removing surprises.
> NGS knows that some external programs have exit code 1 which does not signify an error.
Having different behaviours for different executables hard coded into the shell is one behaviour I purposely avoided. I do completely understand the incentive behind wanting to do this and wouldn't criticise others for doing that but given external programs can change without the shell being aware, they can be overloaded with aliases, functions, etc, and they might even just differ between Linux, BSD, Windows, etc -- well it just seemed like hard coding executable behaviour causes more potential issues than it solves. You also then run into problems where users expect the shell to understand all external executables but there are some you haven't anticipated thus breaking expectation / assumptions. Ultimately it creates a kind of special magic that is unpredictable and outside the control of the shell itself. So instead I've relied on having foundational logic that is consistent. It's the one side of shell programming where I've placed the responsibility onto the developer to get right rather than "automagically" doing what I think they are expecting. This of course does mean I have to focus even harder on ensuring all other aspects of the shell are predictable and low maintenance so that the developer can cover any specific edge cases with ease. Which goes a long way to explaining why I've chosen the path I've chosen (ie reducing the amount of syntax sugar, overloading, etc needed for daily use that one might rely on in a more traditional programming language).
Links:
[0] https://stackoverflow.com/a/57294942 https://stackoverflow.com/a/57294942
[1] https://murex.rocks/docs/GUIDE.apis.html https://murex.rocks/docs/GUIDE.apis.html
[2] https://murex.rocks/docs/commands/openagent.html https://murex.rocks/docs/commands/openagent.html && https://murex.rocks/docs/commands/open.html https://murex.rocks/docs/commands/open.html
[3] https://murex.rocks/docs/commands/event.html https://murex.rocks/docs/commands/event.html (I need to write a lot more documentation around event driving programming in murex)
[4] https://murex.rocks/docs/commands/private.html https://murex.rocks/docs/commands/private.html
[5] https://murex.rocks/docs/commands/args.html https://murex.rocks/docs/commands/args.html
[6] https://github.com/lmorg/murex/blob/19d2aeb27e814a4c6fec77d229cc32d68404c061/lang/types/types.go#L38 https://github.com/lmorg/murex/blob/19d2aeb27e814a4c6fec77d2...
[7] https://murex.rocks/docs/commands/test.html https://murex.rocks/docs/commands/test.html
I should have added (outside of edit time now) that I do like what you've done with NGS. Nothing I've posted above is intended to suggest I disagree with your approach. We're covering mostly the same problems but we've just looked at it from different angles. Which is good -- the more options out there the better I say.
Wow, that's a bunch of reading to do :)
> I've not heard of the term "multiple dispatch"
https://en.wikipedia.org/wiki/Multiple_dispatch https://en.wikipedia.org/wiki/Multiple_dispatch (includes example from NGS too)
In NGS, multiple dispatch is based on CLOS model described at https://en.wikipedia.org/wiki/Common_Lisp_Object_System https://en.wikipedia.org/wiki/Common_Lisp_Object_System , with my own simplifications.
> Powershell does support this with classes
Somewhat similar. Since methods do not live in classes in NGS, I would argue the mechanism in NGS is simpler (more elegant?). You can just define your_method(c1:Class1, c2:Class2).
> murex does it's overloading at the API level[1].
mmm. I see the support but looking at documentation at https://murex.rocks/docs/apis/Unmarshal.html https://murex.rocks/docs/apis/Unmarshal.html , I see it's not exposed into the Murex language, it's in Go. Is this correct?
> where you don't need to think about data types, file formats, etc, murex does the heavy lifting for you in a consistent and predictable way
That is good and what I would expect.
( sorry, running out of time, to be continued :) )
> https://en.wikipedia.org/wiki/Multiple_dispatch https://en.wikipedia.org/wiki/Multiple_dispatch (includes example from NGS too)
Learn something new every day. Seen (and used) this methodology before but wasn't aware it was called that. I'd always heard of it as "overloading" which is conceptually similar but not identically the same.
> mmm. I see the support but looking at documentation at https://murex.rocks/docs/apis/Unmarshal.html https://murex.rocks/docs/apis/Unmarshal.html , I see it's not exposed into the Murex language, it's in Go. Is this correct?
It's APIs written in Go (for performance and convenience -- you wouldn't want to write a YAML marshaller in a shell scripting language). So the methods are exposed as builtins. However you can add method written in murex if you wanted.
The idea being get 99.9% right by default but leave some flexibility for the user to customise if they want.
Murex builtins are also written in a way that they're all optional includes into the core project. Thus you can easily write your own builtins, marshallers, etc in Go if you want performance and then call them from your shell script (or you could write them in Python, Java, Perl, etc and run them as an external executable if you wanted too. But if you do that you lose access to murex's typed pipelines. Which is where the really interesting stuff happens (I haven't yet figured out a non-shitty way to send typed data over POSIX pipes to external executables).
> sorry, running out of time, to be continued :)
Any time :) I'm finding this conversation really interesting and educational