7 ms·

I think what would be more use-full would be a new "shell" which is base on a modern context, incorporating lessens learned from years of cmd-line UX design (ok

I think what would be more use-full would be a new "shell" which is base on a modern context, incorporating lessens learned from years of cmd-line UX design (ok let's be honest many programs didn't learn anything wrt. UX). But adding jet another POSIX shell which maybe does some parts better but in the end has still not a grate UX because it's a POSIX shell seems kinda pointless IMHO.

yoloClin 5y ago
This. I want a shell that will contextually spit out plaintext in interactive mode, then a JSON object when scripted or piped. Pretty sure this is what Powershell does, but the UI just feels so damn unnatural.
- zozbot234 5y ago
  Powershell does not use JSON. It's based on .NET objects.
- kreetx 5y ago
  Perhaps IDEs and shells could converge?
- hnlmorg 5y ago
  Murex does this. eg Take a plain text table, convert it into an SQL table and run an SQL query: » ps aux | select USER, count(*) GROUP BY USER USER count(*) _installcoordinationd 1 _locationd 4 _mdnsresponder 1 _netbios 1 _networkd 1 _nsurlsessiond 2 _reportmemoryexception 1 _softwareupdate 3 _spotlight 5 _timed 1 _usbmuxd 1 _windowserver 2 lmorg 349 root 134 The builtins usually print human readable output when STDOUT is a TTY, or JSON (or JSONlines) when the TTY is a pipe. » fid-list: FID Parent Scope State Run Mode BG Out Pipe Err Pipe Command Parameters 590 0 0 Executing Normal no out err fid-list (subject to change) » fid-list: | cat ["FID","Parent","Scope","State","RunMode","BG","OutPipe","ErrPipe","Command","Parameters"] [615,0,0,"Executing","Normal",false,"out","err",{},"(subject to change) "] [616,0,0,"Executing","Normal",false,"out","err",{},"cat"] and you can reformat to other data types, eg » fid-list: | format csv FID,Parent,Scope,State,RunMode,BG,OutPipe,ErrPipe,Command,Parameters 703,0,0,Executing,Normal,false,out,err,map[],(subject to change) 704,0,0,Executing,Normal,false,out,err,map[],csv and query data within those data structures using tools that are aware of that structural format. Eg Github's API returns a JSON object and we can filter through it to return just the issue ID's and titles: » open https://api.github.com/repos/lmorg/murex/issues | foreach issue { printf "%2s: %s\n" $issue[number] $issue[title] } 348: Potential regression bug in `fg` 347: Version 2.2 Release 342: Install on Fedora 34 fails (issue with `go get` + `bzr`) 340: `append` and `prepend` should `ReadArrayWithType` 318: Establish a testing framework that can work against the compiled executable, sending keystrokes to it 316: struct elements should alter data type to a primitive 311: No autocompletions for `openagent` currently exist 310: Supprt for code blocks in not: `! { code }` 308: `tabulate` leaks a zero length string entry when used against `rsync --help` Source: https://github.com/lmorg/murex https://github.com/lmorg/murex Website: https://murex.rocks https://murex.rocks
  - AnIdiotOnTheNet 5y ago
    As much as this is an improvement in many ways, using JSON for this feels like it doubles down on part of the problem with the current standard. Every tool is rendering JSON only for the next tool in the pipeline to parse it back out.
    - hnlmorg 5y ago
      Yeah, I do understand where you're coming from and I've spent a lot of time considering how I'd re-architect murex to pass raw data across (like Powershell) rather than marshalling and unmarshalling data each side of each pipe. In the end I settled on the design I have because it retains compatibility with the old world while enabling the features of the new world but it also behaves in a predictable way so it's (hopefully) easy for people to reason about. Powershell (and other languages in REPL like Python, LISP, etc) still exists for those who want a something that's ostensibly a programming environment first and a command line second and I think trying to compete with the excellent work there wouldn't be sensible given how mature those solutions already are. But for a lot of people, the majority of their command line usage is just chaining existing commands together and parsing text files. Often they want something terser than $LANG as a lot of command lines are read-once write-many and thus are happy to sacrifice a little in language features for the sake of command line productivity. This is the approach murex takes. Albeit murex does also try to retain readability despite being succint (which is probably the biggest failing of POSIX shells in the modern era). What I've built is definitely not going to be everyone's preferred solution, that's for sure. But it works for me and its open source so hopefully others find it as useful as I do :)
      - TeMPOraL 5y ago
        A good binary format would be best, but JSON is already a step up - at least it's obvious where each value starts and ends. That said, maybe it wouldn't be too hard to offer a binary-serialized JSON format as well (I think BSON is the currently widespread standard)? On a related note, I wonder if and how a pipe could handle "format negotiation" between processes? I.e. is there a way for a CLI app to indicate it can consume and produce structured binary data? Then the piping layer could let compatible apps talk through an efficient protocol, and for anyone else, it would automatically drop to equivalent JSON (and then maybe binarize it back up, if the next thing in the pipeline can handle it).
- tmerse 5y ago
  I'd like most programs to implement a JSON interface, given right flags. Or maybe the existence of some env var like $DEFAULT_OUTPUT=JSON I like this approach: https://github.com/kellyjonbrazil/jc https://github.com/kellyjonbrazil/jc > CLI tool and python library that converts the output of popular command-line tools and file-types to JSON or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts
hnlmorg 5y ago
This is the approach murex tries to take. It breaks POSIX compliance without remorse but retains compatibility wherever practical. This means it can have some of the flexibilities of Powershell (typed pipelines), the flexibilities of IDEs (events, better auto-completions, etc) but still works with all of your everyday command line tools. My ultra long term aim is to integrate the shell into a media rich terminal emulator so using the command line will have the speed and precision of a TUI, the power of an IDE but the rich content of a GUI. I'm a loooong way off achieving that but the shell is already usable and has been my primary shell for ~4 years now. And I welcome any and all feedback on the current and any future builds. At some point I'll publish a paper with my ideas. https://github.com/lmorg/murex https://github.com/lmorg/murex
- cyberpunk 5y ago
  Just a FYI, murex is (also) a very widely used front of house trading platform, which costs an absolute fortune and has an army of lawyers, you may want to reconsider the name..
  - hnlmorg 5y ago
    Yeah, I get this a lot. I wasn't aware of the trading platform when I created the shell and was quite late into the development before I was made aware. The shell and the trading platform exist in different domains so as far as I'm concerned there isn't any trademark violation but if the trading platform (and their lawyers) disagree then I'm open to dialogue with them. I think HN is uniquely placed where it has a lot of trading topics as well as IT so it wasn't until my shell started trending on here that anyone had even noticed the naming conflict. But my shell had been around for a good few years before it started trending on here.
- pxc 5y ago
  I'm surprised I hadn't heard of Murex yet! It looks pretty similar to Elvish but the type system is more foregrounded (and more complete?). Could you help me differentiate it from Elvish in terms of features and design?
worble 5y ago
I quite like elvish (https://elv.sh https://elv.sh) for a non-POSIX compatible shell. It's by no means perfect, and it's not 1.0 so breaking changes are pretty frequent, but I find the language structure and syntax make it worth it. The docs on the site are a little haphazard too, but once you get a feel for it, you probably won't need to refer to them too often.

yoloClin 5y ago

This. I want a shell that will contextually spit out plaintext in interactive mode, then a JSON object when scripted or piped. Pretty sure this is what Powershell does, but the UI just feels so damn unnatural.

zozbot234 5y ago
Powershell does not use JSON. It's based on .NET objects.
kreetx 5y ago
Perhaps IDEs and shells could converge?
hnlmorg 5y ago
Murex does this. eg Take a plain text table, convert it into an SQL table and run an SQL query: » ps aux | select USER, count(*) GROUP BY USER USER count(*) _installcoordinationd 1 _locationd 4 _mdnsresponder 1 _netbios 1 _networkd 1 _nsurlsessiond 2 _reportmemoryexception 1 _softwareupdate 3 _spotlight 5 _timed 1 _usbmuxd 1 _windowserver 2 lmorg 349 root 134 The builtins usually print human readable output when STDOUT is a TTY, or JSON (or JSONlines) when the TTY is a pipe. » fid-list: FID Parent Scope State Run Mode BG Out Pipe Err Pipe Command Parameters 590 0 0 Executing Normal no out err fid-list (subject to change) » fid-list: | cat ["FID","Parent","Scope","State","RunMode","BG","OutPipe","ErrPipe","Command","Parameters"] [615,0,0,"Executing","Normal",false,"out","err",{},"(subject to change) "] [616,0,0,"Executing","Normal",false,"out","err",{},"cat"] and you can reformat to other data types, eg » fid-list: | format csv FID,Parent,Scope,State,RunMode,BG,OutPipe,ErrPipe,Command,Parameters 703,0,0,Executing,Normal,false,out,err,map[],(subject to change) 704,0,0,Executing,Normal,false,out,err,map[],csv and query data within those data structures using tools that are aware of that structural format. Eg Github's API returns a JSON object and we can filter through it to return just the issue ID's and titles: » open https://api.github.com/repos/lmorg/murex/issues | foreach issue { printf "%2s: %s\n" $issue[number] $issue[title] } 348: Potential regression bug in `fg` 347: Version 2.2 Release 342: Install on Fedora 34 fails (issue with `go get` + `bzr`) 340: `append` and `prepend` should `ReadArrayWithType` 318: Establish a testing framework that can work against the compiled executable, sending keystrokes to it 316: struct elements should alter data type to a primitive 311: No autocompletions for `openagent` currently exist 310: Supprt for code blocks in not: `! { code }` 308: `tabulate` leaks a zero length string entry when used against `rsync --help` Source: https://github.com/lmorg/murex https://github.com/lmorg/murex Website: https://murex.rocks https://murex.rocks
- AnIdiotOnTheNet 5y ago
  As much as this is an improvement in many ways, using JSON for this feels like it doubles down on part of the problem with the current standard. Every tool is rendering JSON only for the next tool in the pipeline to parse it back out.
  - hnlmorg 5y ago
    Yeah, I do understand where you're coming from and I've spent a lot of time considering how I'd re-architect murex to pass raw data across (like Powershell) rather than marshalling and unmarshalling data each side of each pipe. In the end I settled on the design I have because it retains compatibility with the old world while enabling the features of the new world but it also behaves in a predictable way so it's (hopefully) easy for people to reason about. Powershell (and other languages in REPL like Python, LISP, etc) still exists for those who want a something that's ostensibly a programming environment first and a command line second and I think trying to compete with the excellent work there wouldn't be sensible given how mature those solutions already are. But for a lot of people, the majority of their command line usage is just chaining existing commands together and parsing text files. Often they want something terser than $LANG as a lot of command lines are read-once write-many and thus are happy to sacrifice a little in language features for the sake of command line productivity. This is the approach murex takes. Albeit murex does also try to retain readability despite being succint (which is probably the biggest failing of POSIX shells in the modern era). What I've built is definitely not going to be everyone's preferred solution, that's for sure. But it works for me and its open source so hopefully others find it as useful as I do :)
    - TeMPOraL 5y ago
      A good binary format would be best, but JSON is already a step up - at least it's obvious where each value starts and ends. That said, maybe it wouldn't be too hard to offer a binary-serialized JSON format as well (I think BSON is the currently widespread standard)? On a related note, I wonder if and how a pipe could handle "format negotiation" between processes? I.e. is there a way for a CLI app to indicate it can consume and produce structured binary data? Then the piping layer could let compatible apps talk through an efficient protocol, and for anyone else, it would automatically drop to equivalent JSON (and then maybe binarize it back up, if the next thing in the pipeline can handle it).
tmerse 5y ago
I'd like most programs to implement a JSON interface, given right flags. Or maybe the existence of some env var like $DEFAULT_OUTPUT=JSON I like this approach: https://github.com/kellyjonbrazil/jc https://github.com/kellyjonbrazil/jc > CLI tool and python library that converts the output of popular command-line tools and file-types to JSON or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts

hnlmorg 5y ago

This is the approach murex tries to take. It breaks POSIX compliance without remorse but retains compatibility wherever practical. This means it can have some of the flexibilities of Powershell (typed pipelines), the flexibilities of IDEs (events, better auto-completions, etc) but still works with all of your everyday command line tools. My ultra long term aim is to integrate the shell into a media rich terminal emulator so using the command line will have the speed and precision of a TUI, the power of an IDE but the rich content of a GUI. I'm a loooong way off achieving that but the shell is already usable and has been my primary shell for ~4 years now. And I welcome any and all feedback on the current and any future builds. At some point I'll publish a paper with my ideas. https://github.com/lmorg/murex https://github.com/lmorg/murex

cyberpunk 5y ago
Just a FYI, murex is (also) a very widely used front of house trading platform, which costs an absolute fortune and has an army of lawyers, you may want to reconsider the name..
- hnlmorg 5y ago
  Yeah, I get this a lot. I wasn't aware of the trading platform when I created the shell and was quite late into the development before I was made aware. The shell and the trading platform exist in different domains so as far as I'm concerned there isn't any trademark violation but if the trading platform (and their lawyers) disagree then I'm open to dialogue with them. I think HN is uniquely placed where it has a lot of trading topics as well as IT so it wasn't until my shell started trending on here that anyone had even noticed the naming conflict. But my shell had been around for a good few years before it started trending on here.
pxc 5y ago
I'm surprised I hadn't heard of Murex yet! It looks pretty similar to Elvish but the type system is more foregrounded (and more complete?). Could you help me differentiate it from Elvish in terms of features and design?

worble 5y ago

I quite like elvish (https://elv.sh https://elv.sh) for a non-POSIX compatible shell. It's by no means perfect, and it's not 1.0 so breaking changes are pretty frequent, but I find the language structure and syntax make it worth it. The docs on the site are a little haphazard too, but once you get a feel for it, you probably won't need to refer to them too often.