4 ms·

> However, Python and Ruby aren't good shell replacements in general. Shell is a domain-specific language for dealing with concurrent processes and the file sys

> However, Python and Ruby aren't good shell replacements in general. Shell is a domain-specific language for dealing with concurrent processes and the file system. But Python and Ruby have too much abstraction over these concepts, sometimes in the name of portability (e.g. to Windows). They hide what's really going on. Excellently put. POSIX shell languages have fantastic capabilities you just can't get in most other languages. I would love to see a more safe, more sane shell language gain enough popularity to change the narrative that "shell scripts are dangerous and impossible to maintain." The contrasts to Python and Ruby made me think of xonsh[1], a Python-based shell that can dynamically switch between Bash-like syntax and standard Python. It's not quite ready to become my daily driver, but I'm still excited about it. [1]: https://xon.sh https://xon.sh

chubot 9y ago
Thanks, and you have put it very well too. As I've learned from many of these threads [1], there most certainly is a narrative that shell scripts are dangerous and impossible to maintain. People are really angry about it! And of course I agree with that! That's the whole reason for Oil. [1] http://www.oilshell.org/blog/2018/01/31.html http://www.oilshell.org/blog/2018/01/31.html
willghatch 9y ago
Shell is my favorite domain-specific language. But many (including myself) would argue that domain-specific languages are generally better embedded. Many projects aiming to mixing shell with general purpose languages find a nice embedded DSL for subprocess/pipeline management. Some others find a convenient way to run shell commands or pipelines by mixing grammars and trying to disambiguate them. [Shameless self-promotion] I've been working on a project that aims to not only have a nice DSL for running process pipelines, but focuses on making a syntax for using all functionality of the host language as a command language, called Rash[1]. It is hosted in Racket, and can be embedded in Racket at the module or expression level, and also normal Racket expressions can be embedded in Rash (they can alternate arbitrarily deep). It supports process pipelines, Racket function/object pipelines, and mixes of the two. It's still alpha and has a TODO list a mile long, but I've been using it as my daily driver interactive shell for months and have loved it so far. [1]: https://github.com/willghatch/racket-rash https://github.com/willghatch/racket-rash
- chubot 9y ago
  FWIW I have your project on my wiki page :) https://github.com/oilshell/oil/wiki/ExternalResources https://github.com/oilshell/oil/wiki/ExternalResources I guess what you mean by embedded is that it should be an embedded DSL in a full-fledged programming language? I don't quite agree, since there are at least 20 projects like that on the wiki page, none of which is popular. Probably the most popular one is eshell, in Emacs Lisp? But if there's something I don't know about I'd be interested in hearing it. This idea goes back at least 20 years, e.g. to scsh. And it hasn't taken off. But certainly I don't begrudge anyone if their favorite environment is Racket and they want to stay in Racket. That's a perfectly reasonable thing. It's just not what I would expect anyone besides racket users to use. One reason I'm interested in shell because it's the lowest common denominator between different groups of programmers. C/C++ programmers, use it heavily, Python, Ruby, JS programmers, Go, etc. Everybody uses it.
  - willghatch 9y ago
    Yes, actually I've looked over all the shells on that wiki page. I think most of them haven't taken off because they either their host language has been poor or unpopular, their design or implementation wasn't great, or they haven't really solved the right problem. I think the idea of embedding a shell into a general-purpose language still has a lot of merit. Most of those projects trying to embed a shell DSL into a general-purpose language, scsh included, are basically for programming and not interactive use. Shells that are only interactive or only for programming end up fulfilling less than half of the purpose of shell, in my view, because the interaction and scripting feed back into each other in a virtuous cycle. The only ones on that list aside from Rash that try to be embedded in a general-purpose language while being good at both interactions and programming are Xonsh and eshell. Xonsh is pretty new (I wasn't aware of it until after I had made Rash), and eshell is in Emacs Lisp (which is not a very good programming language or platform to build on for anything except extending the emacs editor). Rash also tries to be better than the competition by adding object pipelines (much like Powershell, it makes it much more reasonable to make system administration commands in the host language, and have rich interaction and inspection of command results), user-defineable pipeline operators, and generally tighter integration with the host language while still having a light syntax for basic commands. I would like to be able to have my command language tightly integrated with my programming language, and be able to define my system administration commands (and interactive shell plugins and completion) as functions in the host language (while still being able to naturally use external programs). And I would like my shell scripts to be able to grow and potentially even turn into full-fledged programs, or modules of some larger program. I think there are a lot of benefits to the approach I'm using (which would be too long for a comment here). That said, I'm not holding my breath for it to catch on widely any more than I'm holding my breath for Racket to take off as a popular programming language (although I frankly wouldn't mind either one). I think a better Posix shell is certainly a noble effort, because whatever better shell does become popular, we certainly need one. And an automatic upgrade path for existing scripts sounds great. So I salute you and wish you good luck with it. Also, as someone looking at shells and their features to copy the best ones, your wiki is a great resource. So thanks.
rendaw 9y ago
I disagreed with this strongly. > concurrent processes Job control is disabled in shell scripts, and your only other option is juggling process ids. Combined with nearly nonexistent exception handling using anything more than a single process at a time is like pulling teeth. Things like ssh have super awkward workarounds using control pipes. I'm not sure I've ever seen a shell script use concurrent processes in the wild. Python's subprocess library is excellent and makes concurrent processes a breeze. > file system With some minor exceptions, I can't think of any FS ops I'd do in shell that isn't just a couple letters longer in Python and 10 times more flexible. The only reason to use shell is that it has a simple syntax for pipelines. I rewrite any script > 2 lines in Python and have no regrets.

chubot 9y ago

Thanks, and you have put it very well too. As I've learned from many of these threads [1], there most certainly is a narrative that shell scripts are dangerous and impossible to maintain. People are really angry about it! And of course I agree with that! That's the whole reason for Oil. [1] http://www.oilshell.org/blog/2018/01/31.html http://www.oilshell.org/blog/2018/01/31.html

willghatch 9y ago

Shell is my favorite domain-specific language. But many (including myself) would argue that domain-specific languages are generally better embedded. Many projects aiming to mixing shell with general purpose languages find a nice embedded DSL for subprocess/pipeline management. Some others find a convenient way to run shell commands or pipelines by mixing grammars and trying to disambiguate them. [Shameless self-promotion] I've been working on a project that aims to not only have a nice DSL for running process pipelines, but focuses on making a syntax for using all functionality of the host language as a command language, called Rash[1]. It is hosted in Racket, and can be embedded in Racket at the module or expression level, and also normal Racket expressions can be embedded in Rash (they can alternate arbitrarily deep). It supports process pipelines, Racket function/object pipelines, and mixes of the two. It's still alpha and has a TODO list a mile long, but I've been using it as my daily driver interactive shell for months and have loved it so far. [1]: https://github.com/willghatch/racket-rash https://github.com/willghatch/racket-rash

chubot 9y ago
FWIW I have your project on my wiki page :) https://github.com/oilshell/oil/wiki/ExternalResources https://github.com/oilshell/oil/wiki/ExternalResources I guess what you mean by embedded is that it should be an embedded DSL in a full-fledged programming language? I don't quite agree, since there are at least 20 projects like that on the wiki page, none of which is popular. Probably the most popular one is eshell, in Emacs Lisp? But if there's something I don't know about I'd be interested in hearing it. This idea goes back at least 20 years, e.g. to scsh. And it hasn't taken off. But certainly I don't begrudge anyone if their favorite environment is Racket and they want to stay in Racket. That's a perfectly reasonable thing. It's just not what I would expect anyone besides racket users to use. One reason I'm interested in shell because it's the lowest common denominator between different groups of programmers. C/C++ programmers, use it heavily, Python, Ruby, JS programmers, Go, etc. Everybody uses it.
- willghatch 9y ago
  Yes, actually I've looked over all the shells on that wiki page. I think most of them haven't taken off because they either their host language has been poor or unpopular, their design or implementation wasn't great, or they haven't really solved the right problem. I think the idea of embedding a shell into a general-purpose language still has a lot of merit. Most of those projects trying to embed a shell DSL into a general-purpose language, scsh included, are basically for programming and not interactive use. Shells that are only interactive or only for programming end up fulfilling less than half of the purpose of shell, in my view, because the interaction and scripting feed back into each other in a virtuous cycle. The only ones on that list aside from Rash that try to be embedded in a general-purpose language while being good at both interactions and programming are Xonsh and eshell. Xonsh is pretty new (I wasn't aware of it until after I had made Rash), and eshell is in Emacs Lisp (which is not a very good programming language or platform to build on for anything except extending the emacs editor). Rash also tries to be better than the competition by adding object pipelines (much like Powershell, it makes it much more reasonable to make system administration commands in the host language, and have rich interaction and inspection of command results), user-defineable pipeline operators, and generally tighter integration with the host language while still having a light syntax for basic commands. I would like to be able to have my command language tightly integrated with my programming language, and be able to define my system administration commands (and interactive shell plugins and completion) as functions in the host language (while still being able to naturally use external programs). And I would like my shell scripts to be able to grow and potentially even turn into full-fledged programs, or modules of some larger program. I think there are a lot of benefits to the approach I'm using (which would be too long for a comment here). That said, I'm not holding my breath for it to catch on widely any more than I'm holding my breath for Racket to take off as a popular programming language (although I frankly wouldn't mind either one). I think a better Posix shell is certainly a noble effort, because whatever better shell does become popular, we certainly need one. And an automatic upgrade path for existing scripts sounds great. So I salute you and wish you good luck with it. Also, as someone looking at shells and their features to copy the best ones, your wiki is a great resource. So thanks.

rendaw 9y ago

I disagreed with this strongly. > concurrent processes Job control is disabled in shell scripts, and your only other option is juggling process ids. Combined with nearly nonexistent exception handling using anything more than a single process at a time is like pulling teeth. Things like ssh have super awkward workarounds using control pipes. I'm not sure I've ever seen a shell script use concurrent processes in the wild. Python's subprocess library is excellent and makes concurrent processes a breeze. > file system With some minor exceptions, I can't think of any FS ops I'd do in shell that isn't just a couple letters longer in Python and 10 times more flexible. The only reason to use shell is that it has a simple syntax for pipelines. I rewrite any script > 2 lines in Python and have no regrets.