17 ms·

Why Create a New Unix Shell?

• jhillyerd 9y ago
  After using fish for 3 years, I'm finding there is very little reason to have my login shell maintain backwards compatibility with bash. The only time I run into issues is when a command expects manipulate environment variables via bash syntax. I think the fish documentation WRT to scripting could be much better, but the language is more elegant than bash or PowerShell IMHO.
  • aaron-lebo 9y ago
    Non-bash compliant shells suck from the Google, copy, and it works perspective. I even shy away from using zsh because most setups assume bash.
    • oblio 9y ago
      Just launch a subshell and problem solved. I switch between zsh, bash, cmd, powershell quite frequently.
  • CGamesPlay 9y ago
    > The only time I run into issues is when a command expects manipulate environment variables via bash syntax. And in my experience 90% of those are in the form `FOO=bar command` which can be replaced with `env FOO=bar command` and works just fine in fish.
    • ComputerGuru 9y ago
      Both support for `&&` and `||` (instead of `and` and `or`) as well as supporting `FOO=bar command` are under consideration for fish 3.0 to ease the migration path. The former is pretty much going to happen, the latter if we get around to it, DV.
      • geewee 9y ago
        This would be much appreciated. I know there are a few people on my team that cam't use fish due to our npm scripts needing to be compatible with cmd
        • ComputerGuru 9y ago
          Why don’t your npm scripts specify /bin/sh as the interpreter?
    • defined 9y ago
      Where things get problematic is with commands that send a set of environment variables to stdout, like eval $(ssh-agent) Sure you can get addons (like bass[0]) that will translate the sh environment variable settings to fish, but it’s a pain to have to do that (and remember wth it was called). [0] https://github.com/edc/bass https://github.com/edc/bass
  • bodyfour 9y ago
    I sort of agree that there isn't any strong reason for an interactive shell to have 100% bash compatibility. However, when I tried to convert to using fish I found my muscle-memory used too many simple history substitutions like "!!" and "!$" (which actually pre-date bash; they arrived with csh 40 years ago!) which were missing. Ultimately I gave up and went back to bash. It sort of is an "uncanny valley" for a text interface. It feels close enough to a traditional UNIX shell that I start to interact with it like one... but has enough differences that I found myself constantly tripping over them.
    • djsumdog 9y ago
      It was difficult to implement !! and !$ in fish (I rarely actually used them in bash, so it wasn't an issue for me) and I know others who found that annoying. However if you dig around, there are a couple of solutions on Stack Overflow and blog posts that add in that functionality using plugins or functions. It's not exactly the same, but some of them work pretty well.
      • askz 9y ago
        I use !! maybe ten times per day. !$ can be implemented in your PS1 easily.
        • ViViDboarder 9y ago
          I used to. Now I just Ctrl-P.
      • claar 9y ago
        You may be underestimating the importance of these particular operators to experienced users. I at least would be very dissuaded from trying an alternate shell without a simple way to enable these.
  • ViViDboarder 9y ago
    Same. The assumption that both types of users need to have their use case solved by the same language is foreign to me. I use Fish as my shell, but I do scripting with Bash. There’s nothing that prevents you from doing so unless you’re sourcing a file.
  • jernfrost 9y ago
    Interesting seeing so many fish fans. I absolutely love fish. It makes my everyday shell usage so much nicer. But it seems like a totally unknown shell to most people. I never see anybody else use it at any job I've had. I did use fish a bit as a script language, but I decided for anything of any size I much prefer Julia. For typical file system navigation, fish is better, but Julia is actually pretty decent as a shell, despite being a real language. So writing shell scripts in it is pretty nice. In the beginning I wrote separate programs executed from fish shell. But now I just fire up Julia as a shell and run functions directly there interactively.
    • dilap 9y ago
      I remember a few years ago poking at julia and thinking it would make a really good shell language. The thing that killed it for this use at the time was slow startup; is that better now?
      • Sean1708 9y ago
        Much better, it's certainly worth giving it another go. It's still much slower than Python, but it's quick enough that I don't notice it all. $ time julia -e 'println("Hi")' Hi real 0m0.241s user 0m0.216s sys 0m0.196s $ time python3 -c 'print("Hi")' Hi real 0m0.046s user 0m0.020s sys 0m0.000s
        • dilap 9y ago
          Inspired me to install and try; about 350ms on my macbook pro. Much better than it used to be, but still more than you'd want for everyday commands (at least if you're picky about having your computer feel responsive, which I am). :-)
• ComputerGuru 9y ago
  Lots of overlap in design goals with fish, except fish also places a premium on users interactively using the shell (which means friendlier in-repl experience but a balancing act when it comes to features). Fish’ auto completions are incredible, too. Best of luck to them. Another interesting shell to check out is elvish, lots of new ideas there (even if awkward to use). (Disclosure: I’m one of the core fish devs/maintainers. Edit: The entire team is awesome and the others deserve virtually all the credit!)
  • 0xdeadbeefbabe 9y ago
    Do you have anything to say about rc https://9fans.github.io/plan9port/man/man1/rc.html https://9fans.github.io/plan9port/man/man1/rc.html ?
    • fusiongyro 9y ago
      If you like rc, do you like es? http://wryun.github.io/es-shell/ http://wryun.github.io/es-shell/
      • 0xdeadbeefbabe 9y ago
        I don't know. I don't like installing new things on servers in general.
        • askz 9y ago
          But you do test new things you install on your servers somewhere else, at least, I hope so.
        • fusiongyro 9y ago
          So, bash it is. :) I love using fish and enjoy scripting with it too, but I'm hampered by this fact as well, and mostly use bash at work when others might see it or have to use it.
        • ComputerGuru 9y ago
• castis 9y ago
  I see that this person has opted not to use python 3 because it is 'less-suited to shell-like problems' than python 2. In an effort to understand the reasons for actively choosing against 3, does anyone know what problems those would be?
  • wutbrodo 9y ago
    From the post: > I encountered a nice blog post, Replacing Shell Scripts with Python, which, in my opinion, inadvertently proves the opposite point. The Python version is more difficult to write and maintain. Here's the link: https://medium.com/capital-one-developers/bashing-the-bash-replacing-shell-scripts-with-python-d8d201bc0989 https://medium.com/capital-one-developers/bashing-the-bash-r... I think, roughly speaking, the fact that Python 3 is much closer to a sane language for engineering means that it's less suited for scripting.
  • chubot 9y ago
    (author here) I was hoping I would have time to write part 2 of the FAQ before this hit HN, since that is definitely a FAQ. tl;dr I used Python for prototyping; it will be removed. Consider it an implementation detail -- building and running Oil does not require Python, as a portion of the interpreter is bundled in the tarball. Python 2 vs. 3 doesn't really matter. It was in Python 3 at one point. More discussions here: https://www.reddit.com/r/ProgrammingLanguages/comments/7tu30g/why_create_a_new_unix_shell/dth02d2/ https://www.reddit.com/r/ProgrammingLanguages/comments/7tu30...
  • dfox 9y ago
    One reason I can see is that while Python 3's unicode handling is saner for most of usecases it does not work the way one would expect in unix shell.
    • kevin_thibedeau 9y ago
      Python isn't going to reconfigure an ASCII shell on your behalf. It's up to you to enable a Unicode locale and then PY3 just works.
• zapita 9y ago
  Finally a modern shell that understands the importance of COMPATIBILITY! This gives it a realistic chance of getting real adoption. Shells like zsh and fish will never get mainstream adoption because they are not compatible with bash.
  • partycoder 9y ago
    You can always invoke a script and have it execute with the right shell using the shebang line. https://en.wikipedia.org/wiki/Shebang_(Unix) https://en.wikipedia.org/wiki/Shebang_(Unix) I used fish for years, never had a problem with bash or zsh scripts.
  • aspaceman 9y ago
    Why do people care about "COMPATIBILITY" so much w.r.t. shells? It's so easy to use other shells to run your script. > /bin/bash your_script.sh And if your script is written with bash in mind, use a shebang: > #! /bin/bash And it will work perfectly fine on fish. As long as I have a bash binary, why do I need COMPATIBILITY?
    • yjftsjthsd-h 9y ago
      If you use the same shell interactively as you script in, then you only have to learn the one language.
      • jamesgeck0 9y ago
        Except Oil has both POSIX compatibility and Oil. It's not much difference from Fish, where you can largely ignore the built-in scripting language and just write bash scripts if you'd like.
        • yjftsjthsd-h 9y ago
          Yeah, with oil I agree. It appears that fish is sometimes incompatible, though; there's a comment down thread about 'Foo=bar baz' working differently, and I have actually used that interactively.
• wainstead 9y ago
  > Shouldn't we discourage people from writing shell scripts? ...people frequently ask this? Tip #21 of "The Pragmatic Programmer" states: "Use the Power of Command Shells."
  • partycoder 9y ago
    Manipulating processes, exit codes and output is idiomatic in shells and that's where they shine. I totally agree with you. In a general purpose programming language there's a lot of overhead for doing the same things. For maintainability, there are now linters for shell languages that can help making the job easier.
    • yjftsjthsd-h 9y ago
      > linters for shell languages Obligatory in case anyone hasn't seen it: https://www.shellcheck.net/ https://www.shellcheck.net/ Works as a web app or local tool.
      • partycoder 9y ago
        Please note that copying and pasting commands from a web browser into a terminal can result in malicious code being executed. A good idea to double check using a text editor. http://thejh.net/misc/website-terminal-copy-paste http://thejh.net/misc/website-terminal-copy-paste
        • pbhjpbhj 9y ago
          Sounds like sandboxing of pasted code until confirmation would be a good feature for a new shell ...
          • yjftsjthsd-h 9y ago
            There are terminals that do this, I believe.
      • snag 9y ago
• dmix 9y ago
  I checked out Oil previously which looks nice but more of an incremental improvement over Fish/ZSH rather than a significant evolution (it may have changed since then, this was last year). Im most excited about Elvish shell and the language that's being developed around it. The shell is built with Go and feels super fast compared to my plugin-heavy ZSH. The language design is quite nice too but still very alpha. Looking forward to see what it evolves into... https://github.com/elves/elvish https://github.com/elves/elvish
  • chubot 9y ago
    FWIW, OSH is the incremental improvement, and Oil is the new language (explained in the intro to this post.)
    • dmix 9y ago
      I'm aware of the difference, but they are both very much integrated into a single UX. The language was what I'm most interested in because writing ZSH is a giant headache even though I've been doing it for years, it' s still painful. That plus the performance of the shell itself.
• XorNot 9y ago
  The pain of "not quite bash" ultimately is what put me off xonsh as my daily shell. Also losing it whenever I SSH'd somewhere. The latter problem could probably be solved with a wrapper which would pipe and execute the shell (or a bytecode interpreter ala shuttle?) automatically - but I've seen no alternative shell project take this part seriously for the problem space.
• tambourine_man 9y ago
  A little tangential, but I keep wondering if Apple is developing its own shell or will adopt one with a more liberal licence such as Oil or Fish. I mean, they can't keep using Bash 3 forever, right? (Hope)
  • djsumdog 9y ago
    I find it interesting that Apple actually downgraded to an older version of bash at some point to avoid the GPLv3. I reference it in a post about Open Source and business I wrote two years back: http://penguindreams.org/blog/the-philosophy-of-open-source-in-community-and-enterprise-software/ http://penguindreams.org/blog/the-philosophy-of-open-source-...
  • Noctem 9y ago
    I really don't understand why they haven't made ZSH the default. It has a much more powerful REPL and a more permissive license.
  • siteshwar 9y ago
    fish is released under GPLv2 and afaik Apple has a policy to not include any new GPL software in macOS.
    • tambourine_man 9y ago
      I think GPLv2 is fine, GPLv3 is what stopped them from using new versions of Bash. But I'm sure they would prefer MIT, Apache, BSD, etc
  • chubot 9y ago
    I'm not a frequent Mac user, but I wonder about the bash 3 thing too. Maybe they just expect you to install your own shell? I think a lot of people do that with homebrew? I think Apple has largely expunged shell scripts from the startup process with launchd too? That is like their systemd.
• oleks 9y ago
  Why is Oil implemented in Python? IMO Python is a terrible language for writing programming languages.
  • oblio 9y ago
    Why?
    • GhostVII 9y ago
      Maybe because python is very slow?
      • mattgreenrocks 9y ago
        pypy gives decent speed, and RPython is quite impressive. But I'm sure it is ease of prototyping and exploring the design space.
      • Myrmornis 9y ago
        For this to be relevant you'd have to explain that you expect the shell process itself to be doing a lot of CPU-bound work.
  • chubot 9y ago
    I will address that in part 2 of the FAQ, but the short answer is: 1) I prototyped it in Python; the dependency on the Python interpreter will be removed [1] 2) Oil went through many implementation languages, and one incarnation was 2000-3000 lines of C++. But I realized I would NEVER finish that way. The goal is to be compatible with bash, which is a tall order. 3) Oil is heavily metaprogrammed. It's only 16K lines of Python, compared to 160K lines of bash, and it can run some of the most complex bash programs out there. [2] It's more accurate to say Oil is written in Python + ASDL [3], i.e. somewhat in the style of ML. [1] https://news.ycombinator.com/item?id=16277358 https://news.ycombinator.com/item?id=16277358 [2] http://www.oilshell.org/blog/2018/01/15.html http://www.oilshell.org/blog/2018/01/15.html [3] http://www.oilshell.org/blog/tags.html?tag=ASDL#ASDL http://www.oilshell.org/blog/tags.html?tag=ASDL#ASDL
• fiatjaf 9y ago
  What kind of shell can I run on a server without filesystem access that I can open to external untrusted users?
  • chubot 9y ago
    What's your use case for that? If it doesn't have file system access, does that mean it can't run any programs? I believe Oil will be able to do this, because the architecture is very modular. See the last point in the post using the LLVM / GCC analogy. (This type of feature isn't a priority now, but I'm interested in hearing use cases.)
  • Spivak 9y ago
    Restricted bash? You will need the cooperation of your SSH server to fully lock it down but it gives you the tools. https://www.gnu.org/software/bash/manual/html_node/The-Restricted-Shell.html https://www.gnu.org/software/bash/manual/html_node/The-Restr...
• fiatjaf 9y ago
  Wasn't there a in-browser Javascript shell somewhere a while ago? It could fetch URLs and do cool stuff with APIs.
• simias 9y ago
  The idea to use a real programming language as a shell is a common one but I'm not sure it's really a problem that can be solved without reworking the kernel interface. Whatever you do pipes will still be byte streams, error handling will always be using integer return values, you'll always have stdin, stdout and stderr, job control and signal handling will always work pretty much the same way. The kernel interface exposes an interface, the userland app expect the shell to behave in the certain way, there's not a lot of wiggle room to make things differently in between the two. Not that the POSIX-like shell syntax is not all sorts of clunky and odd but I almost consider it a feature, it's a deterrent to force you to move to a "real" scripting language when the concepts become too complex to express in a shell script.
  • laumars 9y ago
    I've been considering this problem myself and I believe there are ways to get around the limitations you've described. It's not pretty (from an ideological perspective) but it does work and is mostly invisible to the end user. The problem is when users hit those weird edge cases where the hack becomes visible. :-/
  • effie 9y ago
    The whole point of these projects is to evolve the shell, go beyond the current bash standard to make the work with the shell more fun and more productive. Unix may seem to have limited default interfaces, but this was by design - the idea was, the programmer/user knows much better what he needs so let him build it. The system's role was shaped to provide robust universal mechanisms. Also, the current standard shell is very limited when compared to even existing unix interfaces. For example, the kernel provides select/poll system calls but standard shell has no facility to use them, so simultaneous processing of two or more data streams without blocking is not currently possible. A new shell could finally provide them.
    • jklowden 9y ago
      Finally? You don’t need a new shell. Write select(1). Job done.
• z3t4 9y ago
  I spent the day making a virtual terminal ... I think the terminal is an unnecessary layer. All program UI is limited by this 40+ year old technology that is the terminal. Instead of making a new shell, make a shell + new user-interface.
  • Myrmornis 9y ago
    That's an interesting comment. One downside is that I believe it would necessarily be tied to a particular choice of OS, whereas with the standard separation, our shells can be cross-platform but our terminal emulators are OS-specific.
  • chubot 9y ago
    I agree, but the first step is to make a shell. I imagine it will be like Vi or Emacs -- it can write to a terminal, or have its own UI. I have been keeping a wiki page: https://github.com/oilshell/oil/wiki/Interactive-Shell https://github.com/oilshell/oil/wiki/Interactive-Shell Although honestly I won't get to any of this in the near future.
  • jiggunjer 9y ago
    A day? It would take me a month reading the kernel code just to know where to start and what to replace.
    • z3t4 9y ago
      I half-hearted implemented the "vt100" ANSI Escape sequences in the most naive way possible into an existing GUI application. Did not touch any kernel code. I think the modern "terminal" is the Browser. With URL's instead of file paths. But I think it's maybe time for something new. In the 70's we got the terminal. 20 years later we got the browser. Now another 20 years have passed. What's the next step ? Terminal -> Browser -> ?? -> AI ?
  • executesorder66 9y ago
    You might be interested in this: https://github.com/withoutboats/notty https://github.com/withoutboats/notty
• cestith 9y ago
  In your FAQ you decry Perl as having no ability to redirect around other programs. Yet you don't explain how any of the following fail to meet those needs.: * http://perldoc.perl.org/functions/open.html http://perldoc.perl.org/functions/open.html * http://perldoc.perl.org/IPC/Open2.html http://perldoc.perl.org/IPC/Open2.html * http://perldoc.perl.org/IPC/Open3.html http://perldoc.perl.org/IPC/Open3.html * http://search.cpan.org/~odc/IPC-Open2-Simple-0.01/lib/IPC/Open2/Simple.pm http://search.cpan.org/~odc/IPC-Open2-Simple-0.01/lib/IPC/Op... * http://search.cpan.org/~exodist/Child-0.013/lib/Child.pm http://search.cpan.org/~exodist/Child-0.013/lib/Child.pm * http://search.cpan.org/~rkrimen/IPC-RunSession-Simple-0.002/lib/IPC/RunSession/Simple.pm http://search.cpan.org/~rkrimen/IPC-RunSession-Simple-0.002/... * http://search.cpan.org/~trski/Proc-Forkmap-0.025/lib/Proc/Forkmap.pm http://search.cpan.org/~trski/Proc-Forkmap-0.025/lib/Proc/Fo... * http://search.cpan.org/~toddr/IPC-Run-0.96/lib/IPC/Run.pm http://search.cpan.org/~toddr/IPC-Run-0.96/lib/IPC/Run.pm * http://search.cpan.org/~ayoung/IPC-Run3-Simple-0.011/lib/IPC/Run3/Simple.pm http://search.cpan.org/~ayoung/IPC-Run3-Simple-0.011/lib/IPC... * http://search.cpan.org/~rjbs/IPC-Run3-0.048/lib/IPC/Run3.pm http://search.cpan.org/~rjbs/IPC-Run3-0.048/lib/IPC/Run3.pm * http://search.cpan.org/~djerius/IPC-PrettyPipe-0.03/lib/IPC/PrettyPipe.pm http://search.cpan.org/~djerius/IPC-PrettyPipe-0.03/lib/IPC/... * http://search.cpan.org/~xan/IPC-Pipeline-1.0/lib/IPC/Pipeline.pm http://search.cpan.org/~xan/IPC-Pipeline-1.0/lib/IPC/Pipelin... * http://search.cpan.org/~sscaffidi/IPC-OpenAny-0.005/lib/IPC/OpenAny.pm http://search.cpan.org/~sscaffidi/IPC-OpenAny-0.005/lib/IPC/... * http://search.cpan.org/~glai/IPC-Exe-2.002001/lib/IPC/Exe.pm http://search.cpan.org/~glai/IPC-Exe-2.002001/lib/IPC/Exe.pm * http://search.cpan.org/~zefram/IPC-Filter-0.005/lib/IPC/Filter.pm http://search.cpan.org/~zefram/IPC-Filter-0.005/lib/IPC/Filt...
  • chubot 9y ago
    Can you show me some code? How do you write this in Perl? f() { echo -- ls / echo -- } f > out.txt f | wc -l
    • tankenmate 9y ago
      Here is probably the simplest answer, it's not totally correct but it's the shortest answer that fits the main criteria. #!/usr/bin/perl use strict; sub f { my $outputFH=shift; print $outputFH "--\n"; open(my $lsFH,"ls /|") or die("pipe ls: $?"); print $outputFH (<$lsFH>); close($lsFH); print $outputFH "--\n"; } open(my $outTxtFH,">","out.txt") or die("open: out.txt:$?"); f($outTxtFH); close($outTxtFH); open(my $wcFH,"|wc -l") or die("pipe wc: $?"); f($wcFH); close($wcFH);
• psibi 9y ago
  Looks nice! Any reason why Apache was used as it's license ?
• kevlar1818 9y ago
  > However, Python and Ruby aren't good shell replacements in general. Shell is a domain-specific language for dealing with concurrent processes and the file system. But Python and Ruby have too much abstraction over these concepts, sometimes in the name of portability (e.g. to Windows). They hide what's really going on. Excellently put. POSIX shell languages have fantastic capabilities you just can't get in most other languages. I would love to see a more safe, more sane shell language gain enough popularity to change the narrative that "shell scripts are dangerous and impossible to maintain." The contrasts to Python and Ruby made me think of xonsh[1], a Python-based shell that can dynamically switch between Bash-like syntax and standard Python. It's not quite ready to become my daily driver, but I'm still excited about it. [1]: https://xon.sh https://xon.sh
  • chubot 9y ago
    Thanks, and you have put it very well too. As I've learned from many of these threads [1], there most certainly is a narrative that shell scripts are dangerous and impossible to maintain. People are really angry about it! And of course I agree with that! That's the whole reason for Oil. [1] http://www.oilshell.org/blog/2018/01/31.html http://www.oilshell.org/blog/2018/01/31.html
  • willghatch 9y ago
    Shell is my favorite domain-specific language. But many (including myself) would argue that domain-specific languages are generally better embedded. Many projects aiming to mixing shell with general purpose languages find a nice embedded DSL for subprocess/pipeline management. Some others find a convenient way to run shell commands or pipelines by mixing grammars and trying to disambiguate them. [Shameless self-promotion] I've been working on a project that aims to not only have a nice DSL for running process pipelines, but focuses on making a syntax for using all functionality of the host language as a command language, called Rash[1]. It is hosted in Racket, and can be embedded in Racket at the module or expression level, and also normal Racket expressions can be embedded in Rash (they can alternate arbitrarily deep). It supports process pipelines, Racket function/object pipelines, and mixes of the two. It's still alpha and has a TODO list a mile long, but I've been using it as my daily driver interactive shell for months and have loved it so far. [1]: https://github.com/willghatch/racket-rash https://github.com/willghatch/racket-rash
• Sir_Cmpwn 9y ago
  I use fish but I hate it. My perfect shell is strictly POSIX sh but has a better interactive experience (better tab completion and typeahead like fish has).
• jernfrost 9y ago
  I essentially solved these problems with fish shell and Julia programming language. For all sorts of interactive stuff I use fish, because it works the way you want for the most common tasks. I can use it to really quickly match and get back previous statements or do a completion. Also much easier to configure and grok than bash, because it has saner syntax and is a simpler shell language. However when writing shell scripts I use a real language like Julia. It integrates very well with the shell world so I don't find it problematic to do this. It is very easy to read output from processes and pipe stuff. Much nicer than say Python or Ruby. You got built in syntax to deal with shell stuff, but then didn't make it crazy so you end up with a mess like perl. Julia is actually a very clean and nice language. Which also happens to blistering fast and have LISP style macros.
• JepZ 9y ago
  Sounds pretty cool and everybody who had to learn bash scripting at some point understands why we need a sane language (my favorite are misplaced spaces in if statements...; Disclaimer: I do and love bash scripting but while the language has cool concepts, some things are just broken by design). Nevertheless, there is one piece in this puzzle I am missing. There does not seem to be a process which manages the 'core software set' across platforms. So after decades we finally have a shell which is available on most operating systems, but how long will it take before Microsoft, Apple, Oracle, etc. will adopt a new shell? So why don't the large OS corporations form a consortium to define something like a 'cross platform run time environment' standard (maybe together with the Linux Foundation and some BSD guys?). I mean its not so much about which shell someone prefers, but more about a common set of interpreters and maybe tool kits. And even more than that it is not about the state but the process/progress. What do you think, do we need such a process or is there another way to solve the cross platform dilemma?
  • webreac 9y ago
    bash is good enough for launching commands and short scripts. When you want to manipulate data that may contain special characters or write not trivial algorithms, it becomes insane. I think it is a feature. It indicates that bash is not the good tool for that. bash, sed, awk are excellent tools. I know all the basic stuff about them and I know when it becomes tricky. When it becomes tricky, I switch to python or perl.
  • sedachv 9y ago
    What you are describing has been around since the late 1980s, IEEE POSIX: https://en.wikipedia.org/wiki/POSIX https://en.wikipedia.org/wiki/POSIX
    • JepZ 9y ago
      Well, POSIX is pretty similar to what I mean, but it has a lot of low level stuff and I doubt that Microsoft has any ambitions to transform Windows into a POSIX compatible OS. I thought more about a higher level standard like adding Python, Lua or Qt to every installation by default. As some of those things are pretty heavy I doubt that it would be a wise choice to include them in POSIX. Just imagine a world were you could simply write a small python script which would start a complete GUI application on different platforms without any additional installation procedures. To my knowledge that is not possible today. AFAIK the only way today is to bundle the dependencies, but that has a lot of negative effects.
• anderspitman 9y ago
  Love it. I think there's plenty of room for innovation in this space. I'm currently in the process of porting a ~700 line bash script (I didn't write it) to Python. Although longterm I think this will be much better for the project, there are still tradeoffs. There are some things that are just so easy to express in shell language, like composing transformations via pipes. Sure, python can do it, but it feels clunky in comparison to me. I would love to see a language like python (including package ecosystem) written with shell use as a first-class citizen.
  • zanchey 9y ago
    There's definitely room for innovation - fish is for me but I'm always impressed with the work being done on both new and old shells. One thing fish doesn't do is get much into the semantics of how processes interoperate, and I'm interested to see if there's a new idea that can gain some traction in that regard.
• JepZ 9y ago
  @chubot: There is one use-case I come across every once in a while: https://stackoverflow.com/questions/356100/how-to-wait-in-bash-for-several-subprocesses-to-finish-and-return-exit-code-0 https://stackoverflow.com/questions/356100/how-to-wait-in-ba... So whenever you want to do things in parallel there is probably a limit to the number of processes you would like to execute in parallel (e.g. the famous compiler limit formula: number of CPU cores +1). It would be great if Oil could support such a use-case out of the box, as easy parallelism without the ability to artificially limit the number of parallel executions is often useless.
  • chubot 9y ago
    Absolutely. In fact, the bash manual explicitly refers to GNU parallel for this use case! I use xargs -P all over the Oil codebase, which does what you want. The trick is to end the file with "$@", and then invoke xargs -P 4 -- $0 my-func. That way xargs can run arbitrary shell functions, not just something like sh -c "..." ! I'm going to write a blog post about this. I also do this with find -exec $0 myfunc ';' https://github.com/oilshell/oil/blob/master/test/spec-runner.sh#L220 https://github.com/oilshell/oil/blob/master/test/spec-runner... However I think Oil will have something built-in to make this parallel process more friendly. I will probably implement xargs so it can run my own shell scripts without GNU xargs, but then add a nicer syntax. (Probably "each", since that's what xargs really does.) This gets into your other question about standard utils, which I'll answer now. Short answer: yes I would like something like that, it's just a matter of development time and priorities. I agree with the problem you point out.
    • JepZ 9y ago
      Sounds pretty cool. My biggest problems with xargs is that I had constantly some weird edge cases, so I try to avoid it. As GNU parallel doesn't seem to be part of standard installations it would be an external dependency for a script, which I try to avoid too. So I ended up using the loop syntax: for i in {0..9}; do echo "$i" & done wait It is not so Unix like, but I find it easier to debug. It would be great if Oil would have a solution for limiting that kind of parallel execution too. I am aware that this isn't simple as there are different options here how to implement it (global limit vs. local limit vs. named limit). Just an idea from the top of my head an idea for an optional named limit: Lets call it 'flow': flow [options] [command]': -n number of max parallel processes -c (optional) identifier of the counter. Example: for i in {0..9}; do flow -c myCounter -n 4 echo "$i" done Just an idea.
• donatj 9y ago
  I was with the guy right up until the bile laden PHP hated started coming in as a justification.
  • chubot 9y ago
    (author here) I didn't intend to criticize PHP, and I don't think I did. I said that you can't convince people not to use bash or PHP by writing posts on the Internet, which is true. I also said that Facebook is replacing PHP, which is true. That's not a criticism of PHP. The fact that huge companies like Yahoo and Facebook can be started with PHP is amazing. I think PHP is a good analogy for bash. It gets the core things right, and it gets a ton of work done. I like languages you can get work done in! That's why I use bash. But both languages also evolved a lot of warts. That's inevitable when you have so many users. They have diverse needs, and you need to preserve backward compatibility, which leads to an awkward evolution.
• devit 9y ago
  Seems an interesting idea, but it's implemented in Python, which means it will never replace bash and probably not achieve any significant adoption unless they rewrite it in Rust first (which they should have written it in to begin with since they started in 2016). The reasons for that are that shells must start very quickly (due to subshells, local ssh, etc.), be fast, have no complex dependencies since they are used to recover broken systems, be portable but also with full support for OS semantics and be written in a language that allows rapid development of robust software, none of which Python does well.
  • chubot 9y ago
    https://news.ycombinator.com/item?id=16277734 https://news.ycombinator.com/item?id=16277734
• aplorbust 9y ago
  This keeps showing up on the front page. No doubt it will have users. Lets say there are two uses of a shell: 1. interactive and 2. non-interactive (scripting). Lets imagine the commandline user is learning about her OS. She learns it is heavily reliant on shell scripts to build and (if desired) to automate starting services. She realises that to understand the OS she will have to learn the shell that the OS developers used for scripting. Then she realises that if she chooses another shell for interactive use, she will have to learn two shells. Finally she realises that any script she writes in the "non-interactive/scripting" shell will also run under the interactive one. But not vice versa. If she only has enough time in life to master one shell, which one should she choose? Over time I found I really cared more about the scripting aspect of a shell than the interactive facet. The scripting shell used by the OS authors might be an Almquist derived shell, for instance. Occasionally the ash Im using gets a new "feature" but not too often. I like that it stays relatively small. The latest "feature" is LINENO. But I also use a smaller version of this shell with no command line history, no tabcomplete, etc. IMO, there is no better way to learn how to reduce keystrokes. It has led to some creativity in this regard for which I am thankful. After "mastering" ash, I started using execlineb, pipeline and fdmove. I am starting to use more components of execline and am continually replacing ash scripts with execline scripts for more and more daily work. I guess we will never see execline on the front page, which I think would be interesting because I would like to hear whatever harsh critique HN can muster. Seeking a better non-interactive/scripting experience, I have experimented with many other shells over the years, and written simple execve "program launchers", but in this vein, I have not found anything that compares to execline. The speed gains and resource conservation are obvious, but with the ability to do "Bernstein-chaining" and the option to use djb low-level functions instead of libc, it is a rare type of project. The speed and cleanliness of the compilation process is, compared to all the other crud one routinely encounters in open source projects, "a thing of beauty". Humble opinion only, but I think others might agree.
  • JdeBP 9y ago
    It was submitted once. * https://news.ycombinator.com/item?id=12600807 https://news.ycombinator.com/item?id=12600807 Laurent Bercot no longer has xyr page about the compilation process. I have since picked up some of the slack there. Although I don't go into things like the way that M. Bernstein avoided autotools. * http://skarnet.org/software/compile.html http://skarnet.org/software/compile.html * http://jdebp.eu./FGA/slashpackage.html http://jdebp.eu./FGA/slashpackage.html
• jiggunjer 9y ago
  So many shells, but how come the virtual terminal hasn't got a revamp or popular alternative? I'd like to see more control over keybinding (e.g. on my machine pressing ctrl+del sends the same character as pressing ctrl+backspace). I understand this would be more a kernel change than a userland one. Screen tiling and visual 'tabs' would also be welcome additions. Not everyone needs a graphic environment, and I refuse to install X just for better keyboard shortcuts on my terminal.
  • aidenn0 9y ago
    dvtm does tiling and tabs Since dvtm also works as a terminal emulator, it seems to me that you could use loadkeys to setup various keycodes to send the proper vt100 escape codes. I've not tried it, but see no reason why it shouldn't work. If you just want "No X install" you can use a frame buffer terminal (fbterm was one I used to use, but it doesn't appear to have been updated in a while, perhaps there is a spiritual successor, or maybe it already does what you want) [edit] YAFT https://github.com/uobikiemukot/yaft https://github.com/uobikiemukot/yaft looks like it's more up to date than fbterm.
  • JdeBP 9y ago
    It has, a fair number of times. * https://unix.stackexchange.com/a/177209/5132 https://unix.stackexchange.com/a/177209/5132 * https://unix.stackexchange.com/a/196102/5132 https://unix.stackexchange.com/a/196102/5132
• shmerl 9y ago
  I recently discovered Fish, which is actually not that recent, but also an interesting shell.
• fnord77 9y ago
  > Oil is taking shell seriously as a programming language, rather than treating it as a text-based UI that can be abused to write programs. erm, there's a big difference between a command scripting language and a programming language. These should be treated as different things. I have years of experience using both, and I really don't want to be doing shell tasks in a programming language and I don't want to write programs in a shell language. Those sorts of hybrids are almost always mediocre. Horses for courses and all that. There's a reason bash keeps being used - it's mature, it's simple, it's easy and people are productive with it.
• desireco42 9y ago
  As a fish user, I didn't really get where the advantages lie as clearly. I am for developing new shells, we really need this, innovation is important. As for new language, I feel like if you want to script things, you can use ruby or python, hell, perl will do and you could be fine. I don't want to be unfair to this effort, I just feel that it is not for me and I am tinkerer.
• deleted 9y ago
  [deleted]
• tyingq 9y ago
  Applaud the idea, but you are fighting a ton of inertia. I have to wonder if drawing a more clear line of when to go to Perl, Python, Lua, Ansible, Golang, etc, might be more fruitful. Sometimes, a shell script solution is just drawing any kind of shell too far outside it's core competency.
• petre 9y ago
  Oil syntax looks pretty much like Tcl and Th[1] so the author could have probably just used Th. It has saner ways to copy arrays than b = [ @a ] which pretty much looks like Perl with added line noise. Why are the [] even necessary when it's clear @a is an array? 1. http://www.sqliteconcepts.org/THManual.pdf http://www.sqliteconcepts.org/THManual.pdf
• breatheoften 9y ago
  I would love an interactive shell that applies a type-checker to my input before running it ... is that really an infeasible desire in 2018 ...?
• unixhero 9y ago
  Ignore the naysayers. I have encountered them when posting my shell activities. You're doing great! Can't wait to try the final release. Maybe even get it into my workflow.
• chrisvalleybay 9y ago
  Why is it called Oil? That name is so loaded. I think it actually will give it less of a chance.
  • 72deluxe 9y ago
    A slippery slope to a worse name? I thought it was pretty slick...... I jest.
  • chubot 9y ago
    I explain the name here: https://www.reddit.com/r/ProgrammingLanguages/comments/7qn14p/oil_success_with_aboriginal_alpine_and_debian/dsqxgjh/ https://www.reddit.com/r/ProgrammingLanguages/comments/7qn14... Bizarrely (to me), more than one person thought the name was a play on the company "Shell Oil". Is that the connotation you got from it? That's unfortunate, but I think as people use it more, the name will take on a different connotation. Guido was fighting "Python == snake" for a long time too (it comes from Monty Python). There were a lot of people that said the name Python was stupid and you couldn't convince your boss to use a language with a name like that.
• zimbatm 9y ago
  What I would really like to see happening in Oil shell is a way to run a strict subset of the bash language. Ideally with a tool to convert existing scripts in that subset.
• fanf2 9y ago
  No mention of plan9 rc or Tcl so it is hard to believe it is really breaking the mould