15 ms·

(author here) Hm I've tried fish, and it seems very nice for interactive use. However I don't see it being used AT ALL for the cloud/linux use case? Those are

by chubot 9y ago

(author here) Hm I've tried fish, and it seems very nice for interactive use. However I don't see it being used AT ALL for the cloud/linux use case? Those are the cases where you tend to get 1000+ lines of shell scripts. For example, I mention Kubernetes/Docker/Chef, and I've never seen fish in that space. I also don't know of any Linux distro that uses fish as their foundation -- they all appear to use a POSIX shell like bash/dash/busybox ash. fish is "on top". See Success with Aboriginal, Alpine, and Debian http://www.oilshell.org/blog/2018/01/15.html http://www.oilshell.org/blog/2018/01/15.html -- these distros are built with thousands of lines of shell scripts. Either 1) I don't know about such usage, 2) people don't know that fish can be used this way, or 3) there is some problem with using fish this way. I link to this post in the FAQ, which I think is a lot closer to Oil: https://ilya-sher.org/2017/07/07/why-next-generation-shell/ https://ilya-sher.org/2017/07/07/why-next-generation-shell/ It's basically the "devops" use case. (And as I mention the main difference between Oil and NGS is that Oil is compatible / has an upgrade path from bash.)

katastic 9y ago
>1000+ lines of shell scripts. Any time I have ever heard that, I've heard someone chime in "You shouldn't be using shell scripts for that." Can you explain a use-case where long shell scripts are superior to a scripting language like Python? p.s. I like that your language is statically typed.
nickez 9y ago
What is your take on Ion? https://doc.redox-os.org/ion-manual https://doc.redox-os.org/ion-manual
- chubot 9y ago
  Ion looks great, but its goals are different -- most importantly, it doesn't provide an upgrade path from bash. (This is also the biggest difference between Oil and fish, Oil and Elvish, etc.) I talked with the authors of Ion on github almost a year ago. Ion was influenced by Oil, in particular this post: http://www.oilshell.org/blog/2016/11/06.html http://www.oilshell.org/blog/2016/11/06.html
  - xiaq 9y ago
    FWIW, POSIX emulation is on my TODO list: https://github.com/elves/elvish/issues/205 https://github.com/elves/elvish/issues/205 Not sure whether I will actually get to it though. Parsing sounds like a lot of headache. Your work is very impressive, but I’m not sure I will want to go through that... :)
    - chubot 9y ago
      Yeah, running ~/.bashrc is tough, and I'm not sure even Oil will get to it. Not only do you need parsing and execution of the bash language, but you also need builtins like bind and set -o vi (i.e. see "help bind"). Maybe bind is easier than I think; maybe it's just a few readline calls. I haven't looked into it. But also, I noticed almost no shell scripts use extended globs like @(foo|bar), whereas the bash-completions package uses them EVERYWHERE. Completion scripts are almost like a different dialect than other shell scripts. If you want to source /etc/bashrc on most distros, you're going to end up with thousands and thousands of lines of plugins from the bash-completions project. The parsing is a big headache, but I think it forced me to learn how people use shell, which will benefit Oil. It's been a full year since our alternative shells thread... might be time to revive it and see what's changed! Congrats on the Elvish releases!
      - xiaq 9y ago
        Good point about bindings and completion scripts. Binding commands themselves are not big problems, the problem is replicating readline commands, there are loads of those. I'm not sure how much work one needs to do to implement things like extended globbing, but my gut feeling is that once you can parse them it's more than half done. The functionalities are quite simple, as my impression goes. I was mainly thinking about environment variables, aliases and wrapper functions when I said sourcing bashrc; that's probably what most people's bashrc files are mostly made of. We totally should revive our alternative shells thread! I'll try to summarize my progress in the last year. Congrats on your progress with Oil too! ^_^
magissima 9y ago
> consider these two groups of shell users: > 1. People who use shell to type a few commands here and there. > 2. People who write scripts, which may get into the hundreds or even thousands of lines. > Oil is aimed at group 2. If you're in group 1, there's admittedly no reason to use it right now. From my perspective, fish is basically the opposite of oil in that it mainly targets group 1 (admittedly I'm biased in that I rarely write shell scripts at all). IME fish scripts as a direct replacement for bash scripts are pretty rare, but that doesn't mean it's not a success if it's aimed at interactive use.
- arghwhat 9y ago
  "to type a few commands here and there" Where's "my shell is my desktop environment"? :/
  - Spivak 9y ago
    /pedantic Everyone's desktop environment is a shell. But any shell you like is suitable for that use case since it doesn't have to be compatible with anything other than your fingers.
kemayo 9y ago
It's an entrenched network effect. There's a lot of existing scripts written targeting POSIX shells, and there's an incentive to have all your distro maintenance scripts written in one language if possible. If you want to switch your existing distro scripts to fish, you need to rewrite a lot of stuff. If you want to start a new distro, "it's written in fish!" isn't a terribly compelling selling point, since I don't think anyone is actively picking distros based on their tooling language. For the general devops case, there's a lot of existing example code out there for bash scripts, and far less for fish. "How do I do X in bash?" is probably going to get you a decent example on Stack Overflow. Devops (I feel) cares a bit more about the installed-everywhere thing, and fish isn't a standard part of most distros. Oil having a goal of being completely sh/bash compatible gives it much greater odds of something like Debian switching to it, since it wouldn't carry huge technical debt along with it.
- toyg 9y ago
  > I don't think anyone is actively picking distros based on their tooling language. I would use the crap out of a Linux distribution built in python.
  - Insanity 9y ago
    yeah I can actually imagine that. like using an IDE based on the language it was written in to extend and tweak it more easily, or a window manager etc.. actually now you mentioned it, barring possible performance issues I'd also like a Python distro
    - rollcat 9y ago
      > an IDE based on the language it was written in I recommend Emacs. Maybe not as a daily driver (it's a matter of preference), but for the experience. I recommend at least a month with it, make sure to write some original Lisp code for your customisations. (You WILL end up customizing it, the defaults are crap.) > or a window manager I recommend Awesome. The core is in C, but that's basically the low-level stuff, the actual WM is written in Lua. If you'd start with an empty init file, you'd have a moderately sized side-project on your hands to put something usable together. But if you'd start from the stock rc, there's a world of endless tweaking and customisation waiting for you... Watch out, both Emacs and Awesome are rabbit holes.
delinka 9y ago
First, I think it's a case of momentum. All the scripts are written in shells that are distributed by default in most distros. sh, bash, etc. I write lots of scripts for fish, and I can't convince my co-workers to install fish to run my scripts. They already have bash, they already know bash, etc. Second, I was recently chided by another fish user for writing fish scripts: "you're using it wrong" was the sentiment. The argument was it's a shell intended to be interactive, but extendable. I don't know how pervasive this line of thinking is, but who's going to learn all about their shell if they're not supposed to write scripts in its language?
- aerique 9y ago
  I've been using fish for years but I never write scripts in it. What's the point? There's sh which is the most portable. Why write scripts in anything else but an actual programming language when things get more complex?
  - msla 9y ago
    The point is the compose-save-edit cycle: Compose the script in the shell, save it to a file, and then, maybe, edit it to add better argument handling or support for corner cases you didn't consider in the interactive environment.
oblio 9y ago
Totally off topic: great blog. I was reading the archives these days, they’re awesome ;)
- chubot 9y ago
  Thanks! Feel free to comment (with the reddit links at the bottom of each post)
  - oblio 9y ago
    My two main comments would be: 1. Keep up the great work (both oil and the blog!) Both seem fun :) 2. Reading your blog really shows that despite the great ideas, shell languages come from an age where computing was still in its infancy. We’re now approaching our teen years, but we’re still ahead of infants. So many awkward design decisions and organic evolutions. A shame that replacing POSIX at this point is a gargantuan undertaking.
arghwhat 9y ago
Back when I started programming, I wrote 1000+ lines of shell scripts. Now, I quite seriously believe that a 1000 line shell script only exists out of error. I still occasionally end up doing 2-300 line dense shell scripts, but not without feeling very dirty along the way. Either split into small, simple shell scripts (which is fine), or a different language. In the cross platform build pipeline at work, I keep a strong discipline when it comes to scripts: They must be short (<=100 lines), and if their complexity exceeds a certain threshold (parsing more than a "grep | cut" here and there, total program states exceeding some low number), then a shell script is no longer acceptable regardless of length. And, well, it's not safe to assume the presence of anything more than a shell. If you are writing and dealing with 1000+ lines of shell scripts, then experience tells me that you are shooting yourself in the foot. With a gatling gun. (I used fish, btw. The interactive experience was nice, but the syntax just felt different without much gain, which was frustrating to someone who often writes inline oneliners. Unlearning bash-isms is not a liberty I can afford, as I need to be proficient when I SSH into a machine I do not own or control. I can't force the entire company to install fish on all our lab servers, nor is it okay to install a shell on another persons' dev machine just because we need to coorperate.)
- kakarot 9y ago
  If your script is over 80-120 lines, it's time for modularization. No sane person writes 1000 line scripts, and the fact that the Oil author uses such a scenario as a reason to use their shell is very disconcerting about the quality of code and I wouldn't touch Oil with a ten-foot pole.
  - kelnos 9y ago
    If that's the case, then there are a lot of insane people out there. I sympathize with your opinion on how things should be (though I may or may not agree), but at the end of the day we need to look at what the reality of the world is, not on the ideal way we think people should be using the tools we already have.
    - kakarot 9y ago
StreamBright 9y ago
One question though: Do you really need 1000+ lines of shell scripts? I almost have 20 years with shell scripting and I still have the rule of not writing longer than 1-2 page shell scripts and use something else for longer projects or break it down to much smaller chunks and invoke the parts with a main.sh. Fish is a for making software engineering and system administration easier by having functionality like command completion (without tab) and it works like a charm. I would never ever would nother to rewrite my bash scripts in fish simply because bash is on all the servers and fish is on none by default. I am not sure what you are trying to solve here but good luck.
- deleted 9y ago
  [deleted]
- chubot 9y ago
  See https://news.ycombinator.com/item?id=16278141 https://news.ycombinator.com/item?id=16278141
  - StreamBright 9y ago
    This is not an answer just a problem statement.
greglindahl 9y ago
I've never met a devops person who wanted to write a 1,000+ line shell script. There are real languages that you can use instead.
jbreiding 9y ago
In the spaces you mention not hearing about fish are ones where you generally don't look for interactive applications/features. You also generally don't take a dependency like another shell when you are likely installing ruby or python already. The two shells target different use cases. Hopefully they are both successful in attracting a good audience.

by chubot 9y ago

katastic 9y ago

>1000+ lines of shell scripts. Any time I have ever heard that, I've heard someone chime in "You shouldn't be using shell scripts for that." Can you explain a use-case where long shell scripts are superior to a scripting language like Python? p.s. I like that your language is statically typed.

nickez 9y ago

What is your take on Ion? https://doc.redox-os.org/ion-manual https://doc.redox-os.org/ion-manual

chubot 9y ago
Ion looks great, but its goals are different -- most importantly, it doesn't provide an upgrade path from bash. (This is also the biggest difference between Oil and fish, Oil and Elvish, etc.) I talked with the authors of Ion on github almost a year ago. Ion was influenced by Oil, in particular this post: http://www.oilshell.org/blog/2016/11/06.html http://www.oilshell.org/blog/2016/11/06.html
- xiaq 9y ago
  FWIW, POSIX emulation is on my TODO list: https://github.com/elves/elvish/issues/205 https://github.com/elves/elvish/issues/205 Not sure whether I will actually get to it though. Parsing sounds like a lot of headache. Your work is very impressive, but I’m not sure I will want to go through that... :)
  - chubot 9y ago
    Yeah, running ~/.bashrc is tough, and I'm not sure even Oil will get to it. Not only do you need parsing and execution of the bash language, but you also need builtins like bind and set -o vi (i.e. see "help bind"). Maybe bind is easier than I think; maybe it's just a few readline calls. I haven't looked into it. But also, I noticed almost no shell scripts use extended globs like @(foo|bar), whereas the bash-completions package uses them EVERYWHERE. Completion scripts are almost like a different dialect than other shell scripts. If you want to source /etc/bashrc on most distros, you're going to end up with thousands and thousands of lines of plugins from the bash-completions project. The parsing is a big headache, but I think it forced me to learn how people use shell, which will benefit Oil. It's been a full year since our alternative shells thread... might be time to revive it and see what's changed! Congrats on the Elvish releases!
    - xiaq 9y ago
      Good point about bindings and completion scripts. Binding commands themselves are not big problems, the problem is replicating readline commands, there are loads of those. I'm not sure how much work one needs to do to implement things like extended globbing, but my gut feeling is that once you can parse them it's more than half done. The functionalities are quite simple, as my impression goes. I was mainly thinking about environment variables, aliases and wrapper functions when I said sourcing bashrc; that's probably what most people's bashrc files are mostly made of. We totally should revive our alternative shells thread! I'll try to summarize my progress in the last year. Congrats on your progress with Oil too! ^_^

magissima 9y ago

> consider these two groups of shell users: > 1. People who use shell to type a few commands here and there. > 2. People who write scripts, which may get into the hundreds or even thousands of lines. > Oil is aimed at group 2. If you're in group 1, there's admittedly no reason to use it right now. From my perspective, fish is basically the opposite of oil in that it mainly targets group 1 (admittedly I'm biased in that I rarely write shell scripts at all). IME fish scripts as a direct replacement for bash scripts are pretty rare, but that doesn't mean it's not a success if it's aimed at interactive use.

arghwhat 9y ago
"to type a few commands here and there" Where's "my shell is my desktop environment"? :/
- Spivak 9y ago
  /pedantic Everyone's desktop environment is a shell. But any shell you like is suitable for that use case since it doesn't have to be compatible with anything other than your fingers.

kemayo 9y ago

It's an entrenched network effect. There's a lot of existing scripts written targeting POSIX shells, and there's an incentive to have all your distro maintenance scripts written in one language if possible. If you want to switch your existing distro scripts to fish, you need to rewrite a lot of stuff. If you want to start a new distro, "it's written in fish!" isn't a terribly compelling selling point, since I don't think anyone is actively picking distros based on their tooling language. For the general devops case, there's a lot of existing example code out there for bash scripts, and far less for fish. "How do I do X in bash?" is probably going to get you a decent example on Stack Overflow. Devops (I feel) cares a bit more about the installed-everywhere thing, and fish isn't a standard part of most distros. Oil having a goal of being completely sh/bash compatible gives it much greater odds of something like Debian switching to it, since it wouldn't carry huge technical debt along with it.

toyg 9y ago
> I don't think anyone is actively picking distros based on their tooling language. I would use the crap out of a Linux distribution built in python.
- Insanity 9y ago
  yeah I can actually imagine that. like using an IDE based on the language it was written in to extend and tweak it more easily, or a window manager etc.. actually now you mentioned it, barring possible performance issues I'd also like a Python distro
  - rollcat 9y ago
    > an IDE based on the language it was written in I recommend Emacs. Maybe not as a daily driver (it's a matter of preference), but for the experience. I recommend at least a month with it, make sure to write some original Lisp code for your customisations. (You WILL end up customizing it, the defaults are crap.) > or a window manager I recommend Awesome. The core is in C, but that's basically the low-level stuff, the actual WM is written in Lua. If you'd start with an empty init file, you'd have a moderately sized side-project on your hands to put something usable together. But if you'd start from the stock rc, there's a world of endless tweaking and customisation waiting for you... Watch out, both Emacs and Awesome are rabbit holes.

delinka 9y ago

First, I think it's a case of momentum. All the scripts are written in shells that are distributed by default in most distros. sh, bash, etc. I write lots of scripts for fish, and I can't convince my co-workers to install fish to run my scripts. They already have bash, they already know bash, etc. Second, I was recently chided by another fish user for writing fish scripts: "you're using it wrong" was the sentiment. The argument was it's a shell intended to be interactive, but extendable. I don't know how pervasive this line of thinking is, but who's going to learn all about their shell if they're not supposed to write scripts in its language?

aerique 9y ago
I've been using fish for years but I never write scripts in it. What's the point? There's sh which is the most portable. Why write scripts in anything else but an actual programming language when things get more complex?
- msla 9y ago
  The point is the compose-save-edit cycle: Compose the script in the shell, save it to a file, and then, maybe, edit it to add better argument handling or support for corner cases you didn't consider in the interactive environment.

oblio 9y ago

Totally off topic: great blog. I was reading the archives these days, they’re awesome ;)

chubot 9y ago
Thanks! Feel free to comment (with the reddit links at the bottom of each post)
- oblio 9y ago
  My two main comments would be: 1. Keep up the great work (both oil and the blog!) Both seem fun :) 2. Reading your blog really shows that despite the great ideas, shell languages come from an age where computing was still in its infancy. We’re now approaching our teen years, but we’re still ahead of infants. So many awkward design decisions and organic evolutions. A shame that replacing POSIX at this point is a gargantuan undertaking.

arghwhat 9y ago

Back when I started programming, I wrote 1000+ lines of shell scripts. Now, I quite seriously believe that a 1000 line shell script only exists out of error. I still occasionally end up doing 2-300 line dense shell scripts, but not without feeling very dirty along the way. Either split into small, simple shell scripts (which is fine), or a different language. In the cross platform build pipeline at work, I keep a strong discipline when it comes to scripts: They must be short (<=100 lines), and if their complexity exceeds a certain threshold (parsing more than a "grep | cut" here and there, total program states exceeding some low number), then a shell script is no longer acceptable regardless of length. And, well, it's not safe to assume the presence of anything more than a shell. If you are writing and dealing with 1000+ lines of shell scripts, then experience tells me that you are shooting yourself in the foot. With a gatling gun. (I used fish, btw. The interactive experience was nice, but the syntax just felt different without much gain, which was frustrating to someone who often writes inline oneliners. Unlearning bash-isms is not a liberty I can afford, as I need to be proficient when I SSH into a machine I do not own or control. I can't force the entire company to install fish on all our lab servers, nor is it okay to install a shell on another persons' dev machine just because we need to coorperate.)

kakarot 9y ago
If your script is over 80-120 lines, it's time for modularization. No sane person writes 1000 line scripts, and the fact that the Oil author uses such a scenario as a reason to use their shell is very disconcerting about the quality of code and I wouldn't touch Oil with a ten-foot pole.
- kelnos 9y ago
  If that's the case, then there are a lot of insane people out there. I sympathize with your opinion on how things should be (though I may or may not agree), but at the end of the day we need to look at what the reality of the world is, not on the ideal way we think people should be using the tools we already have.
  - kakarot 9y ago

StreamBright 9y ago

One question though: Do you really need 1000+ lines of shell scripts? I almost have 20 years with shell scripting and I still have the rule of not writing longer than 1-2 page shell scripts and use something else for longer projects or break it down to much smaller chunks and invoke the parts with a main.sh. Fish is a for making software engineering and system administration easier by having functionality like command completion (without tab) and it works like a charm. I would never ever would nother to rewrite my bash scripts in fish simply because bash is on all the servers and fish is on none by default. I am not sure what you are trying to solve here but good luck.

deleted 9y ago
[deleted]
chubot 9y ago
See https://news.ycombinator.com/item?id=16278141 https://news.ycombinator.com/item?id=16278141
- StreamBright 9y ago
  This is not an answer just a problem statement.

greglindahl 9y ago

I've never met a devops person who wanted to write a 1,000+ line shell script. There are real languages that you can use instead.

jbreiding 9y ago

In the spaces you mention not hearing about fish are ones where you generally don't look for interactive applications/features. You also generally don't take a dependency like another shell when you are likely installing ruby or python already. The two shells target different use cases. Hopefully they are both successful in attracting a good audience.