7 ms·
Someone should push a patch to upstream bash that will make any script longer than 50 lines abort. If you're building something that takes 2500 lines of shell s
by detuur 9y ago
Someone should push a patch to upstream bash that will make any script longer than 50 lines abort. If you're building something that takes 2500 lines of shell script, you're using the wrong tool for the job. You're trying to make a program, but in a syntax (I was going to say language, but shell scripts aren't a programming language) that is utterly unsuited for anything like it.
A shell script has a very simple function: execute a number of tedious but common commands. That's it. Unfortunately someone decided they wanted control flow, so they wrote the horrifying [ and [[ programs, which I still consider to be a crime against humanity. Enter decades of write-once-read-never scripts that run with abysmal efficiency, kill maintainability, and neuter innovation. And somehow the entire Linux ecosystem depends on it and now every distro has to ship a shell with cruft dating back to 1970, with probably as many hidden vulnerabilities. And now every shell that tries to position itself on the market has to implement the same quirks, bugs, and design flaws in order to even be considered.
Let shells be shells, let shell scripts be dumb as bricks, and please use python or even perl for the love of god if you're going to make a build system.
- xellisx 9y agoOne could argue that HTML is a programming language. https://www.youtube.com/watch?v=4A2mWqLUpzw https://www.youtube.com/watch?v=4A2mWqLUpzw
- tacon 9y agoPeter van Roy puts XML and S-expressions in the "descriptive declarative programming" paradigm, with data structures only (no Turing equivalence): https://www.info.ucl.ac.be/~pvr/paradigmsDIAGRAMeng108.jpg https://www.info.ucl.ac.be/~pvr/paradigmsDIAGRAMeng108.jpg
- chubot 9y agoI see this sentiment a lot, but it's not a useful one, as that ship has already sailed. Please read this comment: https://www.reddit.com/r/linux/comments/7lsajn/oil_shell_03_release/dromj6n/ https://www.reddit.com/r/linux/comments/7lsajn/oil_shell_03_... It's not a small amount of shell, and it's not even old in some cases. People are still writing big pieces of shell code. I point out at the end of this comment that Kubernetes, a brand new cluster manager from Google (2014 or so), has 48,000+ lines of shell in its repo. Shell is definitely a terrible language for many things, but that's what I'm trying to fix, in a way that doesn't force you to rewrite all your code. Some things will never be rewritten -- it's like saying "Hey why don't you rewrite Wikipedia in Python and get rid of all the PHP?" It will never happen, for fundamental reasons of economics. Also see this comment for more of the "why": https://www.reddit.com/r/commandline/comments/7c3f9f/osh_02_parsing_one_million_lines_of_shell/dposm7r/ https://www.reddit.com/r/commandline/comments/7c3f9f/osh_02_... I plan to write these up on the blog, as mentioned at the end of this post.
- bch 9y ago> I see this sentiment a lot, but it’s not a useful one... Consider this response for entertainment purposes only. It’s easy to become insult(ed|ing) or get into a holy war, but I’m really not invested in any of that. That said: it is a useful sentiment, and a kubernetes script written in sh(1) is not the answer to the question. That one “can” does not mean one “should”. The reddit comment cites init scripts (I happen to use RC scripts, and haven’t viewed init scripts in a while) as an example of why shell is so great - everybody uses init! 1) of course everybody does. And it -happens- to be written in shell. So what? 2) this is fine! The scripts I see are 10s of lines long and practically flow top->bottom w maybe one or two conditionals. This is more akin to a job control language. A perfect application for sh(1). I just feel it (and you can too): sh(1) (or bash - nobody cares) features feel archaic and brittle. People have written impressive work in brainfuck[0][1], or C that just emits mov[2] instructions; is that a case for brainfuck projects or mov-only assembly? No. It’s just interesting. And kubernetes scaffolding or anything else big that happens (by some miracle) to work are not excuses for huge shell scripts. That’s just tautology - otherwise we’d be talking about the virtue of these scripts. For a boot sequence (rc/init) even if we ignore that the short scripts are actually suitable for the important job, I’d say their other grace is that sh(1) is demanded by POSIX, so you don’t need to wonder if it’s available in this nascent state of booting your OS - you’re guaranteed. The same couldn’t be said for ruby or python. From the “see this comment” comment: > Have you ever written a bash completion script yourself? I did a few years ago and I am scared to ever touch it again! I have a non-deterministic bug that I can't figure out, so I just restart my terminal every time completion gets borked. And I know bash very well. The comment admits what a warzone the space is... Sure, fix it, but we don’t need to promote it. The sentiment of discouraging huge shell scripts the GP expressed is useful. Resist. [0] https://en.wikipedia.org/wiki/Brainfuck https://en.wikipedia.org/wiki/Brainfuck [1] https://rosettacode.org/wiki/Category:Brainf*** https://rosettacode.org/wiki/Category:Brainf*** [2] https://youtu.be/R7EEoWg6Ekk https://youtu.be/R7EEoWg6Ekk
- effie 9y ago> sh(1) (or bash - nobody cares) features feel archaic and brittle Maybe some shell features are not that great, but in general I feel the opposite. Many of the shell features are great and lacking in other, more general-purpose languages - for example, program composition via pipes, stdout/stderr redirection syntax, direct access to the filesystem, interchangeability of a function and a program call, and others. When somebody decides to write a big system, like Kubernetes, they better be pragmatic rather than idealistic. That means choosing the right tool for the job, even if this tool has some deficiences or is considered quirky by some people. Manipulating VMs,containers on localhost or remote hosts, interacting with the OS and the filesystem, all on a unix-like system, it seems to me like shell is the best tool there is for most of the task.
- LeoPanthera 9y ago> shell scripts aren't a programming language Please explain why. Because simple observation seems to prove the opposite.
- nerdponx 9y agoTell that to the Zsh community. Or the community for any embedded scripting language -- Emacs Lisp, Vimscript, etc. Zsh, especially with 'set -eu', is a perfectly suitable, albeit slow, replacement for Perl, which is in my opinion more distasteful.
- majewsky 9y agoAnother example: Arch Linux is an entire distribution developed (at least 90%) in shell scripts. For example, the installer [1] is three shell scripts of 300-400 lines each. Packages are built with `makepkg`, which is a 2400-line shell script. It helps that the Arch developers are good at writing clean shell scripts. [1] In the package "arch-install-scripts".
- chubot 9y agoYes, Alpine's APKBUILD seems to be very much modelled after PKGBUILD, and "abuild" sounds analogous to "makepkg". It's also 100% shell, but it uses busybox ash rather than bash. I prefer pure hand-written shell to an unholy mix of auto-generated shell and Make. The latter is what I remember Debian's build system to be, although admittedly I haven't looked at it in awhile.
- jhasse 9y agoAlpine's apk-build was also written in Shell, but rewritten in C: https://github.com/alpinelinux/apk-tools https://github.com/alpinelinux/apk-tools I wonder what the reason was.
- chubot 9y agoThere is "abuild" to build packages, and "apk" for users to install packages. abuild is definitely still in shell (~2600 lines). I'm not sure if apk used to be in shell, but I wouldn't be surprised if it were. If you have a link to the old version I'm interested. I think the main reason not to use shell for the user side is the version solver. To install packages, you have to solve dependency constraints, which is actually an NP-complete problem! Those heuristics are best coded in C. In contrast, the build side doesn't have to do anything like that, or it can just shell out to "apk" if it does.
- effie 9y agoSo much arrogance and negativity, so little evidence for the claims you make. > A shell script has a very simple function: execute a number of tedious but common commands. That's it. And why is that? What will happen to me if I write more complicated shell scripts? The functionality is there and one can write very expressive and readable programs in shell. I would definitely recommend shell for writing long programs (for example various system administration and web backend tasks which do not require complex libraries). Preferably bash since it is the most common one. > And somehow the entire Linux ecosystem depends on it and now every distro has to ship a shell with cruft dating back to 1970 The 'somehow' has quite a simple rationale, it isn't something that should outrage you. The family of Linux-based OSes take inspiration and compatibility target in unix systems on purpose. Unix systems are primarily operated via shell. The current standard of shell may be an old idea from 70's with some 'cruft', but what language does not have cruft? The shell has proven to be usable and worthy of continued use. The compatibility with the unix shell is a big part of the early success of GNU/Linux and majority of users need it. > And now every shell that tries to position itself on the market has to implement the same quirks, bugs, and design flaws in order to even be considered. I have no idea what you are talking about. > please use python or even perl for the love of god if you're going to make a build system OK, so you think perl and python are better for making a build system. Have you made one?
- paulie_a 9y ago> What will happen to me if I write more complicated shell scripts? As part of my job currently, I am going to delete it. The same goes for poor python code, but I will take bad python over shell scripts any day of the week.
- AlexCoventry 9y ago> As part of my job currently, I am going to delete it. Go away, or I will turn you into a very small shell script. #!/bin/bash find / -name \*.sh -size +10k -type f -print0 | xargs -0 rm
- asfdsfggtfd 9y ago
- cozzyd 9y agoThat would have the nice side effect of killing the systemd/init debate. (Disclaimer: I was foolish enough to write a continuous integration script in bash).
- flukus 9y agoWhat's wrong with a continuous integration script in bash? A basic CI script just has to check source control periodically, run the build (make or something similar) and notify anyone if there's an error and maybe publish the result somewhere. I've done similar with the far less capable batch files. Sure it's not as great as a full on CI server, but it doesn't require the same investment either. For my personal projects I have bash scripts to handle basic CI (compiling and testing), given a bit of time I think I could string cron, make, bash and m4 into a CI server that fits 90%+ of my needs.
- cozzyd 9y agoOh it was great in the sense that it was very quick to write and it works... but it is not very easy to graft on new features.
- chubot 9y agoFWIW I wrote a post that touched on that debate last year: http://www.oilshell.org/blog/2017/01/13.html http://www.oilshell.org/blog/2017/01/13.html (I didn't come down on either side of the debate -- it's more of an exploration.) Comments: https://news.ycombinator.com/item?id=13477842 https://news.ycombinator.com/item?id=13477842
- arca_vorago 9y agoAs a sysadmin you can pry the ba(sh) scripts out of my cold dead hands. I'm not a dev, I'm not really a programmer, I'm the one who has to support all the crazy shit you devs are throwing at the wall to see what sticks. Bash is how I get shit done. I'm just tired of bash getting bashed everytime someone mentions shell scripting, based on what mostly seems to be bandwagon reasoning. Of course if you are writing more than $arbitrarycomplexity drop into perl, etc, but there is so much that can be done quickly, easily, and readably even for a non-programmer like myself that I think there are many benefits to using shell scripts that get far too often ignored. Also, a perfect example of a great bash script that's over 2k loc, and I've got plenty more were that came from: https://github.com/centminmod/centminmod/blob/master/centmin.sh https://github.com/centminmod/centminmod/blob/master/centmin...
- agumonkey 9y agoDon't take this bad but when you're not a programmer it's hard to understand how 2k LoC are probably 50 LoC done wrong. It's a self fulfilling prophecy, since programmers spends their day finding abstraction to turn long and windy into concise, but for the rest of the world we're just lunatics. Now I'm not as pushy as OP, but it's far from baseless bandwagon.
- majewsky 9y ago> it's hard to understand how 2k LoC are probably 50 LoC done wrong I challenge you to reimplement the shell script that grandparent linked in 50 LOC. In fact, shell script has the chance to be a lot more succinct than other languages for a lot of administrative tasks since the invocation of other programs is a first-class citizen. For example, compare this Go snippet: cmd := exec.Command("apt", append([]string{"install"}, dependencies...)) cmd.Stdin = os.Stdin cmd.Stdout = os.Stdout cmd.Stderr = os.Stderr err := cmd.Run() if err != nil { fmt.Fprintln(os.Stderr, err.Error()) os.Exit(1) } to its equivalent in shell script: if ! apt install ${DEPENDENCIES[@]}; then echo "exit status: $!" >&2 exit 1 fi Or even just "apt install ${DEPENDENCIES[@]}" with "set -e". Finally, what is it with sysadmins maintaining large scripts while at the same time quipping that they are not programmers? A script is very much a program, and a scripting language is very much a programming language.
- majewsky 9y ago> And now every shell that tries to position itself on the market has to implement the same quirks, bugs, and design flaws in order to even be considered. That's patently false: https://fishshell.com https://fishshell.com
- Chris2048 9y agofish is praised a lot, but how often is it used? https://github.com/fish-shell/fish-shell/wiki/POSIX-compatibility-thoughts https://github.com/fish-shell/fish-shell/wiki/POSIX-compatib...
- teekert 9y agoThat would be pretty hard to sell to the bioinformatics crowd. My Bash scripts are long, but easy to read. I'm just setting some environment variables, running some algorithms (many output to stdout and can be piped directly to other algorithms), moving the results around, write to a log here and there. Easily over 50 lines. Why would you force me into Python or what ever? I'm just gluing different programs together and moving files around.
- hzhou321 9y agoI have always been wondering why don't we have shell program that don't do scripts -- no flow control and fancy macros, and separate program that run scripts -- no readline or job control (while pipes and redirects are simply treated as syntax)?
- geocar 9y agoWe tried that. It was called the C-shell. Most people would use it on the command line and write scripts in Bourne shell, but some people kept trying to write scripts in C-shell for some incredibly stupid reason, and so it got banned.
- darrenf 9y agoIndeed, I have vivid memories of reading the chapter in Unix Power Tools devoted to this very topic: https://docstore.mik.ua/orelly/unix2.1/upt/ch47_01.htm https://docstore.mik.ua/orelly/unix2.1/upt/ch47_01.htm
- hzhou321 9y agoGood to learn! I always thought C-shell was aimed at scripting part (due to the sound of "C"); I guess I missed the boat.
- v_lisivka 9y agoExample script: #!/bin/bash trap '(( LINENO < 10+3 )) || { echo "Script executed more than 10 lines of code! Aborting."; exit 88; }' DEBUG echo 1 echo 2 echo 3 echo 4 echo 5 echo 6 echo 7 echo 8 echo 9 echo 10 echo 11 $ ./test.sh 1 2 3 4 5 6 7 8 9 10 Script executed more than 10 lines of code! Aborting.