13 ms·
Xonsh: I don't remember how to write a for loop in Bash [video]
- andrewstuart 4y agoWhenever I start writing bash, if it goes beyond 50 lines I rewrite it in Python. I really prefer to write no bash at all its so confusing.
- pas 4y agoI always yearn for compile time checking when writing scripts, because my experience is that these scripts become unmaintainable in about 3 hours. Recently I started using TS for scripting. Not the best, but Ammonite (a Scala scripting env) is a bit too big of a lift.
- MobiusHorizons 4y agoIf you want to understand bash, it’s best to spend some time using basic posix sh (dash on some systems) it is a much less powerful shell, but that highlights the power of composition that it does have. Also use shellcheck its an amazing linter, it will teach you many things about the shell, and actually explains the why for each lint.
- aaaaaaaaaaab 4y agoWhenever I encounter a 100+ line Python script, I rewrite it into a ~10 line bash script.
- ilyash 4y agoWhenever I encounter 20 lines bash script, I rewrite it as 15 lines Next Generation Shell script. It is more correct because of better error handling, more concise because of stdlib, more readable because of saner syntax. Disclosure: I'm the author.
- wdroz 4y agoIn the Google style guide for Shell[1], they use 100 lines as the limit. [1] -- https://google.github.io/styleguide/shellguide.html#when-to-use-shell https://google.github.io/styleguide/shellguide.html#when-to-...
- nokya 4y agoI have a shortcut in my browser bookmark bar just for this. I have been writing bash loops for more than 20 years, but only very sporadically (like 2 times per year). For some reason, I cannot remember this structure. P.s. never had this problem with PHP, Python, Java... P.s.2 I have this shortcut for batch (Windows) and PowerShell, too. :)
- tempxyz 4y agoWell bash does kind of have archaic syntax
- philipov 4y agoWas there ever a time when that kind of syntax was normal in other languages? Or is it just arcane syntax, that never had a time when other languages were built like it? Maybe the problem isn't that it's old, but just that it's bad.
- Kamq 4y agoBash is trying to mimic algol68 (poorly). Allegedly the guy who created bash had a bunch of C macros that made C more algol68-like.
- GranPC 4y agoYou might be thinking of bsh rather than bash: http://oldhome.schmorp.de/marc/bournegol.html http://oldhome.schmorp.de/marc/bournegol.html
- ncmncm 4y agoNo, that was sh. Steve Bourne. I met him once. #define IF if( #define THEN ){ #define ELSE }else{ #define ELSEIF }else if( #define END } #define WHILE while( #define FOR for( These are described as "non-syntactic macros" and are frowned on these days, but it was the Wild West in the early '70s.
- wisnoskij 4y agoI think it was just cobbled together. I think from the start the shell scripts were a way to arrange and call actual programs. If I remember correctly the first conditionals in shell scripting were literally just calls to the program named "test" I believe or something like that. This resulted in a language with little planning and thought put into it as it expanded to include more and more features. The biggest problem really seems to be that they wanted to avoid the use of esoteric syntax like double quotes and brackets, which resulted in a very unspecific syntax that is continually uncertain of if you are type strings or commands, so we have to use even more quotes and brackets and things of that nature. It really makes me wonder why no one seemed to be working on guiding Linux over to some JIT compiled C scripting like TempleOS uses.
- lexandstuff 4y agoConditional statements are even harder to remember. How many square brackets do I need? 1 or 2? Do I use == or -eq? Where does the semicolon go again?
- pas 4y ago[ is an alias for test, a very simple program [[ is the built-in this means you need to write everything separate, test doesn't parse, plus it needs the closing bracket if you use the alias but if-then works with any expression, and you put ; after expressions if test 1 -eq 2 ; then echo "oh noes" ; fi
- duped 4y agoSee that's even stupider than just having syntax for it
- cassepipe 4y agoI somehow already read that in the past but here, in this condensed form, it now actually makes sense. I think you just cured me of my bash fear. Thanks for that explanation. I'll make sure to either use test or [[ and never [ again so that I remember the difference. And I am currently reading man test
- pas 4y agoI think using test would be the best, even if it's awkward, but [[ is a Bashism (so not POSIX shell compatible), and [ is jut an alias for test and requires that dumb closing bracket, and you are already execve()-ing a separate program for comparison, you are already depending on the exit code, so why not make it explicit by using test. (I continue to use [[ and never look back, just forward, trying to picture the shining city on a conveniently sloped hill, where people live without fear in their hearts, where they have compile time checking for their scripts, and the compiler gently pushes them toward sane error handling. Not too much, but definitely not too little. Something like Rust's Result type.)
- sigjuice 4y ago
- pmarreck 4y agoGithub Copilot has largely solved the “Bash weirdness problem” for me personally, frankly. it’s also made it quite a bit more enjoyable to work with.
- xwowsersx 4y agoHah that sounds a decent way to throw together some bash scripts if, like me, you're not super fluent in bash syntax! Might actually have to give Copilot a try. Saw this vid: https://youtu.be/xqQCOigizw4 https://youtu.be/xqQCOigizw4
- bobmaxup 4y agoSounds akin to copying code from forum posts that show up as google results.
- pmarreck 4y agoIt's vastly superior to that. I can tell you haven't tried it.
- deleted 4y ago[deleted]
- maegul 4y agoSo Copilot is its own topic of discussion, but in so far as it and this sentiment are relevant to this thread, I think there's a serious issue here and one particularly relevant to that of Copilot in general. At some point, I think we have to protect the ability for a human to understand and reason about a technology or a piece of software. Thus the need for better and more economic languages etc. To sweep the the difficulties of reasoning about a language under the rug of a non-human AI tool that itself isn't providing human readable information on the software or language in question seems to be, quite frankly, an egregious shortcut for short-term gains. Whatever gains and utility there are in Copilot, bugs creeping in without the developer even understanding them is exactly what many would fear would happen with it. More broadly though, training the developer to be agnostic or ignorant of the issues with the very technology they're using because Copilot is "taking care of it" surely has some negative knock-on effects in terms of developers being able to manage and improve their tech-stacks.
- secondcoming 4y agobash is worse than perl. I usually avoid it with python whenever possible.
- AceJohnny2 4y agoI've hated Perl until I had to write significant Bash code. I still dislike Perl (it has pointers! pointers!), but its syntactic sugar for running shell commands and regexes makes shell-style scripting easier than Python.
- mr_toad 4y agoPerl has symbolic references and hard references (pointers). And if you have symbolic reference to a collection of pointers the syntax gets confusing really fast.
- wruza 4y agoMay I ask what seemingly horrible meaning do you ascribe to “pointers!” here? Perl references reference values like list, hash, scalar just like Python references lists, dicts, tuples, strings or numbers or any values through a hierarchy of these. The only difference is that in Python you can’t have a list/dict itself, only a reference to it. Perl doesn’t “have” references, it has actual container values instead, which are inaccessible in Python. E.g. @mylist is not an “object”, it is literally a collection of values that will splat into a location of its use. `func(@mylist)` will naturally take n arguments, where n == scalar @mylist. (Unless a prototype of func demands a reference.) Or maybe you’re mistaking it for pointer arithmetic?
- AceJohnny2 4y agoLet me get at this Socratically. What's the syntax to pass a hash to a function, and use it there? Syntax for putting a hash inside an array, or vice-versa? How do you get the keys of a hash that's inside an array?
- wruza 4y ago
- 0xbadcafebee 4y agoThe author can't remember how to do a for-loop in Bash, so he invented something that is Python, but also isn't Python, because straight Python doesn't work in the context of Ken Thompson's original command shell from 1971? ....just so the author doesn't have to look at StackOverflow, here is his Python example in the video: for i in range(5): if i: print(i) Here it is in POSIX Shell/Bourne Shell/Bash/whatever: for i in 1 2 3 4 5 do echo $i done The fancy version: for i in `seq 1 5` ; do if [ $i -gt 0 ] ; then echo $i fi done Fancier: for i in $(seq 1 5) ; do [ $i -gt 0 ] && echo $i done Mucho Fancy (Bash-only): while read -r NUM ; do [ "$NUM" -gt 0 ] && echo "$NUM" done < <(seq 1 5) Uber Fancy: Internal Seq w/Arrays & References (Bash v4+ only): _internal_seq () { declare -n aref="$1" local counter="$2" end="$3" while [ "$counter" -le "$end" ] ; do aref+=("$counter") counter="$((counter+1))" done } declare -a numbers=() _internal_seq numbers 1 5 for i in "${numbers[@]}" ; do echo "$i" done The version I actually use as a one-liner: for i in `seq 1 5` ; do echo $i ; done
- LanternLight83 4y agoYou can use brace expansion to avoid calling out to seq in a subshell (`{1..5}` expands to `1 2 3 4 5`)
- 0xbadcafebee 4y agoAwesome, thanks!! Just found the section of the man page, looks like a pretty feature-filled expansion technique: A sequence expression takes the form {x..y[..incr]}, where x and y are either integers or single characters, and incr, an optional increment, is an integer. When integers are sup- plied, the expression expands to each number between x and y, inclusive. Supplied integers may be prefixed with 0 to force each term to have the same width. When either x or y begins with a zero, the shell attempts to force all generated terms to contain the same number of digits, zero-padding where necessary. When characters are supplied, the expres- sion expands to each character lexicographically between x and y, inclusive, using the default C locale. Note that both x and y must be of the same type. When the increment is supplied, it is used as the difference between each term. The default increment is 1 or -1 as appropriate. ...although I just tested it, and the increment and 0-padding only work in non-interactive mode ='(
- loloquwowndueo 4y agoWow all the bash hate. It takes practice and repetition like with most programming languages. I have no problems writing for loops in bash because I do it all the time. Similarly if it’s something I don’t use frequently I will feel the syntax is arcane and unintuitive and have to look it up anyway. It’s fine to prefer one tool over the other but bashing the one you’re less familiar with is unnecessary :)
- HyperSane 4y ago"It takes practice and repetition like with most programming languages. " Except with Bash it really isn't worth it. Learning PowerShell is much better for long scripts.
- redox99 4y agoPowerShell might be better, but it won't be already installed in your typical Linux environment.
- supernovae 4y agodnf install powershell
- loloquwowndueo 4y agoYou can put that in a bash script because guess what - bash is already there ;)
- tolciho 4y ago$ bash /bin/ksh: bash: not found strike one... $ pkglocate powershell | grep \^power $ strike two...
- hulitu 4y agodnf: Bad command or file name :)
- PaulHoule 4y agoYou're not supposed to do it because it is terribly insecure but you can implement iteration in bash by writing a program that writes a bash script and piping into bash. For instance, ls | awk '{print "command " $1}' | bash Somehow the syntax to write loops in bash evades my memory (half because it is unconventional, half because I don't write them as often as I do in other languages) I know there is xargs which I am supposed to use instead of that above pipeline but I find myself having to read the whole man page every time I use it. As wrong as it, awk-bash pipelines are concise and easy to write.
- deleted 4y ago[deleted]
- ryapric 4y agoWeird that no one has yet to actually discuss `xonsh`, which the video is actually about. However, I'm about to do the same lol, and come out as a bash apologist against the sea of derision here. I've spent a decade writing code in maybe 10 different languages, and bash is still what I write the most of -- and its by choice! If you're writing a lot of code that calls other code, bash is kind of the default choice, since that's literally what the shell is for. Once you spend enough time with it, I actually find bash to be surprisingly elegant in what it affords the author in its terseness. Combined with other CLI utilities' own concision, it's an incredibly powerful environment. Sure it can start to appear arcane to unfamiliar folks, but you can work to make it significantly more readable than most people would have you believe. And with e.g. shellcheck, you can do away with a great deal of common footguns.
- arthurcolle 4y agoSorry for unrelated but I've been curious about this for the last few days! Does anyone know why when I have an alias like alias jc='python3.10 -m jupyter console' It still shows python 3.9 if I do print(sys.version)
- tra3 4y agoIs python3.10 an actual binary or a shim or another alias? which python3.10
- arthurcolle 4y agopython3.10 just points to python installation I manually stuck into a location. I think your sibling comment on my GP post is what I should look at next. Makes sense that the kernel needs to also be installed with the right version. I will check this, thank you and your sibling for the idea.
- jenny91 4y agoJupyter runs a "kernel" as a child process that actually runs the code in your notebooks. The python version that runs Jupyter can be/is different from the kernel python version.
- arthurcolle 4y agoThis makes sense, this would explain the difference.
- jyap 4y agoKinda strange and related but one of my first “wow cool” memories of working life was working at a company where we used Unix/Linux servers as workstations. Yeah big iron HP/DEC/Sun boxes. So anyway my co-worker did a for loop directly as a CLI command, probably using Tcsh at the time but it stuck with me. It was like a “wow you can just”: for F in `seq 1 10`; do echo $F done And it stuck with me because he used variable F and to this day that’s my go to variable when just blurting out a bash/zsh for loop at the CLI.
- ncmncm 4y agoYet, that is the wrong way. for I in {1..10}; do .... done
- Stratoscope 4y agoI worked at one company where our build engineer wrote 7500 lines of exceedingly complex Bash code to analyze our source tree and identify "code owners" - i.e. whoever had worked the most on each directory and was most familiar with it. I asked him about it, and he said, "It is a lot of code, but it does a lot. And if I'd written it in C++ it would have been a lot more code than that!" We were a Python shop.
- IncRnd 4y agoThat sounds like some poor programing in bash. The essence of the problem from what you wrote is something like: display the blame for all lines of all files per folder | sort by name | sum counts by name A single command-line can contain all the bash code, since the necessary tools are all available and can be placed together in a pipeline.
- Stratoscope 4y agoIn all fairness, the Bash code really did quite a bit more than that proverbial one-liner. But much of it wasn't calling and tying together those external tools, it was data analysis code that would have been so much simpler in Python. Plus, being a Python shop, any of us could have maintained a Python version, instead of the one engineer who was truly a Bash wizard. Some common sense and manual work might have been even better: just ask everyone in the company to identify the code areas they were familiar with and manually populate the CODEOWNERS files, instead of spending months on a mostly-failed attempt to automate the whole thing. Or, just teach people how to use git blame! After all, many of us touched many different parts of the code. Git blame would tell you immediately who was familiar with the different bits, at a much more granular level than a CODEOWNERS file in a directory.
- IncRnd 4y agoYou've pointed out several ways things could have been done better in that project. They are all clearly important and hard-won knowledge. But, I have found that one of those is super-duper-extremely important: know when not to program! You can just ask other SMEs (subject matter experts) for the answer!
- LanternLight83 4y agoI was a bash sympathizer until a post made the rounds about how, even if you `set -euxo pipefail`, `declare x="$(err)"` will silently eat the return code of the subshell. This applies to `local` too, and tbf is mentioned somewhat offhandedly on some man page, but was the straw that solidified my opinion. There's just no reasonable way that even an experienced user of Bash (ie. the kinds of folks who think that set command has them covered) could be expected to know about this footgun. The solution is to always declare and assign variables separately. Still use it all the time, and I guess "now I know", but, wtf Bash. Safety options are poorly bolted on.
- ryapric 4y agoI don't think you can capture/handle exit codes of subshells anyway, right? I've been operating under that assumption for years, and just don't do subshells if I can avoid them.
- 0xbadcafebee 4y ago$? contains the return status for subshells, commands, pipelines, functions, etc. From the man page, under Special Parameters : ? Expands to the exit status of the most recently executed foreground pipeline. $ foo="$(echo foobar ; exit 3)" ; ret=$? ; echo "output: $foo" ; echo "return status: $ret" output: foobar return status: 3 Also, PIPESTATUS is an array with the return status from each command in a pipeline.
- chubot 4y agoI agree, and that was out of the motivations for https://www.oilshell.org/ https://www.oilshell.org/ It runs your bash scripts but you can also opt into correct error handling. The simple invariant is that it doesn't lose an exit code, and non-zero is fatal by default. Oil 0.10.0 - Can Unix Shell Error Handling Be Fixed Once and For All? https://www.oilshell.org/blog/2022/05/release-0.10.0.html https://www.oilshell.org/blog/2022/05/release-0.10.0.html This took quite awhile, and I became aware of more error handling gotchas than I knew about when starting the project: https://www.oilshell.org/release/latest/doc/error-handling.html https://www.oilshell.org/release/latest/doc/error-handling.h... e.g. it's impossible in bash to even see the error status of a process sub like diff <(sort left.txt) <(sort OOPS) If you have bash scripts that you don't want to rewrite, try 1) run them with OSH 2) Add shopt --set oil:upgrade at the top to get the error handling fixes. Tell me what happens :) https://github.com/oilshell/oil https://github.com/oilshell/oil I spent a long time on that, but the post didn't get read much. I think it's because it takes a lot of bash experience to even understand what the problem is.
- SighMagi 4y agoIt’s got parsers for several versions of Python iirc. I’ve used it for a while, but somehow it feels slower than even bash.
- coopsmoss 4y agoI really want to use xonsh as my daily but everytime I try to do a git rebase it doesn't handle the inputs properly for my editor.
- maegul 4y agoHmmm … feels like a lot of missing the bigger point here. Xonsh, IMO, isn’t about hating bash or loving Python. It’s about making a new language (and shell) that tries to bring what’s good about a nice shell to Python so that when we hit our “this bash script is too big/complex and I need a better language” threshold, we can pick up Python (a reasonable choice) but retain some of the ergonomics of bash for calling and piping data to/from external programs, which Python simply doesn’t have (where subprocess + shlex + stdlib utils just don’t cut it IMO). From my minimal experience/playing around, xonsh seems quite nice and I find it a very believable future where it is a mainstream system scripting tool. Another thing to keep in mind is that AFAICT it is coming out of the researchy-sciency crowd where Python is mainstream and bash a necessary evil. Finding a way to make subprocess calls easy seems an obvious win in that context. Going even further, how nice would it be if something similar were part of the Python standard library? Being a wonderfully ergonomic shell script alternative seems like an instance where python could finally be the actual best at something.
- robertlagrant 4y agoBash, like Vogon spaceships, doesn't look as if it has been designed so much as congealed.